Version 4.2

Login: Handle suspicious login and JSON error
Motivated by #92.
2026-10-19 15:12:16 -07:00 · 2018-12-23 17:07:03 +01:00 · 2018-12-21 22:39:07 +01:00 · 2018-12-21 22:01:28 +01:00 · 2018-12-21 21:58:12 +01:00 · 2018-12-21 17:54:02 +01:00
30 changed files with 1408 additions and 284 deletions
@@ -0,0 +1,24 @@
 ---
 name: Bug report
 about: Create a report to help us improve
 ---
 **Describe the bug**
 A clear and concise description of what the bug is.
 **To Reproduce**
 Steps to reproduce the behavior:
 (e.g. Instaloader command line)
 **Expected behavior**
 A clear and concise description of what you expected to happen (if not obvious).
 **Error messages and tracebacks**
 If applicable, add error messages and tracebacks to help explain your problem.
 **Instaloader version**
 `instaloader --version` output.
 **Additional context**
 Add any other context about the problem here.
@@ -0,0 +1,17 @@
 ---
 name: Feature request
 about: Suggest an idea for this project
 ---
 **Provide us a use case of the feature**
 How could the user invoke the new function? Which problem would it solve? If new information is obtained, how would it be further processed?
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.
 **Describe alternatives you've considered**
 A clear and concise description of any alternative solutions or features you've considered.
 **Additional context**
 Add any other context about the feature request here.
@@ -8,7 +8,7 @@ install:
 - pip install pylint requests
 - pip install -r docs/requirements.txt
 script:
- - python3 -m pylint -r n -d bad-whitespace,missing-docstring,too-many-arguments,locally-disabled,line-too-long,too-many-public-methods,too-many-lines,too-many-instance-attributes,too-many-locals,too-many-branches,too-many-statements,inconsistent-return-statements,invalid-name,wildcard-import,unused-wildcard-import instaloader
+ - python3 -m pylint -r n -d bad-whitespace,missing-docstring,too-many-arguments,locally-disabled,line-too-long,too-many-public-methods,too-many-lines,too-many-instance-attributes,too-many-locals,too-many-branches,too-many-statements,inconsistent-return-statements,invalid-name,wildcard-import,unused-wildcard-import,no-else-return,cyclic-import,unnecessary-pass instaloader
 - make -C docs html SPHINXOPTS="-W -n"
 deploy:
  - provider: pypi
@@ -0,0 +1,8 @@
 Authors
 =======
 Instaloader is written by
 - Alexander Graf (@aandergr)
 - André Koch-Kramer (@Thammus)
 - Lars Lindqvist (@e5150)
@@ -21,7 +21,7 @@
 ::
-    instaloader [--comments] [--geotags] [--stories]
+    instaloader [--comments] [--geotags] [--stories] [--highlights] [--tagged]
                [--login YOUR-USERNAME] [--fast-update]
                profile | "#hashtag" | :stories | :feed | :saved
@@ -75,6 +75,24 @@ Disclaimer
 Instaloader is in no way affiliated with, authorized, maintained or endorsed by Instagram or any of its affiliates or
 subsidiaries. This is an independent and unofficial project. Use at your own risk.
 Instaloader is licensed under an MIT license. Refer to ``LICENSE`` file for more information.
 .. disclaimer-end
-Instaloader is licensed under an MIT license. Refer to ``LICENSE`` file for more information.
+Contributing
 ------------
 As an open source project, Instaloader heavily depends on the contributions from
 its community. See
 `contributing <https://instaloader.github.io/contributing.html>`__
 for how you may help Instaloader to become an even greater tool.
 It is a pleasure for us to share our Instaloader to the world, and we are proud
 to have attracted such an active and motivating community, with so many users
 who share their suggestions and ideas with us. Buying a community-sponsored beer
 or coffee from time to time is very likely to further raise our passion for the
 development of Instaloader.
 | For Donations, we provide a PayPal.Me link and a Bitcoin address.
 |  PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
 |  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
@@ -10,3 +10,19 @@ dd {
 dl.option dd {
    padding-left: 12px;
 }
 a {
    color: #008d06;
 }
 a:hover {
    color: #f48400;
 }
 body {
    font-size: 16px;
 }
 .versionmodified {
    font-size: 14px;
 }
@@ -50,6 +50,8 @@
                  <li><a href="https://github.com/instaloader/instaloader">Instaloader on GitHub</a></li>
                  <li role="separator" class="divider"></li>
                  <li style="padding-left: 20px;"><a class="github-button" href="https://github.com/instaloader/instaloader" data-icon="octicon-star" data-size="large" data-show-count="true" aria-label="Star Instaloader on GitHub">Star</a></li>
                  <li role="separator" class="divider"></li>
                  <li><a href="https://www.paypal.me/aandergr"><b>Donate (via Paypal)</b></a></li>
                </ul>
              </li>
            </ul>
@@ -97,6 +97,9 @@ metadata of a Profile. :class:`Profile` instances can be created with:
 A reference of the many methods provided by the :mod:`instaloader` module is
 provided in the remainder of this document.
 For a list of real code examples using the Instaloader module for advanced
 tasks, see :ref:`codesnippets`.
 ``Instaloader`` (Main Class)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -132,6 +135,17 @@ User Stories
 .. autoclass:: StoryItem
   :no-show-inheritance:
 Highlights
 """"""""""
 .. autoclass:: Highlight
   :no-show-inheritance:
   :inherited-members:
   Bases: :class:`Story`
   .. versionadded:: 4.1
 Profiles
 """"""""
@@ -174,6 +188,10 @@ Exceptions
 .. autoexception:: LoginRequiredException
 .. autoexception:: TwoFactorAuthRequiredException
   .. versionadded:: 4.2
 .. autoexception:: InvalidArgumentException
 .. autoexception:: BadResponseException
@@ -60,15 +60,37 @@ already have a valid session cookie file.
 What to Download
 ^^^^^^^^^^^^^^^^
 .. targets-start
 Instaloader supports the following targets:
 - ``profile``
-   Public profile, or private profile with :option:`--login`. For each profile
+   Public profile, or private profile with :option:`--login`.
-   you download, :option:`--stories` instructs Instaloader to also
+
-   **download the user's stories**.
+   If an already-downloaded profile has been renamed, Instaloader automatically
   finds it by its unique ID and renames the folder accordingly.
   Besides the profile's posts, its current profile picture is downloaded. For
   each profile you download,
   - :option:`--stories`
      instructs Instaloader to also **download the user's stories**,
   - :option:`--highlights`
      to **download highlights of each profile that is downloaded**, and
   - :option:`--tagged`
      to **download posts where the user is tagged**.
 - ``"#hashtag"``
-   Posts with a certain **hashtag** (the quotes are usually neccessary),
+   Posts with a certain **hashtag** (the quotes are usually necessary),
 - ``%location id``
   Posts tagged with a given location; the location ID is the numerical ID
   Instagram labels a location with (e.g.
   \https://www.instagram.com/explore/locations/**362629379**/plymouth-naval-memorial/).
   .. versionadded:: 4.2
 - ``:stories``
   The currently-visible **stories** of your followees (requires
@@ -84,6 +106,14 @@ Instaloader supports the following targets:
   All profiles that are followed by ``profile``, i.e. the *followees* of
   ``profile`` (requires :option:`--login`).
 - ``-post``
   The single **post** with the given shortcode. Must be preceeded by ``--`` in
   the argument list to not be mistaken as an option flag.
   .. versionadded:: 4.1
 .. targets-end
 Instaloader goes through all media matching the specified targets and
 downloads the pictures and videos and their captions. You can specify
@@ -103,7 +133,7 @@ Filename Specification
 ^^^^^^^^^^^^^^^^^^^^^^
 For each target, Instaloader creates a directory named after the target,
-i.e. ``profile``, ``#hashtag``, ``:feed``, etc. and therein saves the
+i.e. ``profile``, ``#hashtag``, ``%location id``, ``:feed``, etc. and therein saves the
 posts in files named after the post's timestamp.
 :option:`--dirname-pattern` allows to configure the directory name of each
@@ -158,7 +188,7 @@ Filter Posts
 .. py:currentmodule:: instaloader
 The options :option:`--post-filter` and :option:`--storyitem-filter`
-allows to specify criterias that posts or story items have to
+allows to specify criteria that posts or story items have to
 meet to be downloaded. If not given, all posts are downloaded.
 The filter string must be a
@@ -169,11 +199,6 @@ where the attributes from :class:`Post` or
 Id est, the following attributes can be used with both
 :option:`--post-filter` and :option:`--storyitem-filter`:
 - :attr:`~Post.is_video` (bool)
   Post/StoryItem is a video. For example, you may skip videos::
      instaloader --post-filter="not is_video" target
 - :attr:`~Post.owner_username` (str), :attr:`~Post.owner_id` (int)
   Owner profile username / userid.
@@ -183,6 +208,15 @@ Id est, the following attributes can be used with both
      instaloader --post-filter="date_utc <= datetime(2018, 5, 31)" target
 - :attr:`~Post.is_video` (bool)
   Post/StoryItem is a video. For example, you may skip videos::
      instaloader --post-filter="not is_video" target
   This is not the same as :option:`--no-videos` and
   :option:`--no-video-thumbnails`, since sidecar posts (posts that contain
   multiple pictures/videos in one post) have this attribute set to False.
 As :option:`--post-filter`, the following attributes can be used additionally:
 - :attr:`~Post.viewer_has_liked` (bool)
@@ -273,3 +307,6 @@ consider taking a look at the :ref:`python-module-instaloader`.
 Instaloader exposes its internally used methods and structures, making it a
 powerful and intuitive Python API for Instagram, allowing to further customize
 obtaining media and metadata.
 Also see :ref:`codesnippets`, where we collect example scripts that use
 Instaloader to achieve more complex tasks.
@@ -8,32 +8,46 @@ Instaloader is invoked with::
   $ instaloader [options] target [target ...]
 where ``target`` is a ``profile``, a ``"#hashtag"``, ``@profile`` (all profiles
-that *profile* is following), or if logged in ``:feed`` (pictures from your
+that *profile* is following), ``%location ID``, or if logged in ``:feed`` (pictures from your
 feed), ``:stories`` (stories of your followees) or ``:saved`` (collection of
 posts marked as saved).
 Here we explain the additional options that can be given to Instaloader to
-customize its behavior.  To get a list of all flags, their abbreviations and
+customize its behavior.  For an
 their descriptions, you may also run ``instaloader --help``.  For an
 introduction on how to use Instaloader, see
 :ref:`download-pictures-from-instagram`.
-What to Download
+To get a list of all flags, their abbreviations and
-^^^^^^^^^^^^^^^^
+their descriptions, you may also run::
-Specify a list of targets (profiles, #hashtags, ``:feed``, ``:stories`` or
+   instaloader --help
 ``:saved``). For each of these, Instaloader creates a folder and stores all
 posts along with the pictures's captions and the current **profile picture**
 there. If an already-downloaded profile has been renamed, Instaloader
 automatically **finds it by its unique ID** and renames the folder likewise.
-.. option:: --profile-pic-only, -P
+Targets
 ^^^^^^^
-   Only download profile picture.
+Specify a list of targets. For each of these, Instaloader creates a folder and
 stores all posts along with the pictures's captions there.
-.. option:: --no-profile-pic
+.. include:: basic-usage.rst
   :start-after: targets-start
   :end-before: targets-end
-   Do not download profile picture.
+- ``filename.json[.xz]``
   Re-Download the given object
 - ``+args.txt``
   Read targets (and options) from given textfile. See :option:`+args.txt`.
 What to Download of each Post
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 .. option:: --no-pictures
   Do not download post pictures. Cannot be used together with
   :option:`--fast-update`. Implies :option:`--no-video-thumbnails`, does not
   imply :option:`--no-videos`.
   .. versionadded:: 4.1
 .. option:: --no-videos, -V
@@ -69,11 +83,6 @@ automatically **finds it by its unique ID** and renames the folder likewise.
   Template to write in txt file for each StoryItem. See
   :ref:`metadata-text-files`.
 .. option:: --stories, -s
   Also **download stories** of each profile that is downloaded. Requires
   :option:`--login`.
 .. option:: --no-metadata-json
   Do not create a JSON file containing the metadata of each post.
@@ -82,8 +91,50 @@ automatically **finds it by its unique ID** and renames the folder likewise.
   Do not xz compress JSON files, rather create pretty formatted JSONs.
 What to Download of each Profile
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 .. option:: --profile-pic-only, -P
   .. deprecated:: 4.1
      Use :option:`--no-posts`.
   Only download profile picture.
 .. option:: --no-posts
   Do not download regular posts.
   .. versionadded:: 4.1
 .. option:: --no-profile-pic
   Do not download profile picture.
 .. option:: --stories, -s
   Also **download stories** of each profile that is downloaded. Requires
   :option:`--login`.
 .. option:: --highlights
   Also **download highlights** of each profile that is downloaded. Requires
   :option:`--login`.
   .. versionadded:: 4.1
 .. option:: --tagged
   Also download posts where each profile is tagged.
   .. versionadded:: 4.1
 .. option:: --stories-only
   .. deprecated:: 4.1
      Use :option:`--stories` :option:`--no-posts`.
   Rather than downloading regular posts of each specified profile, only
   download stories.  Requires :option:`--login`. Does not imply
   :option:`--no-profile-pic`.
@@ -93,6 +144,15 @@ automatically **finds it by its unique ID** and renames the folder likewise.
      If possible, use ``:stories`` target rather than :option:`--stories-only`
      with all your followees. ``:stories`` uses fewer API requests.
 Which Posts to Download
 ^^^^^^^^^^^^^^^^^^^^^^^
 .. option:: --fast-update, -F
   For each target, stop when encountering the first already-downloaded picture.
   This flag is recommended when you use Instaloader to update your personal
   Instagram archive.
 .. option:: --post-filter filter, --only-if filter
   Expression that, if given, must evaluate to True for each post to be
@@ -108,24 +168,10 @@ automatically **finds it by its unique ID** and renames the folder likewise.
   evaluated to :class:`instaloader.StoryItem` attributes.
   See :ref:`filter-posts` for more examples.
 When to Stop Downloading
 ^^^^^^^^^^^^^^^^^^^^^^^^
 If none of these options are given, Instaloader goes through all pictures
 matching the specified targets.
 .. option:: --fast-update, -F
   For each target, stop when encountering the first already-downloaded picture.
   This flag is recommended when you use Instaloader to update your personal
   Instagram archive.
 .. option:: --count COUNT, -c
   Do not attempt to download more than COUNT posts.  Applies only to
-   ``#hashtag`` and ``:feed``.
+   ``#hashtag``, ``%location id``, and ``:feed``.
 Login (Download Private Profiles)
@@ -184,6 +230,14 @@ How to Download
   to ``3``. If a connection fails, it can be manually skipped by hitting
   :kbd:`Control-c`. Set this to ``0`` to retry infinitely.
 .. option:: --commit-mode
   Tries to ensure downloaded images avoid corruption in case of unexpected
   interruption. If the last picture is corrupted, Instaloader will fix the
   picture the next time it is run.
   .. versionadded:: 4.2
 Miscellaneous Options
 ^^^^^^^^^^^^^^^^^^^^^
@@ -192,3 +246,21 @@ Miscellaneous Options
   Disable user interaction, i.e. do not print messages (except errors) and fail
   if login credentials are needed but not given.
   This is handy for running :ref:`instaloader-as-cronjob`.
 .. option:: +args.txt
   Read arguments from file `args.txt`, a shortcut to provide argument from
   file rather than command-line. This provide a convient way to hide login
   info from CLI. and also can use for simplify managment of long arguments.
   .. note::
      text file should separate arg with line break.
      args.txt example::
         --login MYUSENAME
         --password MYPASSWORD
         --fast-update
   .. versionadded:: 4.1
@@ -0,0 +1,131 @@
 .. _codesnippets:
 Advanced Instaloader Examples
 =============================
 .. currentmodule:: instaloader
 .. highlight:: python
 .. contents::
   :backlinks: none
 Here we present code examples that use the :ref:`python-module-instaloader` for
 more advanced Instagram downloading or metadata mining than what is possible
 with the Instaloader command line interface.
 The scripts presented here can be downloaded from our source tree:
 `instaloader/docs/codesnippets/ <https://github.com/instaloader/instaloader/tree/master/docs/codesnippets>`__
 .. For each code snippet:
   - title
   - brief description of what it does / motivation / how it works
   - code, or link to code
   - link to discussion issue
   - link used methods
 Download Posts in a Specific Period
 -----------------------------------
 To only download Instagram pictures (and metadata) that are within a specific
 period, you can play around with :func:`~itertools.dropwhile` and
 :func:`~itertools.takewhile` from :mod:`itertools` like in this snippet.
 .. literalinclude:: codesnippets/121_since_until.py
 See also :class:`Post`, :meth:`Instaloader.download_post`.
 Discussed in :issue:`121`.
 Likes of a Profile / Ghost Followers
 ------------------------------------
 To obtain a list of your inactive followers, i.e. followers that did not like
 any of your pictures, into a file you can use this approach.
 .. literalinclude:: codesnippets/120_ghost_followers.py
 See also :meth:`Profile.get_posts`, :meth:`Post.get_likes`,
 :meth:`Profile.get_followers`, :meth:`Instaloader.load_session_from_file`,
 :meth:`Profile.from_username`.
 Discussed in :issue:`120`.
 Track Deleted Posts
 -------------------
 This script uses Instaloader to obtain a list of currently-online Instagram and
 compares it with the set of posts that you already have downloaded.  It outputs
 a list of posts which are online but not offline (i.e. not yet downloaded) and a
 list of posts which are offline but not online (i.e. deleted in the profile).
 .. literalinclude:: codesnippets/56_track_deleted.py
 See also :func:`load_structure_from_file`, :meth:`Profile.from_username`,
 :meth:`Profile.get_posts`, :class:`Post`.
 Discussed in :issue:`56`.
 Only one Post per User
 ----------------------
 To download only the one most recent post from each user, this snippet creates a
 :class:`set` that contains the users of which a post has already been
 downloaded. While iterating the posts, it checks whether the post's owner
 already is in the set. If not, the post is downloaded from Instagram and the
 user is added to that set.
 .. literalinclude:: codesnippets/113_only_one_per_user.py
 See also :class:`Post`, :meth:`Instaloader.download_post`,
 :attr:`Post.owner_profile`, :class:`Profile`.
 Discussed in :issue:`113`.
 Upgrade Images by Local Copies
 ------------------------------
 The following script finds local versions of images fetched by Instaloader, in
 order to upgrade the downloaded images by locally-found versions with better
 quality. It uses image hashing to identify similar images.
 `updgrade-instaloader-images.py <https://gist.github.com/pavelkryukov/15f93d19a99428a284a8bcec27e0187b>`__ (external link to GitHub Gist)
 Discussed in :issue:`46`.
 Add Captions to Images
 ----------------------
 Instaloader does not modify the downloaded JPEG file. However, one could combine
 it with an imaging library such as Pillow or PIL to render the caption on
 Instagram pictures.  The following shows an approach.
 .. literalinclude:: codesnippets/110_pil_captions.py
 See also :attr:`Post.caption`, :attr:`Post.url`, :meth:`Post.from_shortcode`,
 :func:`load_structure_from_file`.
 Discussed in :issue:`110`.
 Metadata JSON Files
 -------------------
 The JSON files Instaloader saves along with each Post contain all the metadata
 that has been retrieved from Instagram while downloading the picture and
 associated required information.
 With `jq <https://stedolan.github.io/jq/>`__, a command-line JSON processor, the
 metadata can be easily post-processed. For example, Instaloader's JSON files can
 be pretty-formatted with:
 .. code-block:: none
   xzcat 2018-05-13_11-18-45_UTC.json.xz | jq .node
 However, Instaloader tries to do as few metadata requests as possible, so,
 depending on how Instaloader has been invoked, it may occur that these files do
 not contain the complete available metadata structure. Nevertheless, the file
 can be loaded into Instaloader with :func:`load_structure_from_file` and the
 required metadata then be accessed via the :class:`Post` or :class:`Profile`
 attributes, which trigger an Instagram request if that particular information is
 not present in the JSON file.
@@ -0,0 +1,20 @@
 from io import BytesIO
 from requests import get
 from PIL import Image, ImageDraw
 from instaloader import *
 L = Instaloader()
 # Load Post instance
 post = load_structure_from_file(L.context, '2017-10-01_18-53-03_UTC.json.xz')
 # or post = Post.from_shortcode(L.context, SHORTCODE)
 # Render caption
 image = Image.open(BytesIO(get(post.url).content))
 draw = ImageDraw.Draw(image)
 color = 'rgb(0, 0, 0)'  # black color
 draw.text((300,100), post.caption.encode('latin1', errors='ignore'), fill=color)
 # Save image
 image.save('test.jpg')
@@ -0,0 +1,14 @@
 import instaloader
 L = instaloader.Instaloader()
 posts = L.get_hashtag_posts('milfgarden')
 users = set()
 for post in posts:
    if not post.owner_profile in users:
        L.download_post(post, '#milfgarden')
        users.add(post.owner_profile)
    else:
        print("{} from {} skipped.".format(post, post.owner_profile))
@@ -0,0 +1,27 @@
 import instaloader
 L = instaloader.Instaloader()
 USER = 'your_account'
 PROFILE = USER
 # Your preferred way of logging in:
 L.load_session_from_file(USER)
 profile = instaloader.Profile.from_username(L.context, PROFILE)
 likes = set()
 print('Fetching likes of all posts of profile {}.'.format(profile.username))
 for post in profile.get_posts():
    print(post)
    likes = likes | set(post.get_likes())
 print('Fetching followers of profile {}.'.format(profile.username))
 followers = set(profile.get_followers())
 ghosts = followers - likes
 print('Storing ghosts into file.')
 with open('/YOUR PATH/inactive-users.txt', 'w') as f:
    for ghost in ghosts:
        print(ghost.username, file=f)
@@ -0,0 +1,17 @@
 from datetime import datetime
 from itertools import dropwhile, takewhile
 import instaloader
 L = instaloader.Instaloader()
 posts = L.get_hashtag_posts('milfgarden')
 # or
 # posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts()
 SINCE = datetime(2015, 5, 1)
 UNTIL = datetime(2015, 3, 1)
 for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
    print(post.date)
    L.download_post(post, '#milfgarden')
@@ -0,0 +1,34 @@
 from glob import glob
 from sys import argv
 from os import chdir
 from instaloader import Instaloader, Post, Profile, load_structure_from_file
 # Instaloader instantiation - you may pass additional arguments to the constructor here
 L = Instaloader()
 # If desired, load session previously saved with `instaloader -l USERNAME`:
 #L.load_session_from_file(USERNAME)
 try:
    TARGET = argv[1]
 except IndexError:
    raise SystemExit("Pass profile name as argument!")
 # Obtain set of posts that are on hard disk
 chdir(TARGET)
 offline_posts = set(filter(lambda s: isinstance(s, Post),
                           (load_structure_from_file(L.context, file)
                            for file in (glob('*.json.xz') + glob('*.json')))))
 # Obtain set of posts that are currently online
 post_iterator = Profile.from_username(L.context, TARGET).get_posts()
 online_posts = set(post_iterator)
 if online_posts - offline_posts:
    print("Not yet downloaded posts:")
    print(" ".join(str(p) for p in (online_posts - offline_posts)))
 if offline_posts - online_posts:
    print("Deleted posts:")
    print(" ".join(str(p) for p in (offline_posts - online_posts)))
@@ -0,0 +1,4 @@
 # Advanced Instaloader Examples
 This directory contains the scripts that are presented at:
 https://instaloader.github.io/codesnippets.html
@@ -20,6 +20,7 @@
 import os
 import subprocess
 import sys
 import typing
 import sphinx_bootstrap_theme
@@ -39,7 +40,8 @@ import docs.sphinx_autodoc_typehints as sphinx_autodoc_typehints
 extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.githubpages',
-    'sphinx.ext.intersphinx'
+    'sphinx.ext.intersphinx',
    'sphinx.ext.extlinks'
 ]
 autodoc_default_flags = ['show-inheritance', 'members', 'undoc-members']
@@ -49,6 +51,8 @@ autodoc_member_order = 'bysource'
 intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
                       'requests': ('http://docs.python-requests.org/en/master', None)}
 extlinks = {'issue': ('https://github.com/instaloader/instaloader/issues/%s', 'issue #')}
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -371,6 +375,7 @@ html_context = {'current_release': current_release, 'current_release_date': curr
 def setup(app):
    typing.TYPE_CHECKING = True
    app.add_stylesheet("style.css")
    app.connect('autodoc-process-signature', sphinx_autodoc_typehints.process_signature)
    app.connect('autodoc-process-docstring', sphinx_autodoc_typehints.process_docstring)
@@ -1,3 +1,5 @@
 .. _contributing:
 Contributing to Instaloader
 ===========================
@@ -15,7 +17,7 @@ Reporting Bugs
 If you encounter a bug, do not hesitate to report it in our
 `Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. Since
 Instaloader is actively developed, the majority of bugs is fixed within only
-**4 days** after being reported. When reporting a problem, please keep the
+**3 days** after being reported. When reporting a problem, please keep the
 following in mind:
 - Ensure you **use the latest version** of Instaloader. The currently-installed
@@ -23,10 +25,10 @@ following in mind:
 - Briefly **check whether the bug has already been reported**. If you find an
  issue reporting the same bug you encountered, comment there rather than
-  opening a new issue. However, if unsure, it is advised to create a new issue.
+  opening a new issue. However, if unsure, please create a new issue.
 - State **how the bug can be reproduced**, i.e. how Instaloader was invoked
-  when the problem occured (of course, you may anonymize profile names etc.).
+  when the problem occurred (of course, you may anonymize profile names etc.).
 - Include all **error messages and tracebacks** in the report.
@@ -47,13 +49,13 @@ few things to consider:
  you forked from the most recent branch.
 - We use `Pylint <https://www.pylint.org/>`__ for error and syntax checking of
-  ``instaloader.py``. The file ``.travis.yml`` in the project's root directory
+  the source. The file ``.travis.yml`` in the project's root directory
  shows how it is invoked. Note that sometimes it might be better to disable a
  warning rather than adapting the code to Pylint's desires.
 - The documentation source is located in the ``docs`` folder. The file
-  ``cli-options.rst`` is merely an RST-formatted copy of ``ìnstaloader --help``
+  ``cli-options.rst`` is merely an RST-formatted copy of ``instaloader --help``
-  output, of which the source is near the bottom of ``instaloader.py``.
+  output, of which the source is in ``instaloader/__main__.py``.
 - Feel free to contact us, even if you "only" have Proof-of-Concepts or
  not-fully integrated changes. They already might be an advance for the
@@ -81,3 +83,22 @@ Please consider the following:
 - **Be patient**. Naturally, bugs and pull requests have a higher priority than
  feature suggestions. Keep in mind that this is a free software project, and
  unfortunately we only have limited time to work on it.
 Donations
 ---------
 .. donations-start
 It is a pleasure for us to share our Instaloader to the world, and we are proud
 to have attracted such an active and motivating community, with so many users
 who share their suggestions and ideas with us. Buying a community-sponsored beer
 or coffee from time to time is very likely to further raise our passion for the
 development of Instaloader.
 | For donations, we provide a PayPal.Me link and a Bitcoin address.
 |  `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
 |  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
 .. donations-end
 .. (Discussion in :issue:`130`)
@@ -41,9 +41,9 @@ See :ref:`install` for more options on how to install Instaloader.
 ::
-    instaloader [--comments] [--geotags] [--stories]
+    instaloader [--comments] [--geotags] [--stories] [--highlights] [--tagged]
                [--login YOUR-USERNAME] [--fast-update]
-                profile | "#hashtag" |
+                profile | "#hashtag" | %location_id |
                :stories | :feed | :saved
 See :ref:`download-pictures-from-instagram` for a detailed introduction on how
@@ -59,6 +59,7 @@ Instaloader Documentation
   basic-usage
   cli-options
   as-module
   codesnippets
   contributing
 Useful Links
@@ -69,10 +70,23 @@ Useful Links
 - `Issue Tracker / Bug Tracker <https://github.com/instaloader/instaloader/issues>`__
 - `Version History <https://github.com/instaloader/instaloader/releases>`__
 Disclaimer
 ----------
 .. include:: ../README.rst
   :start-after: disclaimer-start
   :end-before: disclaimer-end
 Contributing
 ------------
 As an open source project, Instaloader heavily depends on the contributions from
 its community. See :ref:`contributing` for how you may help Instaloader to
 become an even greater tool.
 .. include:: contributing.rst
   :start-after: donations-start
   :end-before: donations-end
 ..
   * :ref:`genindex`
@@ -33,7 +33,6 @@ Install Instaloader
 - On **Windows 10**, you may download the standalone executable from the
  `current release page <https://github.com/instaloader/instaloader/releases/latest>`__.
- To get the most current version of Instaloader from our
+- To test the most current pre-release or development version of Instaloader::
  `Git repository <https://github.com/instaloader/instaloader>`__::
-     pip3 install --upgrade git+https://github.com/instaloader/instaloader
+     pip3 install --pre instaloader
@@ -1,3 +1,3 @@
 requests
-sphinx
+sphinx!=1.7.7
 sphinx-bootstrap-theme
@@ -1,4 +1,4 @@
-# sphinx_autodoc_typehints from Mar 18 2018.
+# sphinx_autodoc_typehints from Nov 20 2018 (version 1.5.2).
 #  https://github.com/agronholm/sphinx-autodoc-typehints
 # (slightly modified to fix class links, maybe related to agronholm/sphinx-autodoc-typehints#38)
 #
@@ -27,8 +27,8 @@
 import inspect
 from typing import get_type_hints, TypeVar, Any, AnyStr, Generic, Union
-from sphinx.util.inspect import getargspec
+from sphinx.util import logging
-from sphinx.ext.autodoc import formatargspec
+from sphinx.util.inspect import Signature
 try:
    from inspect import unwrap
@@ -51,6 +51,8 @@ except ImportError:
            memo.add(id_func)
        return func
 logger = logging.getLogger(__name__)
 def format_annotation(annotation):
    if inspect.isclass(annotation) and annotation.__module__ == 'builtins':
@@ -162,23 +164,37 @@ def process_signature(app, what: str, name: str, obj, options, signature, return
        return
    obj = unwrap(obj)
-    try:
+    signature = Signature(obj)
-        argspec = getargspec(obj)
+    parameters = [
-    except (TypeError, ValueError):
+        param.replace(annotation=inspect.Parameter.empty)
-        return
+        for param in signature.signature.parameters.values()
    ]
-    if argspec.args:
+    if parameters:
        if what in ('class', 'exception'):
-            del argspec.args[0]
+            del parameters[0]
        elif what == 'method':
            outer = inspect.getmodule(obj)
            for clsname in obj.__qualname__.split('.')[:-1]:
                outer = getattr(outer, clsname)
            method_object = outer.__dict__[obj.__name__]
            if not isinstance(method_object, (classmethod, staticmethod)):
                del argspec.args[0]
-    return formatargspec(obj, *argspec[:-1]), None
+            method_name = obj.__name__
            if method_name.startswith("__") and not method_name.endswith("__"):
                # If the method starts with double underscore (dunder)
                # Python applies mangling so we need to prepend the class name.
                # This doesn't happen if it always ends with double underscore.
                class_name = obj.__qualname__.split('.')[-2]
                method_name = "_{c}{m}".format(c=class_name, m=method_name)
            method_object = outer.__dict__[method_name]
            if not isinstance(method_object, (classmethod, staticmethod)):
                del parameters[0]
    signature.signature = signature.signature.replace(
        parameters=parameters,
        return_annotation=inspect.Signature.empty)
    return signature.format_args().replace('\\', '\\\\'), None
 def process_docstring(app, what, name, obj, options, lines):
@@ -195,8 +211,15 @@ def process_docstring(app, what, name, obj, options, lines):
        except (AttributeError, TypeError):
            # Introspecting a slot wrapper will raise TypeError
            return
        except NameError as exc:
            logger.warning('Cannot resolve forward reference in type annotations of "%s": %s',
                           name, exc)
            type_hints = obj.__annotations__
        for argname, annotation in type_hints.items():
            if argname.endswith('_'):
                argname = '{}\\_'.format(argname[:-1])
            formatted_annotation = format_annotation(annotation)
            if argname == 'return':
@@ -1,7 +1,7 @@
 """Download pictures (or videos) along with their captions and other metadata from Instagram."""
-__version__ = '4.0.2'
+__version__ = '4.2'
 try:
@@ -15,5 +15,5 @@ else:
 from .exceptions import *
 from .instaloader import Instaloader
 from .instaloadercontext import InstaloaderContext
-from .structures import (Post, PostSidecarNode, PostComment, PostLocation, Profile, Story, StoryItem,
+from .structures import (Highlight, Post, PostSidecarNode, PostComment, PostLocation, Profile, Story, StoryItem,
                         load_structure_from_file, save_structure_to_file)
@@ -8,7 +8,8 @@ from argparse import ArgumentParser, SUPPRESS
 from typing import List, Optional
 from . import (Instaloader, InstaloaderException, InvalidArgumentException, Post, Profile, ProfileNotExistsException,
-               StoryItem, __version__, load_structure_from_file)
+               StoryItem, __version__, load_structure_from_file, TwoFactorAuthRequiredException,
               BadCredentialsException)
 from .instaloader import get_default_session_filename
 from .instaloadercontext import default_user_agent
@@ -18,9 +19,9 @@ def usage_string():
    argv0 = os.path.basename(sys.argv[0])
    argv0 = "instaloader" if argv0 == "__main__.py" else argv0
    return """
-{0} [--comments] [--geotags] [--stories]
+{0} [--comments] [--geotags] [--stories] [--highlights] [--tagged]
 {2:{1}} [--login YOUR-USERNAME] [--fast-update]
-{2:{1}} profile | "#hashtag" | :stories | :feed | :saved
+{2:{1}} profile | "#hashtag" | %%location_id | :stories | :feed | :saved
 {0} --help""".format(argv0, len(argv0), '')
@@ -58,11 +59,11 @@ def filterstr_to_filterfunc(filter_str: str, item_type: type):
 def _main(instaloader: Instaloader, targetlist: List[str],
          username: Optional[str] = None, password: Optional[str] = None,
-          sessionfile: Optional[str] = None, max_count: Optional[int] = None,
+          sessionfile: Optional[str] = None,
-          profile_pic: bool = True, profile_pic_only: bool = False,
+          download_profile_pic: bool = True, download_posts=True,
          download_stories: bool = False, download_highlights: bool = False, download_tagged: bool = False,
          fast_update: bool = False,
-          stories: bool = False, stories_only: bool = False,
+          max_count: Optional[int] = None, post_filter_str: Optional[str] = None,
          post_filter_str: Optional[str] = None,
          storyitem_filter_str: Optional[str] = None) -> None:
    """Download set of profiles, hashtags etc. and handle logging in and session files if desired."""
    # Parse and generate filter function
@@ -84,12 +85,22 @@ def _main(instaloader: Instaloader, targetlist: List[str],
            instaloader.context.log("Session file does not exist yet - Logging in.")
        if not instaloader.context.is_logged_in or username != instaloader.test_login():
            if password is not None:
-                instaloader.login(username, password)
+                try:
                    instaloader.login(username, password)
                except TwoFactorAuthRequiredException:
                    while True:
                        try:
                            code = input("Enter 2FA verification code: ")
                            instaloader.two_factor_login(code)
                            break
                        except BadCredentialsException:
                            pass
            else:
                instaloader.interactive_login(username)
        instaloader.context.log("Logged in as %s." % username)
    # Try block for KeyboardInterrupt (save session on ^C)
    profiles = set()
    anonymous_retry_profiles = set()
    try:
        # Generate set of profiles, already downloading non-profile targets
        for target in targetlist:
@@ -121,11 +132,17 @@ def _main(instaloader: Instaloader, targetlist: List[str],
                if target[0] == '@':
                    instaloader.context.log("Retrieving followees of %s..." % target[1:])
                    profile = Profile.from_username(instaloader.context, target[1:])
-                    followees = profile.get_followees()
+                    for followee in profile.get_followees():
-                    profiles.update([followee.username for followee in followees])
+                        instaloader.save_profile_id(followee)
                        profiles.add(followee)
                elif target[0] == '#':
                    instaloader.download_hashtag(hashtag=target[1:], max_count=max_count, fast_update=fast_update,
                                                 post_filter=post_filter)
                elif target[0] == '-':
                    instaloader.download_post(Post.from_shortcode(instaloader.context, target[1:]), target)
                elif target[0] == "%":
                    instaloader.download_location(location=target[1:], max_count=max_count, fast_update=fast_update,
                                                  post_filter=post_filter)
                elif target == ":feed":
                    instaloader.download_feed_posts(fast_update=fast_update, max_count=max_count,
                                                    post_filter=post_filter)
@@ -135,26 +152,43 @@ def _main(instaloader: Instaloader, targetlist: List[str],
                    instaloader.download_saved_posts(fast_update=fast_update, max_count=max_count,
                                                     post_filter=post_filter)
                else:
-                    profiles.add(target)
+                    try:
                        profile = instaloader.check_profile_id(target)
                        if instaloader.context.is_logged_in and profile.has_blocked_viewer:
                            if download_profile_pic or ((download_posts or download_tagged) and not profile.is_private):
                                raise ProfileNotExistsException("{} blocked you; But we download her anonymously."
                                                                .format(target))
                            else:
                                instaloader.context.error("{} blocked you.".format(target))
                        else:
                            profiles.add(profile)
                    except ProfileNotExistsException as err:
                        # Not only our profile.has_blocked_viewer condition raises ProfileNotExistsException,
                        # check_profile_id() also does, since access to blocked profile may be responded with 404.
                        if instaloader.context.is_logged_in and (download_profile_pic or download_posts or
                                                                 download_tagged):
                            instaloader.context.log(err)
                            instaloader.context.log("Trying again anonymously, helps in case you are just blocked.")
                            with instaloader.anonymous_copy() as anonymous_loader:
                                with instaloader.context.error_catcher():
                                    anonymous_retry_profiles.add(anonymous_loader.check_profile_id(target))
                                    instaloader.context.error("Warning: {} will be downloaded anonymously (\"{}\")."
                                                              .format(target, err))
                        else:
                            raise
        if len(profiles) > 1:
-            instaloader.context.log("Downloading {} profiles: {}".format(len(profiles), ' '.join(profiles)))
+            instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
-        # Iterate through profiles list and download them
+                                                                         ' '.join([p.username for p in profiles])))
-        for target in profiles:
+        instaloader.download_profiles(profiles,
-            with instaloader.context.error_catcher(target):
+                                      download_profile_pic, download_posts, download_tagged, download_highlights,
-                try:
+                                      download_stories, fast_update, post_filter, storyitem_filter)
-                    instaloader.download_profile(target, profile_pic, profile_pic_only, fast_update,
+        if anonymous_retry_profiles:
-                                                 stories, stories_only, post_filter=post_filter,
+            instaloader.context.log("Downloading anonymously: {}"
-                                                 storyitem_filter=storyitem_filter)
+                                    .format(' '.join([p.username for p in anonymous_retry_profiles])))
-                except ProfileNotExistsException as err:
+            with instaloader.anonymous_copy() as anonymous_loader:
-                    if instaloader.context.is_logged_in:
+                anonymous_loader.download_profiles(anonymous_retry_profiles,
-                        instaloader.context.log(err)
+                                                   download_profile_pic, download_posts, download_tagged,
-                        instaloader.context.log("Trying again anonymously, helps in case you are just blocked.")
+                                                   fast_update=fast_update, post_filter=post_filter)
                        with instaloader.anonymous_copy() as anonymous_loader:
                            with instaloader.context.error_catcher():
                                anonymous_loader.download_profile(target, profile_pic, profile_pic_only,
                                                                  fast_update, post_filter=post_filter)
                    else:
                        raise
    except KeyboardInterrupt:
        print("\nInterrupted by user.", file=sys.stderr)
    # Save session if it is useful
@@ -174,72 +208,96 @@ def main():
    parser = ArgumentParser(description=__doc__, add_help=False, usage=usage_string(),
                            epilog="Report issues at https://github.com/instaloader/instaloader/issues. "
                                   "The complete documentation can be found at "
-                                   "https://instaloader.github.io/.")
+                                   "https://instaloader.github.io/.",
                            fromfile_prefix_chars='+')
-    g_what = parser.add_argument_group('What to Download',
+    g_targets = parser.add_argument_group("What to Download",
-                                       'Specify a list of profiles or #hashtags. For each of these, Instaloader '
+                                          "Specify a list of targets. For each of these, Instaloader creates a folder "
-                                       'creates a folder and '
+                                          "and downloads all posts. The following targets are supported:")
-                                       'downloads all posts along with the pictures\'s '
+    g_targets.add_argument('profile', nargs='*',
-                                       'captions and the current profile picture. '
+                           help="Download profile. If an already-downloaded profile has been renamed, Instaloader "
-                                       'If an already-downloaded profile has been renamed, Instaloader automatically '
+                                "automatically finds it by its unique ID and renames the folder likewise.")
-                                       'finds it by its unique ID and renames the folder likewise.')
+    g_targets.add_argument('_at_profile', nargs='*', metavar="@profile",
-    g_what.add_argument('profile', nargs='*', metavar='profile|#hashtag',
+                           help="Download all followees of profile. Requires --login. "
-                        help='Name of profile or #hashtag to download. '
+                                "Consider using :feed rather than @yourself.")
-                             'Alternatively, if --login is given: @<profile> to download all followees of '
+    g_targets.add_argument('_hashtag', nargs='*', metavar='"#hashtag"', help="Download #hashtag.")
-                             '<profile>; the special targets '
+    g_targets.add_argument('_location', nargs='*', metavar='%location_id', help="Download %%location_id.")
-                             ':feed to download pictures from your feed; '
+    g_targets.add_argument('_feed', nargs='*', metavar=":feed",
-                             ':stories to download the stories of your followees; or '
+                           help="Download pictures from your feed. Requires --login.")
-                             ':saved to download the posts marked as saved.')
+    g_targets.add_argument('_stories', nargs='*', metavar=":stories",
-    g_what.add_argument('-P', '--profile-pic-only', action='store_true',
+                           help="Download the stories of your followees. Requires --login.")
-                        help='Only download profile picture.')
+    g_targets.add_argument('_saved', nargs='*', metavar=":saved",
-    g_what.add_argument('--no-profile-pic', action='store_true',
+                           help="Download the posts that you marked as saved. Requires --login.")
    g_targets.add_argument('_singlepost', nargs='*', metavar="-- -shortcode",
                           help="Download the post with the given shortcode")
    g_targets.add_argument('_json', nargs='*', metavar="filename.json[.xz]",
                           help="Re-Download the given object.")
    g_targets.add_argument('_fromfile', nargs='*', metavar="+args.txt",
                           help="Read targets (and options) from given textfile.")
    g_post = parser.add_argument_group("What to Download of each Post")
    g_prof = parser.add_argument_group("What to Download of each Profile")
    g_prof.add_argument('-P', '--profile-pic-only', action='store_true',
                        help=SUPPRESS)
    g_prof.add_argument('--no-posts', action='store_true',
                        help="Do not download regular posts.")
    g_prof.add_argument('--no-profile-pic', action='store_true',
                        help='Do not download profile picture.')
-    g_what.add_argument('-V', '--no-videos', action='store_true',
+    g_post.add_argument('--no-pictures', action='store_true',
                        help='Do not download post pictures. Cannot be used together with --fast-update. '
                             'Implies --no-video-thumbnails, does not imply --no-videos.')
    g_post.add_argument('-V', '--no-videos', action='store_true',
                        help='Do not download videos.')
-    g_what.add_argument('--no-video-thumbnails', action='store_true',
+    g_post.add_argument('--no-video-thumbnails', action='store_true',
                        help='Do not download thumbnails of videos.')
-    g_what.add_argument('-G', '--geotags', action='store_true',
+    g_post.add_argument('-G', '--geotags', action='store_true',
                        help='Download geotags when available. Geotags are stored as a '
                             'text file with the location\'s name and a Google Maps link. '
                             'This requires an additional request to the Instagram '
                             'server for each picture, which is why it is disabled by default.')
-    g_what.add_argument('-C', '--comments', action='store_true',
+    g_post.add_argument('-C', '--comments', action='store_true',
                        help='Download and update comments for each post. '
                             'This requires an additional request to the Instagram '
                             'server for each post, which is why it is disabled by default.')
-    g_what.add_argument('--no-captions', action='store_true',
+    g_post.add_argument('--no-captions', action='store_true',
                        help='Do not create txt files.')
-    g_what.add_argument('--post-metadata-txt', action='append',
+    g_post.add_argument('--post-metadata-txt', action='append',
                        help='Template to write in txt file for each Post.')
-    g_what.add_argument('--storyitem-metadata-txt', action='append',
+    g_post.add_argument('--storyitem-metadata-txt', action='append',
                        help='Template to write in txt file for each StoryItem.')
-    g_what.add_argument('--no-metadata-json', action='store_true',
+    g_post.add_argument('--no-metadata-json', action='store_true',
                        help='Do not create a JSON file containing the metadata of each post.')
-    g_what.add_argument('--metadata-json', action='store_true',
+    g_post.add_argument('--metadata-json', action='store_true',
                        help=SUPPRESS)
-    g_what.add_argument('--no-compress-json', action='store_true',
+    g_post.add_argument('--no-compress-json', action='store_true',
                        help='Do not xz compress JSON files, rather create pretty formatted JSONs.')
-    g_what.add_argument('-s', '--stories', action='store_true',
+    g_prof.add_argument('-s', '--stories', action='store_true',
                        help='Also download stories of each profile that is downloaded. Requires --login.')
-    g_what.add_argument('--stories-only', action='store_true',
+    g_prof.add_argument('--stories-only', action='store_true',
-                        help='Rather than downloading regular posts of each specified profile, only download '
+                        help=SUPPRESS)
-                             'stories. Requires --login. Does not imply --no-profile-pic.')
+    g_prof.add_argument('--highlights', action='store_true',
-    g_what.add_argument('--post-filter', '--only-if', metavar='filter',
+                        help='Also download highlights of each profile that is downloaded. Requires --login.')
    g_prof.add_argument('--tagged', action='store_true',
                        help='Also download posts where each profile is tagged.')
    g_cond = parser.add_argument_group("Which Posts to Download")
    g_cond.add_argument('-F', '--fast-update', action='store_true',
                        help='For each target, stop when encountering the first already-downloaded picture. This '
                             'flag is recommended when you use Instaloader to update your personal Instagram archive.')
    g_cond.add_argument('--post-filter', '--only-if', metavar='filter',
                        help='Expression that, if given, must evaluate to True for each post to be downloaded. Must be '
                             'a syntactically valid python expression. Variables are evaluated to '
                             'instaloader.Post attributes. Example: --post-filter=viewer_has_liked.')
-    g_what.add_argument('--storyitem-filter', metavar='filter',
+    g_cond.add_argument('--storyitem-filter', metavar='filter',
                        help='Expression that, if given, must evaluate to True for each storyitem to be downloaded. '
                             'Must be a syntactically valid python expression. Variables are evaluated to '
                             'instaloader.StoryItem attributes.')
-    g_stop = parser.add_argument_group('When to Stop Downloading',
+    g_cond.add_argument('-c', '--count',
                                       'If none of these options are given, Instaloader goes through all pictures '
                                       'matching the specified targets.')
    g_stop.add_argument('-F', '--fast-update', action='store_true',
                        help='For each target, stop when encountering the first already-downloaded picture. This '
                             'flag is recommended when you use Instaloader to update your personal Instagram archive.')
    g_stop.add_argument('-c', '--count',
                        help='Do not attempt to download more than COUNT posts. '
                             'Applies only to #hashtag and :feed.')
@@ -276,6 +334,10 @@ def main():
                       help='Maximum number of connection attempts until a request is aborted. Defaults to 3. If a '
                            'connection fails, it can be manually skipped by hitting CTRL+C. Set this to 0 to retry '
                            'infinitely.')
    g_how.add_argument('--commit-mode', action='store_true',
                       help='Tries to ensure downloaded images avoid corruption in case of unexpected interruption. '
                       'If the last picture is corrupted, Instaloader will fix the picture the next time it is run. '
                       'Requires the JSON metadata to be saved.')
    g_misc = parser.add_argument_group('Miscellaneous Options')
    g_misc.add_argument('-q', '--quiet', action='store_true',
@@ -309,8 +371,20 @@ def main():
                raise SystemExit("--no-captions and --post-metadata-txt or --storyitem-metadata-txt given; "
                                 "That contradicts.")
        if args.no_pictures and args.fast_update:
            raise SystemExit('--no-pictures and --fast-update cannot be used together.')
        # Determine what to download
        download_profile_pic = not args.no_profile_pic or args.profile_pic_only
        download_posts = not (args.no_posts or args.stories_only or args.profile_pic_only)
        download_stories = args.stories or args.stories_only
        if args.commit_mode and args.no_metadata_json:
            raise SystemExit('--commit-mode requires JSON metadata to be saved.')
        loader = Instaloader(sleep=not args.no_sleep, quiet=args.quiet, user_agent=args.user_agent,
                             dirname_pattern=args.dirname_pattern, filename_pattern=args.filename_pattern,
                             download_pictures=not args.no_pictures,
                             download_videos=not args.no_videos, download_video_thumbnails=not args.no_video_thumbnails,
                             download_geotags=args.geotags,
                             download_comments=args.comments, save_metadata=not args.no_metadata_json,
@@ -318,18 +392,20 @@ def main():
                             post_metadata_txt_pattern=post_metadata_txt_pattern,
                             storyitem_metadata_txt_pattern=storyitem_metadata_txt_pattern,
                             graphql_rate_limit=args.graphql_rate_limit,
-                             max_connection_attempts=args.max_connection_attempts)
+                             max_connection_attempts=args.max_connection_attempts,
                             commit_mode=args.commit_mode)
        _main(loader,
              args.profile,
              username=args.login.lower() if args.login is not None else None,
              password=args.password,
              sessionfile=args.sessionfile,
-              max_count=int(args.count) if args.count is not None else None,
+              download_profile_pic=download_profile_pic,
-              profile_pic=not args.no_profile_pic,
+              download_posts=download_posts,
-              profile_pic_only=args.profile_pic_only,
+              download_stories=download_stories,
              download_highlights=args.highlights,
              download_tagged=args.tagged,
              fast_update=args.fast_update,
-              stories=args.stories,
+              max_count=int(args.count) if args.count is not None else None,
              stories_only=args.stories_only,
              post_filter_str=args.post_filter,
              storyitem_filter_str=args.storyitem_filter)
        loader.close()
@@ -33,6 +33,10 @@ class LoginRequiredException(InstaloaderException):
    pass
 class TwoFactorAuthRequiredException(InstaloaderException):
    pass
 class InvalidArgumentException(InstaloaderException):
    pass
@@ -1,5 +1,6 @@
 import getpass
 import json
 import lzma
 import os
 import platform
 import re
@@ -11,11 +12,11 @@ from contextlib import contextmanager, suppress
 from datetime import datetime, timezone
 from functools import wraps
 from io import BytesIO
-from typing import Callable, Iterator, List, Optional, Any
+from typing import Any, Callable, Iterator, List, Optional, Set, Union
 from .exceptions import *
 from .instaloadercontext import InstaloaderContext
-from .structures import JsonExportable, Post, PostLocation, Profile, Story, StoryItem, save_structure_to_file
+from .structures import Highlight, JsonExportable, Post, PostLocation, Profile, Story, StoryItem, save_structure_to_file, load_structure_from_file
 def get_default_session_filename(username: str) -> str:
@@ -49,11 +50,11 @@ class _ArbitraryItemFormatter(string.Formatter):
    def __init__(self, item: Any):
        self._item = item
-    def get_field(self, field_name, args, kwargs):
+    def get_value(self, key, args, kwargs):
        """Override to substitute {ATTRIBUTE} by attributes of our _item."""
-        if hasattr(self._item, field_name):
+        if hasattr(self._item, key):
-            return self._item.__getattribute__(field_name), None
+            return getattr(self._item, key)
-        return super().get_field(field_name, args, kwargs)
+        return super().get_value(key, args, kwargs)
    def format_field(self, value, format_spec):
        """Override :meth:`string.Formatter.format_field` to have our
@@ -70,7 +71,10 @@ class _PostPathFormatter(_ArbitraryItemFormatter):
    def vformat(self, format_string, args, kwargs):
        """Override :meth:`string.Formatter.vformat` for character substitution in paths for Windows, see issue #84."""
        ret = super().vformat(format_string, args, kwargs)
-        return ret.replace(':', '\ua789') if platform.system() == 'Windows' else ret
+        if platform.system() == 'Windows':
            ret = ret.replace(':', '\ua789').replace('<', '\ufe64').replace('>', '\ufe65').replace('\"', '\uff02')
            ret = ret.replace('\\', '\uff3c').replace('|', '\uff5c').replace('?', '\ufe16').replace('*', '\uff0a')
        return ret
 class Instaloader:
@@ -80,6 +84,7 @@ class Instaloader:
    :param user_agent: :option:`--user-agent`
    :param dirname_pattern: :option:`--dirname-pattern`, default is ``{target}``
    :param filename_pattern: :option:`--filename-pattern`, default is ``{date_utc}_UTC``
    :param download_pictures: not :option:`--no-pictures`
    :param download_videos: not :option:`--no-videos`
    :param download_video_thumbnails: not :option:`--no-video-thumbnails`
    :param download_geotags: :option:`--geotags`
@@ -91,6 +96,7 @@ class Instaloader:
       txt file.
    :param storyitem_metadata_txt_pattern: :option:`--storyitem-metadata-txt`, default is empty (=none)
    :param max_connection_attempts: :option:`--max-connection-attempts`
    :param commit_mode: :option:`--commit-mode`
    .. attribute:: context
@@ -102,6 +108,7 @@ class Instaloader:
                 user_agent: Optional[str] = None,
                 dirname_pattern: Optional[str] = None,
                 filename_pattern: Optional[str] = None,
                 download_pictures=True,
                 download_videos: bool = True,
                 download_video_thumbnails: bool = True,
                 download_geotags: bool = True,
@@ -111,13 +118,15 @@ class Instaloader:
                 post_metadata_txt_pattern: str = None,
                 storyitem_metadata_txt_pattern: str = None,
                 graphql_rate_limit: Optional[int] = None,
-                 max_connection_attempts: int = 3):
+                 max_connection_attempts: int = 3,
                 commit_mode: bool = False):
        self.context = InstaloaderContext(sleep, quiet, user_agent, graphql_rate_limit, max_connection_attempts)
        # configuration parameters
        self.dirname_pattern = dirname_pattern or "{target}"
        self.filename_pattern = filename_pattern or "{date_utc}_UTC"
        self.download_pictures = download_pictures
        self.download_videos = download_videos
        self.download_video_thumbnails = download_video_thumbnails
        self.download_geotags = download_geotags
@@ -128,16 +137,26 @@ class Instaloader:
            else post_metadata_txt_pattern
        self.storyitem_metadata_txt_pattern = '' if storyitem_metadata_txt_pattern is None \
            else storyitem_metadata_txt_pattern
        self.commit_mode = commit_mode
        if self.commit_mode and not self.save_metadata:
            raise InvalidArgumentException("Commit mode requires JSON metadata to be saved.")
        # Used to keep state in commit mode
        self._committed = None
    @contextmanager
    def anonymous_copy(self):
        """Yield an anonymous, otherwise equally-configured copy of an Instaloader instance; Then copy its error log."""
        new_loader = Instaloader(self.context.sleep, self.context.quiet, self.context.user_agent, self.dirname_pattern,
-                                 self.filename_pattern, self.download_videos, self.download_video_thumbnails,
+                                 self.filename_pattern, download_pictures=self.download_pictures,
-                                 self.download_geotags, self.download_comments, self.save_metadata,
+                                 download_videos=self.download_videos,
-                                 self.compress_json, self.post_metadata_txt_pattern,
+                                 download_video_thumbnails=self.download_video_thumbnails,
-                                 self.storyitem_metadata_txt_pattern, self.context.graphql_count_per_slidingwindow,
+                                 download_geotags=self.download_geotags, download_comments=self.download_comments,
-                                 self.context.max_connection_attempts)
+                                 save_metadata=self.save_metadata, compress_json=self.compress_json,
                                 post_metadata_txt_pattern=self.post_metadata_txt_pattern,
                                 storyitem_metadata_txt_pattern=self.storyitem_metadata_txt_pattern,
                                 graphql_rate_limit=self.context.graphql_count_per_slidingwindow,
                                 max_connection_attempts=self.context.max_connection_attempts)
        new_loader.context.query_timestamps = self.context.query_timestamps
        yield new_loader
        self.context.error_log.extend(new_loader.context.error_log)
@@ -163,9 +182,15 @@ class Instaloader:
        if filename_suffix is not None:
            filename += '_' + filename_suffix
        filename += '.' + file_extension
-        if os.path.isfile(filename):
+        # A post is considered "commited" if the json file exists and is not malformed.
-            self.context.log(filename + ' exists', end=' ', flush=True)
+        if self.commit_mode:
-            return False
+            if self._committed and os.path.isfile(filename):
                self.context.log(filename + ' exists', end=' ', flush=True)
                return False
        else:
            if os.path.isfile(filename):
                self.context.log(filename + ' exists', end=' ', flush=True)
                return False
        self.context.get_and_write_raw(url, filename)
        os.utime(filename, (datetime.now().timestamp(), mtime.timestamp()))
        return True
@@ -266,18 +291,19 @@ class Instaloader:
        def _epoch_to_string(epoch: datetime) -> str:
            return epoch.strftime('%Y-%m-%d_%H-%M-%S')
-        profile_pic_url = profile.get_profile_pic_url()
+        profile_pic_url = profile.profile_pic_url
        with self.context.get_anonymous_session() as anonymous_session:
            date_object = datetime.strptime(anonymous_session.head(profile_pic_url).headers["Last-Modified"],
                                            '%a, %d %b %Y %H:%M:%S GMT')
        profile_pic_extension = 'jpg'
        if ((format_string_contains_key(self.dirname_pattern, 'profile') or
             format_string_contains_key(self.dirname_pattern, 'target'))):
            filename = '{0}/{1}_UTC_profile_pic.{2}'.format(self.dirname_pattern.format(profile=profile.username.lower(),
                                                                                        target=profile.username.lower()),
-                                                            _epoch_to_string(date_object), profile_pic_url[-3:])
+                                                            _epoch_to_string(date_object), profile_pic_extension)
        else:
            filename = '{0}/{1}_{2}_UTC_profile_pic.{3}'.format(self.dirname_pattern.format(), profile.username.lower(),
-                                                                _epoch_to_string(date_object), profile_pic_url[-3:])
+                                                                _epoch_to_string(date_object), profile_pic_extension)
        if os.path.isfile(filename):
            self.context.log(filename + ' already exists')
            return None
@@ -320,9 +346,30 @@ class Instaloader:
        return self.context.test_login()
    def login(self, user: str, passwd: str) -> None:
-        """Log in to instagram with given username and password and internally store session object"""
+        """Log in to instagram with given username and password and internally store session object.
        :raises InvalidArgumentException: If the provided username does not exist.
        :raises BadCredentialsException: If the provided password is wrong.
        :raises ConnectionException: If connection to Instagram failed.
        :raises TwoFactorAuthRequiredException: First step of 2FA login done, now call :meth:`Instaloader.two_factor_login`."""
        self.context.login(user, passwd)
    def two_factor_login(self, two_factor_code) -> None:
        """Second step of login if 2FA is enabled.
        Not meant to be used directly, use :meth:`Instaloader.two_factor_login`.
        :raises InvalidArgumentException: No two-factor authentication pending.
        :raises BadCredentialsException: 2FA verification code invalid.
        .. versionadded:: 4.2"""
        self.context.two_factor_login(two_factor_code)
    def format_filename(self, item: Union[Post, StoryItem], target: Optional[str] = None):
        """Format filename of a :class:`Post` or :class:`StoryItem` according to ``filename-pattern`` parameter.
        .. versionadded:: 4.1"""
        return _PostPathFormatter(item).format(self.filename_pattern, target=target)
    def download_post(self, post: Post, target: str) -> bool:
        """
        Download everything associated with one instagram post node, i.e. picture, caption and video.
@@ -333,30 +380,32 @@ class Instaloader:
        """
        dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target)
-        filename = dirname + '/' + _PostPathFormatter(post).format(self.filename_pattern, target=target)
+        filename = dirname + '/' + self.format_filename(post, target=target)
        os.makedirs(os.path.dirname(filename), exist_ok=True)
        # Download the image(s) / video thumbnail and videos within sidecars if desired
-        downloaded = False
+        downloaded = True
-        if post.typename == 'GraphSidecar':
+        self._committed = self.check_if_committed(filename)
-            edge_number = 1
+        if self.download_pictures:
-            for sidecar_node in post.get_sidecar_nodes():
+            if post.typename == 'GraphSidecar':
-                # Download picture or video thumbnail
+                edge_number = 1
-                if not sidecar_node.is_video or self.download_video_thumbnails is True:
+                for sidecar_node in post.get_sidecar_nodes():
-                    downloaded |= self.download_pic(filename=filename, url=sidecar_node.display_url,
+                    # Download picture or video thumbnail
-                                                    mtime=post.date_local, filename_suffix=str(edge_number))
+                    if not sidecar_node.is_video or self.download_video_thumbnails is True:
-                # Additionally download video if available and desired
+                        downloaded &= self.download_pic(filename=filename, url=sidecar_node.display_url,
-                if sidecar_node.is_video and self.download_videos is True:
+                                                        mtime=post.date_local, filename_suffix=str(edge_number))
-                    downloaded |= self.download_pic(filename=filename, url=sidecar_node.video_url,
+                    # Additionally download video if available and desired
-                                                    mtime=post.date_local, filename_suffix=str(edge_number))
+                    if sidecar_node.is_video and self.download_videos is True:
-                edge_number += 1
+                        downloaded &= self.download_pic(filename=filename, url=sidecar_node.video_url,
-        elif post.typename == 'GraphImage':
+                                                        mtime=post.date_local, filename_suffix=str(edge_number))
-            downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
+                    edge_number += 1
-        elif post.typename == 'GraphVideo':
+            elif post.typename == 'GraphImage':
            if self.download_video_thumbnails is True:
                downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
-        else:
+            elif post.typename == 'GraphVideo':
-            self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename))
+                if self.download_video_thumbnails is True:
                    downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
            else:
                self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename))
        # Save caption if desired
        metadata_string = _ArbitraryItemFormatter(post).format(self.post_metadata_txt_pattern).strip()
@@ -365,7 +414,7 @@ class Instaloader:
        # Download video if desired
        if post.is_video and self.download_videos is True:
-            downloaded |= self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local)
+            downloaded &= self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local)
        # Download geotags if desired
        if self.download_geotags and post.location:
@@ -391,37 +440,45 @@ class Instaloader:
        :param userids: List of user IDs to be processed in terms of downloading their stories, or None.
        """
-        if userids is None:
+        if not userids:
            data = self.context.graphql_query("d15efd8c0c5b23f0ef71f18bf363c704",
                                              {"only_stories": True})["data"]["user"]
            if data is None:
                raise BadResponseException('Bad stories reel JSON.')
            userids = list(edge["node"]["id"] for edge in data["feed_reels_tray"]["edge_reels_tray_to_reel"]["edges"])
-        stories = self.context.graphql_query("bf41e22b1c4ba4c9f31b844ebb7d9056",
+        def _userid_chunks():
-                                             {"reel_ids": userids, "precomposed_overlay": False})["data"]
+            userids_per_query = 100
            for i in range(0, len(userids), userids_per_query):
                yield userids[i:i + userids_per_query]
-        yield from (Story(self.context, media) for media in stories['reels_media'])
+        for userid_chunk in _userid_chunks():
            stories = self.context.graphql_query("bf41e22b1c4ba4c9f31b844ebb7d9056",
                                                 {"reel_ids": userid_chunk, "precomposed_overlay": False})["data"]
            yield from (Story(self.context, media) for media in stories['reels_media'])
    @_requires_login
    def download_stories(self,
-                         userids: Optional[List[int]] = None,
+                         userids: Optional[List[Union[int, Profile]]] = None,
                         fast_update: bool = False,
-                         filename_target: str = ':stories',
+                         filename_target: Optional[str] = ':stories',
                         storyitem_filter: Optional[Callable[[StoryItem], bool]] = None) -> None:
        """
        Download available stories from user followees or all stories of users whose ID are given.
        Does not mark stories as seen.
        To use this, one needs to be logged in
-        :param userids: List of user IDs to be processed in terms of downloading their stories
+        :param userids: List of user IDs or Profiles to be processed in terms of downloading their stories
        :param fast_update: If true, abort when first already-downloaded picture is encountered
        :param filename_target: Replacement for {target} in dirname_pattern and filename_pattern
               or None if profile name should be used instead
        :param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded
        """
        if not userids:
            self.context.log("Retrieving all visible stories...")
        else:
            userids = [p if isinstance(p, int) else p.userid for p in userids]
        for user_story in self.get_stories(userids):
            name = user_story.owner_username
@@ -435,7 +492,7 @@ class Instaloader:
                self.context.log("[%3i/%3i] " % (count, totalcount), end="", flush=True)
                count += 1
                with self.context.error_catcher('Download story from user {}'.format(name)):
-                    downloaded = self.download_storyitem(item, filename_target)
+                    downloaded = self.download_storyitem(item, filename_target if filename_target else name)
                    if fast_update and not downloaded:
                        break
@@ -449,7 +506,7 @@ class Instaloader:
        date_local = item.date_local
        dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target)
-        filename = dirname + '/' + _PostPathFormatter(item).format(self.filename_pattern, target=target)
+        filename = dirname + '/' + self.format_filename(item, target=target)
        os.makedirs(os.path.dirname(filename), exist_ok=True)
        downloaded = False
        if not item.is_video or self.download_video_thumbnails is True:
@@ -467,6 +524,62 @@ class Instaloader:
        self.context.log()
        return downloaded
    @_requires_login
    def get_highlights(self, user: Union[int, Profile]) -> Iterator[Highlight]:
        """Get all highlights from a user.
        To use this, one needs to be logged in.
        .. versionadded:: 4.1
        :param user: ID or Profile of the user whose highlights should get fetched.
        """
        userid = user if isinstance(user, int) else user.userid
        data = self.context.graphql_query("7c16654f22c819fb63d1183034a5162f",
                                          {"user_id": userid, "include_chaining": False, "include_reel": False,
                                           "include_suggested_users": False, "include_logged_out_extras": False,
                                           "include_highlight_reels": True})["data"]["user"]['edge_highlight_reels']
        if data is None:
            raise BadResponseException('Bad highlights reel JSON.')
        yield from (Highlight(self.context, edge['node'], user if isinstance(user, Profile) else None)
                    for edge in data['edges'])
    @_requires_login
    def download_highlights(self,
                            user: Union[int, Profile],
                            fast_update: bool = False,
                            filename_target: Optional[str] = None,
                            storyitem_filter: Optional[Callable[[StoryItem], bool]] = None) -> None:
        """
        Download available highlights from a user whose ID is given.
        To use this, one needs to be logged in.
        .. versionadded:: 4.1
        :param user: ID or Profile of the user whose highlights should get downloaded.
        :param fast_update: If true, abort when first already-downloaded picture is encountered
        :param filename_target: Replacement for {target} in dirname_pattern and filename_pattern
               or None if profile name and the highlights' titles should be used instead
        :param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded
        """
        for user_highlight in self.get_highlights(user):
            name = user_highlight.owner_username
            self.context.log("Retrieving highlights \"{}\" from profile {}".format(user_highlight.title, name))
            totalcount = user_highlight.itemcount
            count = 1
            for item in user_highlight.get_items():
                if storyitem_filter is not None and not storyitem_filter(item):
                    self.context.log("<{} skipped>".format(item), flush=True)
                    continue
                self.context.log("[%3i/%3i] " % (count, totalcount), end="", flush=True)
                count += 1
                with self.context.error_catcher('Download highlights \"{}\" from user {}'.format(user_highlight.title, name)):
                    downloaded = self.download_storyitem(item, filename_target
                                                         if filename_target
                                                         else '{}/{}'.format(name, user_highlight.title))
                    if fast_update and not downloaded:
                        break
    @_requires_login
    def get_feed_posts(self) -> Iterator[Post]:
        """Get Posts of the user's feed.
@@ -546,6 +659,59 @@ class Instaloader:
                if fast_update and not downloaded:
                    break
    def get_location_posts(self, location: str) -> Iterator[Post]:
        """Get Posts which are listed by Instagram for a given Location.
        :return:  Iterator over Posts of a location's posts
        .. versionadded:: 4.2
        """
        has_next_page = True
        end_cursor = None
        while has_next_page:
            if end_cursor:
                params = {'__a': 1, 'max_id': end_cursor}
            else:
                params = {'__a': 1}
            location_data = self.context.get_json('explore/locations/{0}/'.format(location),
                                                  params)['graphql']['location']['edge_location_to_media']
            yield from (Post(self.context, edge['node']) for edge in location_data['edges'])
            has_next_page = location_data['page_info']['has_next_page']
            end_cursor = location_data['page_info']['end_cursor']
    def download_location(self, location: str,
                          max_count: Optional[int] = None,
                          post_filter: Optional[Callable[[Post], bool]] = None,
                          fast_update: bool = False) -> None:
        """Download pictures of one location.
        To download the last 30 pictures with location 362629379, do::
            loader = Instaloader()
            loader.download_location(362629379, max_count=30)
        :param location: Location to download, as Instagram numerical ID
        :param max_count: Maximum count of pictures to download
        :param post_filter: function(post), which returns True if given picture should be downloaded
        :param fast_update: If true, abort when first already-downloaded picture is encountered
        .. versionadded:: 4.2
        """
        self.context.log("Retrieving pictures for location {}...".format(location))
        count = 1
        for post in self.get_location_posts(location):
            if max_count is not None and count > max_count:
                break
            self.context.log('[{0:3d}] %{1} '.format(count, location), end='', flush=True)
            if post_filter is not None and not post_filter(post):
                self.context.log('<skipped>')
                continue
            count += 1
            with self.context.error_catcher('Download location {}'.format(location)):
                downloaded = self.download_post(post, target='%' + location)
                if fast_update and not downloaded:
                    break
    @_requires_login
    def get_explore_posts(self) -> Iterator[Post]:
        """Get Posts which are worthy of exploring suggested by Instagram.
@@ -606,6 +772,46 @@ class Instaloader:
                if fast_update and not downloaded:
                    break
    def download_tagged(self, profile: Profile, fast_update: bool = False,
                        target: Optional[str] = None,
                        post_filter: Optional[Callable[[Post], bool]] = None) -> None:
        """Download all posts where a profile is tagged.
        .. versionadded:: 4.1"""
        if target is None:
            target = profile.username + '/:tagged'
        self.context.log("Retrieving tagged posts for profile {}.".format(profile.username))
        count = 1
        for post in profile.get_tagged_posts():
            self.context.log("[%3i/???] " % (count), end="", flush=True)
            count += 1
            if post_filter is not None and not post_filter(post):
                self.context.log('<{} skipped>'.format(post))
            with self.context.error_catcher('Download tagged {}'.format(profile.username)):
                downloaded = self.download_post(post, target)
                if fast_update and not downloaded:
                    break
    def _get_id_filename(self, profile_name: str) -> str:
        if ((format_string_contains_key(self.dirname_pattern, 'profile') or
             format_string_contains_key(self.dirname_pattern, 'target'))):
            return '{0}/id'.format(self.dirname_pattern.format(profile=profile_name.lower(),
                                                               target=profile_name.lower()))
        else:
            return '{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower())
    def save_profile_id(self, profile: Profile):
        """
        Store ID of profile locally.
        .. versionadded:: 4.0.6
        """
        os.makedirs(self.dirname_pattern.format(profile=profile.username,
                                                target=profile.username), exist_ok=True)
        with open(self._get_id_filename(profile.username), 'w') as text_file:
            text_file.write(str(profile.userid) + "\n")
            self.context.log("Stored ID {0} for profile {1}.".format(profile.userid, profile.username))
    def check_profile_id(self, profile_name: str) -> Profile:
        """
        Consult locally stored ID of profile with given name, check whether ID matches and whether name
@@ -618,12 +824,7 @@ class Instaloader:
        with suppress(ProfileNotExistsException):
            profile = Profile.from_username(self.context, profile_name)
        profile_exists = profile is not None
-        if ((format_string_contains_key(self.dirname_pattern, 'profile') or
+        id_filename = self._get_id_filename(profile_name)
             format_string_contains_key(self.dirname_pattern, 'target'))):
            id_filename = '{0}/id'.format(self.dirname_pattern.format(profile=profile_name.lower(),
                                                                      target=profile_name.lower()))
        else:
            id_filename = '{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower())
        try:
            with open(id_filename, 'rb') as id_file:
                profile_id = int(id_file.read())
@@ -652,26 +853,114 @@ class Instaloader:
        except (FileNotFoundError, ValueError):
            pass
        if profile_exists:
-            os.makedirs(self.dirname_pattern.format(profile=profile_name.lower(),
+            self.save_profile_id(profile)
                                                    target=profile_name.lower()), exist_ok=True)
            with open(id_filename, 'w') as text_file:
                text_file.write(str(profile.userid) + "\n")
                self.context.log("Stored ID {0} for profile {1}.".format(profile.userid, profile_name))
            return profile
        raise ProfileNotExistsException("Profile {0} does not exist.".format(profile_name))
-    def download_profile(self, profile_name: str,
+    def download_profiles(self, profiles: Set[Profile],
                          profile_pic: bool = True, posts: bool = True,
                          tagged: bool = False, highlights: bool = False, stories: bool = False,
                          fast_update: bool = False,
                          post_filter: Optional[Callable[[Post], bool]] = None,
                          storyitem_filter: Optional[Callable[[Post], bool]] = None,
                          raise_errors: bool = False):
        """High-level method to download set of profiles.
        :param profiles: Set of profiles to download.
        :param profile_pic: not :option:`--no-profile-pic`.
        :param posts: not :option:`--no-posts`.
        :param tagged: :option:`--tagged`.
        :param highlights: :option:`--highlights`.
        :param stories: :option:`--stories`.
        :param fast_update: :option:`--fast-update`.
        :param post_filter: :option:`--post-filter`.
        :param storyitem_filter: :option:`--post-filter`.
        :param raise_errors:
           Whether :exc:`LoginRequiredException` and :exc:`PrivateProfileNotFollowedException` should be raised or
           catched and printed with :meth:`InstaloaderContext.error_catcher`.
        .. versionadded:: 4.1"""
        def _error_raiser(_str):
            yield
        error_handler = _error_raiser if raise_errors else self.context.error_catcher
        for profile in profiles:
            with error_handler(profile.username):
                profile_name = profile.username
                # Save metadata as JSON if desired.
                if self.save_metadata:
                    json_filename = '{0}/{1}_{2}'.format(self.dirname_pattern.format(profile=profile_name,
                                                                                     target=profile_name),
                                                         profile_name, profile.userid)
                    self.save_metadata_json(json_filename, profile)
                # Download profile picture
                if profile_pic:
                    with self.context.error_catcher('Download profile picture of {}'.format(profile_name)):
                        self.download_profilepic(profile)
                # Catch some errors
                if profile.is_private and (tagged or highlights or posts):
                    if not self.context.is_logged_in:
                        raise LoginRequiredException("--login=USERNAME required.")
                    if not profile.followed_by_viewer and self.context.username != profile.username:
                        raise PrivateProfileNotFollowedException("Private but not followed.")
                # Download tagged, if requested
                if tagged:
                    with self.context.error_catcher('Download tagged of {}'.format(profile_name)):
                        self.download_tagged(profile, fast_update=fast_update, post_filter=post_filter)
                # Download highlights, if requested
                if highlights:
                    with self.context.error_catcher('Download highlights of {}'.format(profile_name)):
                        self.download_highlights(profile, fast_update=fast_update, storyitem_filter=storyitem_filter)
                # Iterate over pictures and download them
                if posts:
                    self.context.log("Retrieving posts from profile {}.".format(profile_name))
                    totalcount = profile.mediacount
                    count = 1
                    for post in profile.get_posts():
                        self.context.log("[%3i/%3i] " % (count, totalcount), end="", flush=True)
                        count += 1
                        if post_filter is not None and not post_filter(post):
                            self.context.log('<skipped>')
                            continue
                        with self.context.error_catcher("Download {} of {}".format(post, profile_name)):
                            downloaded = self.download_post(post, target=profile_name)
                            if fast_update and not downloaded:
                                break
        if stories and profiles:
            with self.context.error_catcher("Download stories"):
                self.context.log("Downloading stories")
                self.download_stories(userids=list(profiles), fast_update=fast_update, filename_target=None,
                                      storyitem_filter=storyitem_filter)
    def download_profile(self, profile_name: Union[str, Profile],
                         profile_pic: bool = True, profile_pic_only: bool = False,
                         fast_update: bool = False,
                         download_stories: bool = False, download_stories_only: bool = False,
                         download_tagged: bool = False, download_tagged_only: bool = False,
                         post_filter: Optional[Callable[[Post], bool]] = None,
                         storyitem_filter: Optional[Callable[[StoryItem], bool]] = None) -> None:
-        """Download one profile"""
+        """Download one profile
        .. deprecated:: 4.1
           Use :meth:`Instaloader.download_profiles`.
        """
        # Get profile main page json
        # check if profile does exist or name has changed since last download
        # and update name and json data if necessary
-        profile = self.check_profile_id(profile_name.lower())
+        if isinstance(profile_name, str):
            profile = self.check_profile_id(profile_name.lower())
        else:
            profile = profile_name
        profile_name = profile.username
@@ -714,6 +1003,13 @@ class Instaloader:
        if download_stories_only:
            return
        # Download tagged, if requested
        if download_tagged or download_tagged_only:
            with self.context.error_catcher('Download tagged of {}'.format(profile_name)):
                self.download_tagged(profile, fast_update=fast_update, post_filter=post_filter)
        if download_tagged_only:
            return
        # Iterate over pictures and download them
        self.context.log("Retrieving posts from profile {}.".format(profile_name))
        totalcount = profile.mediacount
@@ -729,17 +1025,47 @@ class Instaloader:
                if fast_update and not downloaded:
                    break
    def check_if_committed(self, filename: str) -> bool:
        """Checks to see if the current post has been committed.
        A post is considered committed if its json metadata file exists and is not malformed.
        .. versionadded:: 4.2
        """
        if os.path.isfile(filename + '.json.xz'):
            filename += '.json.xz'
        elif os.path.isfile(filename + '.json'):
            filename += '.json'
        else:
            return False
        try:
            load_structure_from_file(self.context, filename)
            return True
        except (FileNotFoundError, lzma.LZMAError, json.decoder.JSONDecodeError):
            return False
    def interactive_login(self, username: str) -> None:
        """Logs in and internally stores session, asking user for password interactively.
-        :raises LoginRequiredException: when in quiet mode."""
+        :raises LoginRequiredException: when in quiet mode.
        :raises InvalidArgumentException: If the provided username does not exist.
        :raises ConnectionException: If connection to Instagram failed."""
        if self.context.quiet:
            raise LoginRequiredException("Quiet mode requires given password or valid session file.")
-        password = None
+        try:
-        while password is None:
+            password = None
-            password = getpass.getpass(prompt="Enter Instagram password for %s: " % username)
+            while password is None:
-            try:
+                password = getpass.getpass(prompt="Enter Instagram password for %s: " % username)
-                self.login(username, password)
+                try:
-            except BadCredentialsException as err:
+                    self.login(username, password)
-                print(err, file=sys.stderr)
+                except BadCredentialsException as err:
-                password = None
+                    print(err, file=sys.stderr)
                    password = None
        except TwoFactorAuthRequiredException:
            while True:
                try:
                    code = input("Enter 2FA verification code: ")
                    self.two_factor_login(code)
                    break
                except BadCredentialsException:
                    pass
@@ -9,6 +9,7 @@ import textwrap
 import time
 import urllib.parse
 from contextlib import contextmanager
 from datetime import datetime, timedelta
 from typing import Any, Callable, Dict, Iterator, Optional
 import requests
@@ -56,8 +57,9 @@ class InstaloaderContext:
        self.quiet = quiet
        self.max_connection_attempts = max_connection_attempts
        self._graphql_page_length = 50
-        self.graphql_count_per_slidingwindow = graphql_count_per_slidingwindow or 20
+        self.graphql_count_per_slidingwindow = graphql_count_per_slidingwindow or 200
        self._root_rhx_gis = None
        self.two_factor_auth_pending = None
        # error log, filled with error() and printed at the end of Instaloader.main()
        self.error_log = []
@@ -68,16 +70,21 @@ class InstaloaderContext:
        # Can be set to True for testing, disables supression of InstaloaderContext._error_catcher
        self.raise_all_errors = False
        # Cache profile from id (mapping from id to Profile)
        self.profile_id_cache = dict()
    @contextmanager
    def anonymous_copy(self):
        session = self._session
        username = self.username
        self._session = self.get_anonymous_session()
        self.username = None
-        yield self
+        try:
-        self._session.close()
+            yield self
-        self.username = username
+        finally:
-        self._session = session
+            self._session.close()
            self.username = username
            self._session = session
    @property
    def is_logged_in(self) -> bool:
@@ -170,34 +177,94 @@ class InstaloaderContext:
        return data["data"]["user"]["username"] if data["data"]["user"] is not None else None
    def login(self, user, passwd):
-        """Not meant to be used directly, use :meth:`Instaloader.login`."""
+        """Not meant to be used directly, use :meth:`Instaloader.login`.
        :raises InvalidArgumentException: If the provided username does not exist.
        :raises BadCredentialsException: If the provided password is wrong.
        :raises ConnectionException: If connection to Instagram failed.
        :raises TwoFactorAuthRequiredException: First step of 2FA login done, now call :meth:`Instaloader.two_factor_login`."""
        import http.client
        # pylint:disable=protected-access
        http.client._MAXHEADERS = 200
        session = requests.Session()
        session.cookies.update({'sessionid': '', 'mid': '', 'ig_pr': '1',
-                                'ig_vw': '1920', 'csrftoken': '',
+                                'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '',
                                's_network': '', 'ds_user_id': ''})
        session.headers.update(self._default_http_header())
-        self._sleep()
+        session.get('https://www.instagram.com/web/__mid/')
-        resp = session.get('https://www.instagram.com/')
+        csrf_token = session.cookies.get_dict()['csrftoken']
-        session.headers.update({'X-CSRFToken': resp.cookies['csrftoken']})
+        session.headers.update({'X-CSRFToken': csrf_token})
        # Not using self.get_json() here, because we need to access csrftoken cookie
        self._sleep()
        login = session.post('https://www.instagram.com/accounts/login/ajax/',
                             data={'password': passwd, 'username': user}, allow_redirects=True)
-        session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
+        try:
-        if login.status_code == 200:
+            resp_json = login.json()
-            self._session = session
+        except json.decoder.JSONDecodeError:
-            if user == self.test_login():
+            raise ConnectionException("Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason))
-                self.username = user
+        if resp_json.get('two_factor_required'):
            two_factor_session = copy_session(session)
            two_factor_session.headers.update({'X-CSRFToken': csrf_token})
            two_factor_session.cookies.update({'csrftoken': csrf_token})
            self.two_factor_auth_pending = (two_factor_session,
                                            user,
                                            resp_json['two_factor_info']['two_factor_identifier'])
            raise TwoFactorAuthRequiredException("Login error: two-factor authentication required.")
        if resp_json.get('checkpoint_url'):
            raise ConnectionException("Login: Checkpoint required. Point your browser to "
                                      "https://www.instagram.com{}, "
                                      "follow the instructions, then retry.".format(resp_json.get('checkpoint_url')))
        if resp_json['status'] != 'ok':
            if 'message' in resp_json:
                raise ConnectionException("Login error: \"{}\" status, message \"{}\".".format(resp_json['status'],
                                                                                               resp_json['message']))
            else:
-                self.username = None
+                raise ConnectionException("Login error: \"{}\" status.".format(resp_json['status']))
-                self._session = None
+        if not resp_json['authenticated']:
-                raise BadCredentialsException('Login error! Check your credentials!')
+            if resp_json['user']:
-        else:
+                # '{"authenticated": false, "user": true, "status": "ok"}'
-            raise ConnectionException('Login error! Connection error!')
+                raise BadCredentialsException('Login error: Wrong password.')
            else:
                # '{"authenticated": false, "user": false, "status": "ok"}'
                # Raise InvalidArgumentException rather than BadCredentialException, because BadCredentialException
                # triggers re-asking of password in Instaloader.interactive_login(), which makes no sense if the
                # username is invalid.
                raise InvalidArgumentException('Login error: User {} does not exist.'.format(user))
        # '{"authenticated": true, "user": true, "userId": ..., "oneTapPrompt": false, "status": "ok"}'
        session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
        self._session = session
        self.username = user
    def two_factor_login(self, two_factor_code):
        """Second step of login if 2FA is enabled.
        Not meant to be used directly, use :meth:`Instaloader.two_factor_login`.
        :raises InvalidArgumentException: No two-factor authentication pending.
        :raises BadCredentialsException: 2FA verification code invalid.
        .. versionadded:: 4.2"""
        if not self.two_factor_auth_pending:
            raise InvalidArgumentException("No two-factor authentication pending.")
        (session, user, two_factor_id) = self.two_factor_auth_pending
        login = session.post('https://www.instagram.com/accounts/login/ajax/two_factor/',
                             data={'username': user, 'verificationCode': two_factor_code, 'identifier': two_factor_id},
                             allow_redirects=True)
        resp_json = login.json()
        if resp_json['status'] != 'ok':
            if 'message' in resp_json:
                raise BadCredentialsException("Login error: {}".format(resp_json['message']))
            else:
                raise BadCredentialsException("Login error: \"{}\" status.".format(resp_json['status']))
        session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
        self._session = session
        self.username = user
        self.two_factor_auth_pending = None
    def _sleep(self):
        """Sleep a short time if self.sleep is set. Called before each request to instagram.com."""
        if self.sleep:
-            time.sleep(min(random.expovariate(0.6), 5.0))
+            time.sleep(min(random.expovariate(0.7), 5.0))
    def get_json(self, path: str, params: Dict[str, Any], host: str = 'www.instagram.com',
                 session: Optional[requests.Session] = None, _attempt=1) -> Dict[str, Any]:
@@ -228,7 +295,8 @@ class InstaloaderContext:
        if is_graphql_query and not query_not_limited:
            waittime = graphql_query_waittime()
            if waittime > 0:
-                self.log('\nToo many queries in the last time. Need to wait {} seconds.'.format(waittime))
+                self.log('\nToo many queries in the last time. Need to wait {} seconds, until {:%H:%M}.'
                         .format(waittime, datetime.now() + timedelta(seconds=waittime)))
                time.sleep(waittime)
            if self.query_timestamps is not None:
                self.query_timestamps.append(time.monotonic())
@@ -281,9 +349,11 @@ class InstaloaderContext:
                if isinstance(err, TooManyRequestsException):
                    print(textwrap.fill(text_for_429), file=sys.stderr)
                    if is_graphql_query:
                        print("Made {} GraphQL requests.".format(len(self.query_timestamps)), file=sys.stderr)
                        waittime = graphql_query_waittime(untracked_queries=True)
                        if waittime > 0:
-                            self.log('The request will be retried in {} seconds.'.format(waittime))
+                            self.log('The request will be retried in {} seconds, at {:%H:%M}.'
                                     .format(waittime, datetime.now() + timedelta(seconds=waittime)))
                            time.sleep(waittime)
                self._sleep()
                return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1)
@@ -93,10 +93,9 @@ class Post:
        return b64encode(mediaid.to_bytes(9, 'big'), b'-_').decode().replace('A', ' ').lstrip().replace(' ', 'A')
    def _asdict(self):
        node = self._node
        if self._full_metadata_dict:
-            node = self._full_metadata_dict
+            node.update(self._full_metadata_dict)
        else:
            node = self._node
        if self._owner_profile:
            node['owner'] = self.owner_profile._asdict()
        if self._location:
@@ -206,10 +205,7 @@ class Post:
    @property
    def typename(self) -> str:
        """Type of post, GraphImage, GraphVideo or GraphSidecar"""
-        if '__typename' in self._node:
+        return self._field('__typename')
            return self._node['__typename']
        # if __typename is not in node, it is an old image or video
        return 'GraphImage'
    def get_sidecar_nodes(self) -> Iterator[PostSidecarNode]:
        """Sidecar nodes of a Post with typename==GraphSidecar."""
@@ -374,7 +370,6 @@ class Profile:
    def __init__(self, context: InstaloaderContext, node: Dict[str, Any]):
        assert 'username' in node
        self._context = context
        self._has_highlight_reels = None
        self._has_public_story = None
        self._node = node
        self._rhx_gis = None
@@ -403,6 +398,8 @@ class Profile:
        :param profile_id: userid
        :raises: :class:`ProfileNotExistsException`, :class:`ProfileHasNoPicsException`
        """
        if profile_id in context.profile_id_cache:
            return context.profile_id_cache[profile_id]
        data = context.graphql_query("472f257a40c653c64c666ce877d59d2b",
                                     {'id': str(profile_id), 'first': 1},
                                     rhx_gis=context.root_rhx_gis)['data']['user']
@@ -416,7 +413,9 @@ class Profile:
                raise ProfileHasNoPicsException("Profile with ID {0}: no pics found.".format(str(profile_id)))
            else:
                raise LoginRequiredException("Login required to determine username (ID: " + str(profile_id) + ").")
-        return Post(context, data['edges'][0]['node']).owner_profile
+        profile = Post(context, data['edges'][0]['node']).owner_profile
        context.profile_id_cache[profile_id] = profile
        return profile
    def _asdict(self):
        json_node = self._node.copy()
@@ -516,15 +515,12 @@ class Profile:
    @property
    def has_highlight_reels(self) -> bool:
        """
-        This becomes `True` if the :class:`Profile` has any stories currently available,
+        .. deprecated:: 4.0.6
-        even if not viewable by the viewer.
+           Always returns `True` since :issue:`153`.
        Before broken, this indicated whether the :class:`Profile` had available stories.
        """
-        if not self._has_highlight_reels:
+        return True
            with self._context.anonymous_copy() as anonymous_context:
                data = anonymous_context.get_json(path='api/v1/users/{}/info/'.format(self.userid),
                                                  params={}, host='i.instagram.com')
            self._has_highlight_reels = data['user']['has_highlight_reels']
        return self._has_highlight_reels
    @property
    def has_public_story(self) -> bool:
@@ -545,6 +541,8 @@ class Profile:
    @property
    def has_viewable_story(self) -> bool:
        """
        .. deprecated:: 4.0.6
        Some stories are private. This property determines if the :class:`Profile`
        has at least one story which can be viewed using the associated :class:`InstaloaderContext`,
        i.e. the viewer has privileges to view it.
@@ -563,16 +561,18 @@ class Profile:
    def requested_by_viewer(self) -> bool:
        return self._metadata('requested_by_viewer')
    @property
    def profile_pic_url(self) -> str:
        """Return URL of profile picture
        .. versionadded:: 4.0.3"""
        return self._metadata("profile_pic_url_hd")
    def get_profile_pic_url(self) -> str:
-        """Return URL of profile picture"""
+        """.. deprecated:: 4.0.3
-        try:
+
-            with self._context.get_anonymous_session() as anonymous_session:
+	   Use :attr:`profile_pic_url`."""
-                data = self._context.get_json(path='api/v1/users/{0}/info/'.format(self.userid), params={},
+        return self.profile_pic_url
                                              host='i.instagram.com', session=anonymous_session)
            return data["user"]["hd_profile_pic_url_info"]["url"]
        except (InstaloaderException, KeyError) as err:
            self._context.error('{} Unable to fetch high quality profile pic.'.format(err))
            return self._metadata("profile_pic_url_hd")
    def get_posts(self) -> Iterator[Post]:
        """Retrieve all posts from a profile."""
@@ -600,6 +600,18 @@ class Profile:
                                                    self._rhx_gis,
                                                    self._metadata('edge_saved_media')))
    def get_tagged_posts(self) -> Iterator[Post]:
        """Retrieve all posts where a profile is tagged.
        .. versionadded:: 4.0.7"""
        self._obtain_metadata()
        yield from (Post(self._context, node, self if int(node['owner']['id']) == self.userid else None) for node in
                    self._context.graphql_node_list("e31a871f7301132ceaab56507a66bbb7",
                                                    {'id': self.userid},
                                                    'https://www.instagram.com/{0}/'.format(self.username),
                                                    lambda d: d['data']['user']['edge_user_to_photos_of_you'],
                                                    self._rhx_gis))
    def get_followers(self) -> Iterator['Profile']:
        """
        Retrieve list of followers of given profile.
@@ -757,7 +769,7 @@ class Story:
           # story is a Story object
           for item in story.get_items():
               # item is a StoryItem object
-               L.download_storyitem(item, ':stores')
+               L.download_storyitem(item, ':stories')
    This class implements == and is hashable.
@@ -780,7 +792,7 @@ class Story:
        return NotImplemented
    def __hash__(self) -> int:
-        return hash(self._unique_id)
+        return hash(self.unique_id)
    @property
    def unique_id(self) -> str:
@@ -843,6 +855,83 @@ class Story:
        yield from (StoryItem(self._context, item, self.owner_profile) for item in reversed(self._node['items']))
 class Highlight(Story):
    """
    Structure representing a user's highlight with its associated story items.
    Provides methods for accessing highlight properties, as well as :meth:`Highlight.get_items` to request associated
    :class:`StoryItem` nodes. Highlights are returned by :meth:`Instaloader.get_highlights`.
    With a logged-in :class:`Instaloader` instance `L`, you may download all highlights of a :class:`Profile` instance
    USER with::
       for highlight in L.get_highlights(USER):
           # highlight is a Highlight object
           for item in highlight.get_items():
               # item is a StoryItem object
               L.download_storyitem(item, '{}/{}'.format(highlight.owner_username, highlight.title))
    This class implements == and is hashable.
    :param context: :class:`InstaloaderContext` instance used for additional queries if necessary.
    :param node: Dictionary containing the available information of the highlight as returned by Instagram.
    :param owner: :class:`Profile` instance representing the owner profile of the highlight.
    """
    def __init__(self, context: InstaloaderContext, node: Dict[str, Any], owner: Optional[Profile] = None):
        super().__init__(context, node)
        self._owner_profile = owner
        self._items = None
    def __repr__(self):
        return '<Highlight by {}: {}>'.format(self.owner_username, self.title)
    @property
    def unique_id(self) -> int:
        """A unique ID identifying this set of highlights."""
        return int(self._node['id'])
    @property
    def owner_profile(self) -> Profile:
        """:class:`Profile` instance of the highlights' owner."""
        if not self._owner_profile:
            self._owner_profile = Profile(self._context, self._node['owner'])
        return self._owner_profile
    @property
    def title(self) -> str:
        """The title of these highlights."""
        return self._node['title']
    @property
    def cover_url(self) -> str:
        """URL of the highlights' cover."""
        return self._node['cover_media']['thumbnail_src']
    @property
    def cover_cropped_url(self) -> str:
        """URL of the cropped version of the cover."""
        return self._node['cover_media_cropped_thumbnail']['url']
    def _fetch_items(self):
        if not self._items:
            self._items = self._context.graphql_query("45246d3fe16ccc6577e0bd297a5db1ab",
                                                      {"reel_ids": [], "tag_names": [], "location_ids": [],
                                                       "highlight_reel_ids": [str(self.unique_id)],
                                                       "precomposed_overlay": False})['data']['reels_media'][0]['items']
    @property
    def itemcount(self) -> int:
        """Count of items associated with the :class:`Highlight` instance."""
        self._fetch_items()
        return len(self._items)
    def get_items(self) -> Iterator[StoryItem]:
        """Retrieve all associated highlight items."""
        self._fetch_items()
        yield from (StoryItem(self._context, item, self.owner_profile) for item in self._items)
 JsonExportable = Union[Post, Profile, StoryItem]
@@ -8,14 +8,19 @@ from itertools import islice
 import instaloader
 PROFILE_WITH_HIGHLIGHTS = 325732271
 PUBLIC_PROFILE = "selenagomez"
 PUBLIC_PROFILE_ID = 460563723
 HASHTAG = "kitten"
 LOCATION = "362629379"
 OWN_USERNAME = "aandergr"
 NORMAL_MAX_COUNT = 2
 PAGING_MAX_COUNT = 15
 PRIVATE_PROFILE = "aandergr"
 # Preserve query timestamps (rate control) between tests to not get rate limited
 instaloadercontext_query_timestamps = list()
 class TestInstaloaderAnonymously(unittest.TestCase):
@@ -27,23 +32,31 @@ class TestInstaloaderAnonymously(unittest.TestCase):
                                         download_comments=True,
                                         save_metadata=True)
        self.L.context.raise_all_errors = True
        self.L.context.query_timestamps = instaloadercontext_query_timestamps
    def tearDown(self):
        global instaloadercontext_query_timestamps
        instaloadercontext_query_timestamps = self.L.context.query_timestamps
        self.L.close()
        os.chdir('/')
        print("Removing {}".format(self.dir))
        shutil.rmtree(self.dir)
    def post_paging_test(self, iterator):
        previous_post = None
        for post in islice(iterator, PAGING_MAX_COUNT):
            print(post)
            if previous_post:
                self.assertTrue(post.date_utc < previous_post.date_utc)
            previous_post = post
    @unittest.SkipTest
    def test_public_profile_download(self):
        self.L.download_profile(PUBLIC_PROFILE, profile_pic=False, fast_update=True)
        self.L.download_profile(PUBLIC_PROFILE, profile_pic=False, fast_update=True)
    def test_public_profile_paging(self):
-        for count, post in enumerate(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts()):
+        self.post_paging_test(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts())
            print(post)
            if count == PAGING_MAX_COUNT:
                break
    def test_profile_pic_download(self):
        self.L.download_profile(PUBLIC_PROFILE, profile_pic_only=True)
@@ -51,6 +64,21 @@ class TestInstaloaderAnonymously(unittest.TestCase):
    def test_hashtag_download(self):
        self.L.download_hashtag(HASHTAG, NORMAL_MAX_COUNT)
    def test_hashtag_paging(self):
        for count, post in enumerate(self.L.get_hashtag_posts(HASHTAG)):
            print(post)
            if count == PAGING_MAX_COUNT:
                break
    def test_location_download(self):
        self.L.download_location(LOCATION, NORMAL_MAX_COUNT)
    def test_location_paging(self):
        for count, post in enumerate(self.L.get_location_posts(LOCATION)):
            print(post)
            if count == PAGING_MAX_COUNT:
                break
    def test_get_id_by_username(self):
        self.assertEqual(PUBLIC_PROFILE_ID,
                         instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).userid)
@@ -65,6 +93,11 @@ class TestInstaloaderAnonymously(unittest.TestCase):
            self.assertEqual(post, post2)
            break
    def test_public_profile_tagged_paging(self):
        for post in islice(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_tagged_posts(),
                           PAGING_MAX_COUNT):
            print(post)
 class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
@@ -86,11 +119,16 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
            for item in user_story.get_items():
                print(item)
    def test_highlights_paging(self):
        for user_highlight in self.L.get_highlights(PROFILE_WITH_HIGHLIGHTS):
            print("Retrieving {} highlights \"{}\" from profile {}".format(user_highlight.itemcount,
                                                                           user_highlight.title,
                                                                           user_highlight.owner_username))
            for item in user_highlight.get_items():
                print(item)
    def test_private_profile_paging(self):
-        for count, post in enumerate(instaloader.Profile.from_username(self.L.context, PRIVATE_PROFILE).get_posts()):
+        self.post_paging_test(instaloader.Profile.from_username(self.L.context, PRIVATE_PROFILE).get_posts())
            print(post)
            if count == PAGING_MAX_COUNT:
                break
    def test_feed_download(self):
        self.L.download_feed_posts(NORMAL_MAX_COUNT)
Author	SHA1	Message	Date
Alexander Graf	2e7835850a	Version 4.2	2018-12-23 17:07:03 +01:00
Alexander Graf	89b56bdfa8	Login: Handle suspicious login and JSON error Motivated by #92.	2018-12-21 22:39:07 +01:00
Alexander Graf	1387d396dd	Second Release Candidate for Version 4.2	2018-12-21 22:01:28 +01:00
André Koch-Kramer	4ee867c61b	Implement two-factor authentication Closes #200.	2018-12-21 21:58:12 +01:00
Alexander Graf	edba6959d9	First Release Candidate for Version 4.2	2018-12-21 17:54:02 +01:00
Alexander Graf	08327c4117	Profile: don't access removed iphone info endpoint Don't access the endpoint that was used to access the HD version of the profile picture. This addresses #209.	2018-12-21 17:06:32 +01:00
Alexander Graf	4ab0fea82a	login: print that 2FA is not supported if required 2FA support is requested in issue #200, and also mentioned in #92.	2018-12-21 16:54:56 +01:00
Alexander Graf	a735b02bac	doc: Add 'versionadded' directives to new features	2018-12-21 14:43:33 +01:00
Stijn Peeters	be5d02ef3b	Location search (#212 ) * Add %location search option Search for posts for a given location ID using %[location id] as the query * Document %location search * Make pylint happy * Use correct paths for location results * Fix —help output Add description of location argument and fix output error for short help. * Add unit tests for location download * Add extra unit test for locations	2018-12-17 21:35:31 +01:00
Alexander Graf	1ab9e44104	Update sphinx_autodoc_typehints to version 1.5.2	2018-12-13 16:23:14 +01:00
André Koch-Kramer	6212d22e26	Merge branch 'master' of https://github.com/sushilicious/instaloader into v4.2-dev	2018-12-13 15:53:06 +01:00
André Koch-Kramer	aa859b9f54	Ignore pylint's cyclic-import and unnecessary-pass	2018-12-13 15:29:46 +01:00
sushilicious	ffda2265bd	Added _committed to __init__ to make pylint shut up	2018-12-09 02:12:45 -05:00
sushilicious	e753406a6c	Just in case, check whether picture exists for commit This is in addition to the other commit logic.	2018-12-09 02:12:45 -05:00
sushilicious	30555b576c	Added commit mode The commit mode ensures pictures are not corrupted when Instaloader is unexpectedly interrupted. In the case that the last picture is corrupted because of an interruption, Instaloader will redownload the picture. Since the metadata is the last object saved to disk, we can consider a post as "committed" if its json metadata file exists and is not malformed. Instaloader should download any posts which are not committed. Downside is commit mode requires metadata to be saved.	2018-12-09 02:12:45 -05:00
Alexander Graf	ece0d11915	Release of version 4.1.1	2018-10-07 15:17:51 +02:00
Alexander Graf	41e734e1b8	Fix fast_update in case of no download If Instaloader was called with --fast-update and encountered a post where nothing has been downloaded because of flags that disable download of certain types of media, the download was aborted. This behavior is now fixed. Closes #183.	2018-10-07 14:59:16 +02:00
Alexander Graf	b7ea0d5953	download_profiles: Check errors only if required Fixes #178, where with `--profile-pic-only` an error message is printed (without any further implications) when downloading profile pictures of private profiles without being logged in.	2018-09-23 10:09:08 +02:00
Alexander Graf	325e89c4b2	Print how many requests were made on 429	2018-09-06 09:17:38 +02:00
Alexander Graf	e248020927	Release of version 4.1	2018-09-02 21:57:20 +02:00
Alexander Graf	5a7b76db31	Merge branch 'v4.1-dev'	2018-09-02 21:56:16 +02:00
Alexander Graf	1a239e17cb	First Release Candidate for Version 4.1	2018-08-31 17:42:59 +02:00
Alexander Graf	bb8749b753	Adjust rate control to current rate limits	2018-08-31 17:17:23 +02:00
Alexander Graf	1f311d3e1e	First Beta Release for Version 4.1	2018-08-31 12:11:43 +02:00
Alexander Graf	862c51fa81	Profile.from_id graphql query is not rate limited	2018-08-30 13:59:46 +02:00
Alexander Graf	cc15cb5857	Cache and reuse profiles for Profile.from_id()	2018-08-30 13:52:16 +02:00
Alexander Graf	18707ee232	docs: Better explanation for is_video Closes #173.	2018-08-30 10:16:56 +02:00
Alexander Graf	0e534ba519	Use username/:tagged as target for tagged posts Subdirs in profile folders were introduced with --highlights, and moving :tagged posts there allows to further call instaloader as instaloader [flags] */ (#154)	2018-08-30 09:57:42 +02:00
Alexander Graf	084424e856	docs/codesnippets: Link to source tree on github	2018-08-30 09:18:58 +02:00
Alexander Graf	567a04a661	format_filename method to apply filename pattern	2018-08-30 09:13:49 +02:00
Alexander Graf	1394e8e9f5	First Alpha Release for Version 4.1	2018-08-24 12:51:52 +02:00
Alexander Graf	86204d165d	Merge branch 'master' into v4.1-dev	2018-08-24 12:45:12 +02:00
Alexander Graf	d0e2ac4388	docs/install: Suggest pre-release rather than git	2018-08-24 12:42:51 +02:00
Alexander Graf	60d47be2f3	Unit test for Instaloader.get_highlights()	2018-08-24 12:33:28 +02:00
Alexander Graf	f988762cb1	download_profiles: raise_errors parameter With raise_errors=True it behaves like now-deprecated download_profile().	2018-08-24 12:21:45 +02:00
Alexander Graf	b443cc6654	Minor doc fixes and improvements	2018-08-24 11:53:14 +02:00
André Koch-Kramer	584c69d93c	Update docs and docstrings concerning highlights	2018-08-24 00:23:19 +02:00
André Koch-Kramer	cbdd85ef07	Highlights downloadable through CLI By using --highlights all available highlight stories of target profiles will get downloaded. Closes #162.	2018-08-23 23:17:48 +02:00
André Koch-Kramer	2e517e972f	Extend _PostPathFormatter to replace more chars On Windows, all forbidden characters now get replaced with similar looking unicode chars.	2018-08-23 22:19:08 +02:00
André Koch-Kramer	54572fb1fc	Make the Highlight stuff accept Profile objects download_highlights(), get_highlights() and the Highlight class now accept and use the owner's Profile rather than creating it themselves.	2018-08-23 22:19:08 +02:00
André Koch-Kramer	158c1433bb	Revert accidental change of comment.id to comment.unique_id	2018-08-23 22:19:08 +02:00
Alexander Graf	43f52198ff	Fix docs building and show Highlights inheritance	2018-08-23 22:19:08 +02:00
André Koch-Kramer	91d5d5f867	Add class and functions for downloading highlights Requested in #162.	2018-08-23 22:19:08 +02:00
Alexander Graf	0dcc912987	doc: Note new features with versionadded	2018-08-23 17:25:28 +02:00
Alexander Graf	180d398ad7	docs: Exclude sphinx-1.7.7 to fix travis build	2018-08-23 16:14:02 +02:00
Alexander Graf	f262c4347f	Create issue templates	2018-08-23 15:11:09 +02:00
Alexander Graf	b91e73b090	set docs font size to 16px	2018-08-23 14:44:37 +02:00
Alexander Graf	e388a1c966	--no-posts; Deprecate --{profile-pic,stories}-only	2018-08-20 16:29:14 +02:00
Alexander Graf	0f0ac13d72	reorder --help output	2018-08-20 14:27:58 +02:00
Alexander Graf	06b7edd6d5	--no-pictures flag to not download post pictures Closes #131.	2018-08-20 14:27:58 +02:00
Alexander Graf	5f57345f1b	"instaloader -- -SHORTCODE" to fetch single post Closes #129.	2018-08-20 14:27:58 +02:00
Alexander Graf	f7e6632f68	Document --tagged{,-only} in docs/cli-options.rst	2018-08-20 10:44:20 +02:00
AndCycle	9258b84695	Allow reading arguments from file Squashed commit of the following (pr #161): commit `58235a99b0` Author: AndCycle <andcycle@andcycle.idv.tw> Date: Thu Aug 16 17:57:26 2018 +0800 doc: proper fix rst syntax commit `6620e3583c` Author: AndCycle <andcycle@andcycle.idv.tw> Date: Thu Aug 16 17:29:25 2018 +0800 doc: fix syntax commit `7a048a3c04` Author: AndCycle <andcycle@andcycle.idv.tw> Date: Thu Aug 16 17:09:26 2018 +0800 doc: add info about read args from file. commit `c4ceaf9636` Author: AndCycle <andcycle@andcycle.idv.tw> Date: Thu Aug 16 17:05:07 2018 +0800 alter `fromfile_prefix_chars` to plus sign to avoid conflict. commit `d469b52b95` Author: AndCycle <andcycle@andcycle.idv.tw> Date: Mon Aug 6 19:10:23 2018 +0800 * Allow reading arguments from file function referenced from instagram-scraper, which provide a convenient way to hide sensitive information.	2018-08-20 09:00:51 +02:00
Lars Lindqvist	e00d77c234	Basic CLI support for tagged posts. Squashed commit of the following (pr #154): commit `8fd56c379f` Merge: `08f0ee7` `a3777ca` Author: Lars Lindqvist <e5150@users.noreply.github.com> Date: Wed Aug 15 20:23:23 2018 +0200 Merge branch 'master' into master commit `08f0ee795c` Merge: `700b3a8` `dcea0e9` Author: Lars Lindqvist <e5150@users.noreply.github.com> Date: Sun Aug 5 15:25:55 2018 +0200 Merge branch 'master' into master commit `700b3a8d09` Author: Lars Lindqvist <lars.lindqvist@yandex.ru> Date: Sat Aug 4 16:26:59 2018 +0200 Basic CLI support for tagged posts. commit `5e3cd10cbc` Merge: `af564f5` `92653dc` Author: Lars Lindqvist <e5150@users.noreply.github.com> Date: Fri Aug 3 19:33:24 2018 +0200 Merge branch 'master' into master commit `af564f5174` Author: Lars Lindqvist <e5150@users.noreply.github.com> Date: Fri Aug 3 19:25:57 2018 +0200 Fix owner_profile for Profile.get_tagged_posts() commit `3cde1f7db4` Author: Lars Lindqvist <e5150@users.noreply.github.com> Date: Thu Jul 26 19:51:33 2018 +0200 Add meth:get_tagged_posts to Profile	2018-08-20 08:50:29 +02:00
Alexander Graf	a3777cad5d	Download profile pic unless --no-profile-pic Fixes #165. To download (S = Stories, P = Profile Pic, p = Posts) p --no-profile-pic P --profile-pic-only Pp (default) S --no-profile-pic --stories-only Sp --no-profile-pic --stories SP --stories-only SPp --stories	2018-08-13 17:57:28 +02:00
André Koch-Kramer	8303f1d1c5	Release of version 4.0.8	2018-08-07 20:07:41 +02:00
Alexander Graf	998333b7cd	Revert "Mention IRC channel #instaloader on Freenode" This reverts commit `8423e3bd00`. The channel did not have as much activity as expected and an IRC channel does not work if there are not at least a few people in it.	2018-08-07 19:57:10 +02:00
André Koch-Kramer	77ca78e977	Fix logging out when download profile_pic fails Add try-finally-block to InstaloaderContext.anonymous_copy(). Fixes #159.	2018-08-07 19:16:08 +02:00
Alexander Graf	dcea0e9672	Release of version 4.0.7	2018-08-05 09:37:18 +02:00
Alexander Graf	809b23a996	doc: Mark Profile.has_higlight_reels as deprecated	2018-08-05 09:13:49 +02:00
Alexander Graf	c634f21dcc	Fix anon-retry for profiles that blocked viewer This was broken with `50a5330fec`.	2018-08-05 09:01:22 +02:00
Alexander Graf	a070ed42ae	get_stories(): Fetch in chunks of 100 users It seems there is a limit of how many user stories can be requested at once. Now, Instaloader.get_stories() chunks the stories request. Fixes #157 and fixes #134.	2018-08-05 08:06:53 +02:00
Alexander Graf	a34d750152	rate handling: Show %H:%M when request is retried	2018-08-05 08:02:35 +02:00
Lars Lindqvist	bfa5d8ed41	Fix :stories `50a5330fec` breaks `:stories` if userids is None (non-iterable).	2018-08-05 03:29:24 +02:00
Alexander Graf	7f81985911	_ArbitraryItemFormatter: Allow chaining attributes Override get_value() rather than get_field(), to allow chaining of item attributes, such as --post-metadata-txt={owner_profile.profile_pic_url}. Motivated by #151.	2018-08-04 11:41:19 +02:00
Lars Lindqvist	b5aa268344	Fix owner_profile for Profile.get_tagged_posts()	2018-08-04 09:41:23 +02:00
André Koch-Kramer	92653dcdb9	Release of version 4.0.6	2018-08-02 16:55:59 +02:00
André Koch-Kramer	1b5c12c8fa	Add unittest for Profile.get_tagged_posts()	2018-08-02 16:48:06 +02:00
Lars Lindqvist	bc29ffbd44	Add meth:get_tagged_posts to Profile	2018-08-02 16:41:51 +02:00
André Koch-Kramer	50a5330fec	Change download of stories when not using :stories If using --stories or --stories-only the stories got donwloaded along with the profiles one by one. Now, the stories get downloaded in a similar aproach like when using the :stories target, i.e. download_stories() gets only called once. Profile.has_highlight_reels is broken and now always returns true. This fixes #153.	2018-08-02 16:20:27 +02:00
André Koch-Kramer	1739844758	Fix login by allowing more http headers Because Instagram is bombarding over hundret of headers on login it is necessary to increase http.client._MAXHEADERS. Fixes #152.	2018-07-30 17:50:22 +02:00
André Koch-Kramer	0ebb223214	Added BTC address to donations section Motivated by #130.	2018-07-23 23:36:46 +02:00
Alexander Graf	3d4109ded3	travis disable pylint's no-else-return	2018-07-22 11:04:11 +02:00
Alexander Graf	8423e3bd00	Mention IRC channel #instaloader on Freenode	2018-07-22 10:55:29 +02:00
Alexander Graf	5b0f8b75cc	Add AUTHORS.md listing Instaloader's authors	2018-07-22 10:54:54 +02:00
Alexander Graf	3cb33603e5	doc: Use Instaloader brand colors for links	2018-07-09 21:32:18 +02:00
Alexander Graf	47db7f7ffa	Release of version 4.0.5	2018-07-09 10:18:27 +02:00
Alexander Graf	2567bec975	unittests: Test order of Profile.get_posts() Motivated by #140.	2018-07-09 10:08:18 +02:00
Alexander Graf	2df1419d68	Fix file extension of profile picture Fix #142.	2018-07-09 10:07:16 +02:00
Alexander Graf	8211c9daf1	Documentation Improvements - Fix Spelling - codesnippets: Improve introduction texts; Write subsection about metadata JSON files	2018-06-17 18:10:19 +02:00
Alexander Graf	969d4e3aa4	Post: Write all ever-obtained metadata into JSON The Post._asdict() code assumed that Post._node always is a subset of Post._full_metadata_dict. However, it turned out, that `edge_liked_by`, `thumbnail_src` and `thumbnail_resources` may be contained in Post._node, but not in Post._full_metadata_dict. Now the keys of both dictionaries are saved in the JSON file. Fixes #137.	2018-06-15 17:07:28 +02:00
Alexander Graf	4cf1ae847d	Mention donation option in doc (Suggested by #130)	2018-06-13 08:39:29 +02:00
André Koch-Kramer	a276a0dc59	Release of version 4.0.4	2018-06-07 15:46:47 +02:00
André Koch-Kramer	cc34b8340f	Fix detection of sidecars and videos This was noticeable when downloading hashtags. Fixes #133	2018-06-07 15:32:23 +02:00
Alexander Graf	45fe509ac4	doc: Link codesnippets in basic-usage	2018-06-04 12:06:41 +02:00
Alexander Graf	ae492ed68b	New doc section: codesnippets / Advanced Examples Presents code examples that use the instaloader module for more advanced tasks than what is possible with the Instaloader command line interface. Presents #46, #56, #110, #113, #120, #121.	2018-06-04 11:57:28 +02:00
Alexander Graf	0e433294ad	unittests: Preserve query timestamps between tests This avoids getting rate limited while running all the tests.	2018-05-30 13:16:16 +02:00
Alexander Graf	a807408d8f	Release of version 4.0.3	2018-05-23 08:57:36 +02:00
Alexander Graf	86035b8d2d	Fix obtaining csrftoken for login Fixes #122. Further fixes processing of login JSON response regarding error handling.	2018-05-23 08:50:37 +02:00
Alexander Graf	a3eb93c21a	Profile: Cache iphone data structure if queried Profile.get_profile_pic_url() and Profile.has_highlight_reels did the same query without remembering the response, resulting in that query being made twice when downloading stories and profile pic of a profile.	2018-05-19 09:52:07 +02:00