Release of version 4.10.1

It fails due to a problem with installing dependencies, which is already
fixed in a3dc75c04e.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,32 @@
+---
+name: 🐛 Bug report
+about: Create a report to help us improve 🤕
+labels: bug
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+(e.g. Instaloader command line)
+
+**Expected behavior**
+A clear and concise description of what you expected to happen (even if it seems obvious).
+
+**Error messages and tracebacks**
+If applicable, add error messages and tracebacks to help explain your problem.
+
+**Instaloader version**
+`instaloader --version` output.
+
+**Additional context**
+Add any other context about the problem here.
+
+<!--
+Also see https://instaloader.github.io/contributing.html for how to report a bug.
+
+Please understand that a bug issue that lacks the required information cannot be
+processed.
+-->

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Contributing to Instaloader
+    url: https://instaloader.github.io/contributing.html
+    about: Guidelines for how to report an issue.
+  - name: Instaloader Troubleshooting
+    url: https://instaloader.github.io/troubleshooting.html
+    about: Many common problems and their workarounds are described here.
+  - name: Instaloader Documentation
+    url: https://instaloader.github.io/
+    about: Many questions might be already answered here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,31 @@
+---
+name: 🚀 Feature Suggestion
+about: Suggest an idea for this project 💡
+labels: "feature suggestion"
+
+---
+
+**Provide us a use case of the feature**
+How could the user invoke the new function? Which problem would it solve?
+If new information is obtained, how would it be further processed?
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+How do you achieve the here-described goal without the feature you are requesting?
+
+**If the feature request is accepted, would you be willing to submit a pull request?**
+Yes / No
+<!-- Help can be provided if you need assistance submitting a PR -->
+
+**Additional context**
+Add any other context about the feature request here.
+
+<!-- 
+Also see https://instaloader.github.io/contributing.html for how to suggest a feature.
+
+Please understand that a feature suggestion issue that lacks the required information cannot be
+processed.
+-->

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,18 @@
+---
+name: ❓ Question
+about: If you have a question about how to use Instaloader that's not well covered by our documentation
+labels: question
+
+---
+
+Your question here...
+
+<!--
+People will be better able and more motivated to provide help if your question
+
+ - has a concise and meaningful title,
+ - is not covered by the documentation: https://instaloader.github.io/,
+ - is actually related to Instaloader (this is not a Python help forum),
+ - has not already been answered here or on other sites such as Stack Overflow: https://stackoverflow.com/questions/tagged/instaloader,
+ - is written in a polite and welcoming tone.
+-->

.github/funding.yml

@@ -0,0 +1,2 @@
+github: [aandergr]
+custom: "https://instaloader.github.io/sponsors.html"

.github/pull_request_template.md

@@ -0,0 +1,18 @@
+
+
+<!--
+Thanks for your willingness to contribute to Instaloader! Please briefly
+describe
+
+- A motivation for this change, e.g.
+  - Fixes # .
+  - More general: What problem does the pull request solve?
+
+- The changes proposed in this pull request
+
+- The completeness of this change
+  - Is it just a proof of concept?
+  - Is the documentation updated (if appropriate)?
+  - Do you consider it ready to be merged or is it a draft?
+  - Can we help you at some point?
+-->

.github/workflows/documentation.yml

@@ -0,0 +1,31 @@
+name: "Documentation"
+
+on: [push, pull_request]
+
+jobs:
+  docs:
+    name: "Documentation"
+    runs-on: ubuntu-latest
+    steps:
+      - name: "Checkout Instaloader Repository"
+        uses: actions/checkout@v2
+        with:
+          fetch-depth: 0  # needed for building docs
+      - name: "Setup Python"
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.8
+      - name: "Install Dependencies"
+        run: |
+          python -m pip install pipenv==2023.2.4
+          pipenv --python `python --version | grep -Eo '3\.[0-9]+'` sync --dev
+      - name: "Build Documentation"
+        run: pipenv --python `python --version | grep -Eo '3\.[0-9]+'` run make -C docs html SPHINXOPTS="-W -n"
+      - name: "Deploy Documentation"
+        if: github.event_name == 'push' && github.ref == 'refs/heads/master'
+        uses: JamesIves/github-pages-deploy-action@v4.2.5
+        with:
+          branch: master
+          repository-name: instaloader/instaloader.github.io
+          folder: docs/_build/html
+          token: ${{ secrets.PAGES_DEPLOYMENT_TOKEN }}

.github/workflows/lint.yml

@@ -0,0 +1,27 @@
+name: PyLint and MyPy
+
+on: [push, pull_request]
+
+jobs:
+  lint:
+    name: PyLint and MyPy
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.8", "3.9", "3.10"]
+    steps:
+      - name: Checkout Instaloader Repository
+        uses: actions/checkout@v2
+      - name: Setup Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install Dependencies
+        run: |
+          python -m pip install pipenv==2023.2.4
+          pipenv --python `python --version | grep -Eo '3\.[0-9]+'` sync --dev
+      - name: PyLint
+        run: pipenv --python `python --version | grep -Eo '3\.[0-9]+'` run pylint instaloader
+      - name: MyPy
+        run: pipenv --python `python --version | grep -Eo '3\.[0-9]+'` run mypy -m instaloader

.github/workflows/packages.yml

@@ -0,0 +1,36 @@
+name: "Publish Packages"
+
+on:
+  push:
+    tags:
+      - v*
+
+jobs:
+  publish:
+    name: "Publish Package"
+    runs-on: ubuntu-latest
+    steps:
+      - name: "Checkout Repository"
+        uses: actions/checkout@v2
+      - name: "Setup Python"
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.8
+      - name: "Get the tagged version"
+        id: get_version
+        env:
+          GITHUB_REF: ${{ github.ref }}
+        run: echo ::set-output name=VERSION::${GITHUB_REF/refs\/tags\/v/}
+        shell: bash
+      - name: "Generate Python Package"
+        run: python setup.py sdist
+      - name: "Publish Distribution to PyPI"
+        uses: pypa/gh-action-pypi-publish@master
+        with:
+          password: ${{ secrets.PYPI_PASSWORD }}
+      - name: "Publish Distribution to AUR"
+        run: deploy/arch/deploy.sh $VERSION_TAG
+        env:
+          VERSION_TAG: v${{ steps.get_version.outputs.VERSION }}
+          AUR_KEY: ${{ secrets.AUR_KEY }}
+          AUR_IV: ${{ secrets.AUR_IV }}

.github/workflows/stale.yml

@@ -0,0 +1,34 @@
+name: Mark stale issues and pull requests
+
+on:
+  schedule:
+  - cron: "10 14 * * *"
+
+jobs:
+  stale:
+
+    name: Mark stale issues and pull requests
+    runs-on: ubuntu-latest
+    
+    steps:
+    - uses: actions/stale@v3
+      with:
+        repo-token: ${{ secrets.GITHUB_TOKEN }}
+        only-labels: 'question'
+        stale-issue-message: 'There has been no activity on this question for an extended period of time. This issue will be closed after further 14 days of inactivity.'
+        stale-issue-label: 'stale'
+        exempt-issue-labels: 'leave open'
+        days-before-stale: 21
+        days-before-close: -1
+        remove-stale-when-updated: false
+    - uses: actions/stale@v1
+      with:
+        repo-token: ${{ secrets.GITHUB_TOKEN }}
+        stale-issue-message: 'There has been no activity on this issue for an extended period of time. This issue will be closed after further 14 days of inactivity.'
+        stale-pr-message: 'There has been no activity on this Pull Request for an extended period of time. This Pull Request will be closed after further 14 days of inactivity.'
+        stale-issue-label: 'stale'
+        stale-pr-label: 'stale'
+        exempt-issue-label: 'leave open'
+        exempt-pr-label: 'leave open'
+        days-before-stale: 189
+        days-before-close: 14

.github/workflows/windows_exe.yml

@@ -0,0 +1,48 @@
+name: Windows EXE build
+
+on: [push, pull_request]
+
+jobs:
+  build:
+    name: Windows EXE build
+    runs-on: windows-latest
+    steps:
+      - name: Checkout Instaloader repository
+        uses: actions/checkout@v2
+      - name: Setup Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: "3.9"
+          architecture: x64
+      - name: Get the tagged version
+        id: get_version
+        env:
+          GITHUB_REF: ${{ github.ref }}
+        run: echo ::set-output name=VERSION::${GITHUB_REF/refs\/tags\/v/}
+        shell: bash
+      - name: Run custom python script for EXE creation
+        env:
+          VERSION_TAG: v${{ steps.get_version.outputs.VERSION }}
+        run: python deploy/windows/create_exe.py
+      - name: Create draft release
+        if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+        id: create_release
+        uses: actions/create-release@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          tag_name: v${{ steps.get_version.outputs.VERSION }}
+          release_name: Version ${{ steps.get_version.outputs.VERSION }}
+          draft: true
+          prerelease: contains(steps.get_version.outputs.VERSION, 'a') || contains(steps.get_version.outputs.VERSION, 'b') || contains(steps.get_version.outputs.VERSION, 'c')
+      - name: Upload EXE as release asset
+        if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+        id: upload-release-asset
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: ./instaloader-v${{ steps.get_version.outputs.VERSION }}-windows-standalone.zip
+          asset_name: instaloader-v${{ steps.get_version.outputs.VERSION }}-windows-standalone.zip
+          asset_content_type: application/zip

.gitignore

@@ -1 +1,2 @@
 __pycache__/
+.mypy_cache/

.pylintrc

@@ -0,0 +1,646 @@
+[MAIN]
+
+# Analyse import fallback blocks. This can be used to support both Python 2 and
+# 3 compatible code, which means that the block might have code that exists
+# only in one or another interpreter, leading to false positives when analysed.
+analyse-fallback-blocks=no
+
+# Clear in-memory caches upon conclusion of linting. Useful if running pylint
+# in a server-like mode.
+clear-cache-post-run=no
+
+# Load and enable all available extensions. Use --list-extensions to see a list
+# all available extensions.
+#enable-all-extensions=
+
+# In error mode, messages with a category besides ERROR or FATAL are
+# suppressed, and no reports are done by default. Error mode is compatible with
+# disabling specific errors.
+#errors-only=
+
+# Always return a 0 (non-error) status code, even if lint errors are found.
+# This is primarily useful in continuous integration scripts.
+#exit-zero=
+
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code.
+extension-pkg-allow-list=
+
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
+# for backward compatibility.)
+extension-pkg-whitelist=
+
+# Return non-zero exit code if any of these messages/categories are detected,
+# even if score is above --fail-under value. Syntax same as enable. Messages
+# specified are enabled, while categories only check already-enabled messages.
+fail-on=
+
+# Specify a score threshold under which the program will exit with error.
+fail-under=10
+
+# Interpret the stdin as a python script, whose filename needs to be passed as
+# the module_or_package argument.
+#from-stdin=
+
+# Files or directories to be skipped. They should be base names, not paths.
+ignore=CVS
+
+# Add files or directories matching the regular expressions patterns to the
+# ignore-list. The regex matches against paths and can be in Posix or Windows
+# format. Because '\\' represents the directory delimiter on Windows systems,
+# it can't be used as an escape character.
+ignore-paths=
+
+# Files or directories matching the regular expression patterns are skipped.
+# The regex matches against base names, not paths. The default value ignores
+# Emacs file locks
+ignore-patterns=^\.#
+
+# List of module names for which member attributes should not be checked
+# (useful for modules/projects where namespaces are manipulated during runtime
+# and thus existing member attributes cannot be deduced by static analysis). It
+# supports qualified module names, as well as Unix pattern matching.
+ignored-modules=
+
+# Python code to execute, usually for sys.path manipulation such as
+# pygtk.require().
+#init-hook=
+
+# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
+# number of processors available to use, and will cap the count on Windows to
+# avoid hangs.
+jobs=1
+
+# Control the amount of potential inferred values when inferring a single
+# object. This can help the performance when dealing with large functions or
+# complex, nested conditions.
+limit-inference-results=100
+
+# List of plugins (as comma separated values of python module names) to load,
+# usually to register additional checkers.
+load-plugins=
+
+# Pickle collected data for later comparisons.
+persistent=yes
+
+# Minimum Python version to use for version dependent checks. Will default to
+# the version used to run pylint.
+py-version=3.8
+
+# Discover python modules and packages in the file system subtree.
+recursive=no
+
+# When enabled, pylint would attempt to guess common misconfiguration and emit
+# user-friendly hints instead of false-positive error messages.
+suggestion-mode=yes
+
+# Allow loading of arbitrary C extensions. Extensions are imported into the
+# active Python interpreter and may run arbitrary code.
+unsafe-load-any-extension=no
+
+# In verbose mode, extra non-checker-related info will be displayed.
+#verbose=
+
+
+[BASIC]
+
+# Naming style matching correct argument names.
+argument-naming-style=snake_case
+
+# Regular expression matching correct argument names. Overrides argument-
+# naming-style. If left empty, argument names will be checked with the set
+# naming style.
+#argument-rgx=
+
+# Naming style matching correct attribute names.
+attr-naming-style=snake_case
+
+# Regular expression matching correct attribute names. Overrides attr-naming-
+# style. If left empty, attribute names will be checked with the set naming
+# style.
+#attr-rgx=
+
+# Bad variable names which should always be refused, separated by a comma.
+bad-names=foo,
+          bar,
+          baz,
+          toto,
+          tutu,
+          tata
+
+# Bad variable names regexes, separated by a comma. If names match any regex,
+# they will always be refused
+bad-names-rgxs=
+
+# Naming style matching correct class attribute names.
+class-attribute-naming-style=any
+
+# Regular expression matching correct class attribute names. Overrides class-
+# attribute-naming-style. If left empty, class attribute names will be checked
+# with the set naming style.
+#class-attribute-rgx=
+
+# Naming style matching correct class constant names.
+class-const-naming-style=UPPER_CASE
+
+# Regular expression matching correct class constant names. Overrides class-
+# const-naming-style. If left empty, class constant names will be checked with
+# the set naming style.
+#class-const-rgx=
+
+# Naming style matching correct class names.
+class-naming-style=PascalCase
+
+# Regular expression matching correct class names. Overrides class-naming-
+# style. If left empty, class names will be checked with the set naming style.
+#class-rgx=
+
+# Naming style matching correct constant names.
+const-naming-style=UPPER_CASE
+
+# Regular expression matching correct constant names. Overrides const-naming-
+# style. If left empty, constant names will be checked with the set naming
+# style.
+#const-rgx=
+
+# Minimum line length for functions/classes that require docstrings, shorter
+# ones are exempt.
+docstring-min-length=-1
+
+# Naming style matching correct function names.
+function-naming-style=snake_case
+
+# Regular expression matching correct function names. Overrides function-
+# naming-style. If left empty, function names will be checked with the set
+# naming style.
+#function-rgx=
+
+# Good variable names which should always be accepted, separated by a comma.
+good-names=i,
+           j,
+           k,
+           ex,
+           Run,
+           _
+
+# Good variable names regexes, separated by a comma. If names match any regex,
+# they will always be accepted
+good-names-rgxs=
+
+# Include a hint for the correct naming format with invalid-name.
+include-naming-hint=no
+
+# Naming style matching correct inline iteration names.
+inlinevar-naming-style=any
+
+# Regular expression matching correct inline iteration names. Overrides
+# inlinevar-naming-style. If left empty, inline iteration names will be checked
+# with the set naming style.
+#inlinevar-rgx=
+
+# Naming style matching correct method names.
+method-naming-style=snake_case
+
+# Regular expression matching correct method names. Overrides method-naming-
+# style. If left empty, method names will be checked with the set naming style.
+#method-rgx=
+
+# Naming style matching correct module names.
+module-naming-style=snake_case
+
+# Regular expression matching correct module names. Overrides module-naming-
+# style. If left empty, module names will be checked with the set naming style.
+#module-rgx=
+
+# Colon-delimited sets of names that determine each other's naming style when
+# the name regexes allow several styles.
+name-group=
+
+# Regular expression which should only match function or class names that do
+# not require a docstring.
+no-docstring-rgx=^_
+
+# List of decorators that produce properties, such as abc.abstractproperty. Add
+# to this list to register other decorators that produce valid properties.
+# These decorators are taken in consideration only for invalid-name.
+property-classes=abc.abstractproperty
+
+# Regular expression matching correct type variable names. If left empty, type
+# variable names will be checked with the set naming style.
+#typevar-rgx=
+
+# Naming style matching correct variable names.
+variable-naming-style=snake_case
+
+# Regular expression matching correct variable names. Overrides variable-
+# naming-style. If left empty, variable names will be checked with the set
+# naming style.
+#variable-rgx=
+
+
+[CLASSES]
+
+# Warn about protected attribute access inside special methods
+check-protected-access-in-special-methods=no
+
+# List of method names used to declare (i.e. assign) instance attributes.
+defining-attr-methods=__init__,
+                      __new__,
+                      setUp,
+                      __post_init__
+
+# List of member names, which should be excluded from the protected access
+# warning.
+exclude-protected=_asdict,
+                  _fields,
+                  _replace,
+                  _source,
+                  _make
+
+# List of valid names for the first argument in a class method.
+valid-classmethod-first-arg=cls
+
+# List of valid names for the first argument in a metaclass class method.
+valid-metaclass-classmethod-first-arg=mcs
+
+
+[DESIGN]
+
+# List of regular expressions of class ancestor names to ignore when counting
+# public methods (see R0903)
+exclude-too-few-public-methods=
+
+# List of qualified class names to ignore when counting class parents (see
+# R0901)
+ignored-parents=
+
+# Maximum number of arguments for function / method.
+max-args=5
+
+# Maximum number of attributes for a class (see R0902).
+max-attributes=7
+
+# Maximum number of boolean expressions in an if statement (see R0916).
+max-bool-expr=5
+
+# Maximum number of branch for function / method body.
+max-branches=12
+
+# Maximum number of locals for function / method body.
+max-locals=15
+
+# Maximum number of parents for a class (see R0901).
+max-parents=7
+
+# Maximum number of public methods for a class (see R0904).
+max-public-methods=20
+
+# Maximum number of return / yield for function / method body.
+max-returns=6
+
+# Maximum number of statements in function / method body.
+max-statements=50
+
+# Minimum number of public methods for a class (see R0903).
+min-public-methods=2
+
+
+[EXCEPTIONS]
+
+# Exceptions that will emit a warning when caught.
+overgeneral-exceptions=builtins.BaseException,builtins.Exception
+
+
+[FORMAT]
+
+# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
+expected-line-ending-format=
+
+# Regexp for a line that is allowed to be longer than the limit.
+ignore-long-lines=^\s*(# )?<?https?://\S+>?$
+
+# Number of spaces of indent required inside a hanging or continued line.
+indent-after-paren=4
+
+# String used as indentation unit. This is usually "    " (4 spaces) or "\t" (1
+# tab).
+indent-string='    '
+
+# Maximum number of characters on a single line.
+max-line-length=120
+
+# Maximum number of lines in a module.
+max-module-lines=1000
+
+# Allow the body of a class to be on the same line as the declaration if body
+# contains single statement.
+single-line-class-stmt=no
+
+# Allow the body of an if to be on the same line as the test if there is no
+# else.
+single-line-if-stmt=no
+
+
+[IMPORTS]
+
+# List of modules that can be imported at any level, not just the top level
+# one.
+allow-any-import-level=
+
+# Allow explicit reexports by alias from a package __init__.
+allow-reexport-from-package=no
+
+# Allow wildcard imports from modules that define __all__.
+allow-wildcard-with-all=no
+
+# Deprecated modules which should not be used, separated by a comma.
+deprecated-modules=
+
+# Output a graph (.gv or any supported image format) of external dependencies
+# to the given file (report RP0402 must not be disabled).
+ext-import-graph=
+
+# Output a graph (.gv or any supported image format) of all (i.e. internal and
+# external) dependencies to the given file (report RP0402 must not be
+# disabled).
+import-graph=
+
+# Output a graph (.gv or any supported image format) of internal dependencies
+# to the given file (report RP0402 must not be disabled).
+int-import-graph=
+
+# Force import order to recognize a module as part of the standard
+# compatibility libraries.
+known-standard-library=
+
+# Force import order to recognize a module as part of a third party library.
+known-third-party=enchant
+
+# Couples of modules and preferred modules, separated by a comma.
+preferred-modules=
+
+
+[LOGGING]
+
+# The type of string formatting that logging methods do. `old` means using %
+# formatting, `new` is for `{}` formatting.
+logging-format-style=old
+
+# Logging modules to check that the string format arguments are in logging
+# function parameter format.
+logging-modules=logging
+
+
+[MESSAGES CONTROL]
+
+# Only show warnings with the listed confidence levels. Leave empty to show
+# all. Valid levels: HIGH, CONTROL_FLOW, INFERENCE, INFERENCE_FAILURE,
+# UNDEFINED.
+confidence=HIGH,
+           CONTROL_FLOW,
+           INFERENCE,
+           INFERENCE_FAILURE,
+           UNDEFINED
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once). You can also use "--disable=all" to
+# disable everything first and then re-enable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use "--disable=all --enable=classes
+# --disable=W".
+disable=bad-inline-option,
+        consider-using-f-string,
+        cyclic-import,
+        deprecated-pragma,
+        file-ignored,
+        invalid-name,
+        locally-disabled,
+        missing-class-docstring,
+        missing-function-docstring,
+        missing-module-docstring,
+        no-else-raise,
+        no-else-return,
+        raw-checker-failed,
+        suppressed-message,
+        too-many-arguments,
+        too-many-branches,
+        too-many-instance-attributes,
+        too-many-lines,
+        too-many-locals,
+        too-many-public-methods,
+        too-many-statements,
+        unnecessary-lambda-assignment,
+        unnecessary-pass,
+        unspecified-encoding,
+        unused-wildcard-import,
+        use-dict-literal,
+        use-list-literal,
+        use-symbolic-message-instead,
+        useless-suppression,
+        wildcard-import
+
+# Enable the message, report, category or checker with the given id(s). You can
+# either give multiple identifier separated by comma (,) or put this option
+# multiple time (only on the command line, not in the configuration file where
+# it should appear only once). See also the "--disable" option for examples.
+enable=c-extension-no-member
+
+
+[METHOD_ARGS]
+
+# List of qualified names (i.e., library.method) which require a timeout
+# parameter e.g. 'requests.api.get,requests.api.post'
+timeout-methods=requests.api.delete,requests.api.get,requests.api.head,requests.api.options,requests.api.patch,requests.api.post,requests.api.put,requests.api.request
+
+
+[MISCELLANEOUS]
+
+# List of note tags to take in consideration, separated by a comma.
+notes=FIXME,
+      XXX,
+      TODO
+
+# Regular expression of note tags to take in consideration.
+notes-rgx=
+
+
+[REFACTORING]
+
+# Maximum number of nested blocks for function / method body
+max-nested-blocks=5
+
+# Complete name of functions that never returns. When checking for
+# inconsistent-return-statements if a never returning function is called then
+# it will be considered as an explicit return statement and no message will be
+# printed.
+never-returning-functions=sys.exit,argparse.parse_error
+
+
+[REPORTS]
+
+# Python expression which should return a score less than or equal to 10. You
+# have access to the variables 'fatal', 'error', 'warning', 'refactor',
+# 'convention', and 'info' which contain the number of messages in each
+# category, as well as 'statement' which is the total number of statements
+# analyzed. This score is used by the global evaluation report (RP0004).
+evaluation=max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))
+
+# Template used to display messages. This is a python new-style format string
+# used to format the message information. See doc for all details.
+msg-template=
+
+# Set the output format. Available formats are text, parseable, colorized, json
+# and msvs (visual studio). You can also give a reporter class, e.g.
+# mypackage.mymodule.MyReporterClass.
+#output-format=
+
+# Tells whether to display a full report or only the messages.
+reports=no
+
+# Activate the evaluation score.
+score=yes
+
+
+[SIMILARITIES]
+
+# Comments are removed from the similarity computation
+ignore-comments=yes
+
+# Docstrings are removed from the similarity computation
+ignore-docstrings=yes
+
+# Imports are removed from the similarity computation
+ignore-imports=yes
+
+# Signatures are removed from the similarity computation
+ignore-signatures=yes
+
+# Minimum lines number of a similarity.
+min-similarity-lines=4
+
+
+[SPELLING]
+
+# Limits count of emitted suggestions for spelling mistakes.
+max-spelling-suggestions=4
+
+# Spelling dictionary name. Available dictionaries: none. To make it work,
+# install the 'python-enchant' package.
+spelling-dict=
+
+# List of comma separated words that should be considered directives if they
+# appear at the beginning of a comment and should not be checked.
+spelling-ignore-comment-directives=fmt: on,fmt: off,noqa:,noqa,nosec,isort:skip,mypy:
+
+# List of comma separated words that should not be checked.
+spelling-ignore-words=
+
+# A path to a file that contains the private dictionary; one word per line.
+spelling-private-dict-file=
+
+# Tells whether to store unknown words to the private dictionary (see the
+# --spelling-private-dict-file option) instead of raising a message.
+spelling-store-unknown-words=no
+
+
+[STRING]
+
+# This flag controls whether inconsistent-quotes generates a warning when the
+# character used as a quote delimiter is used inconsistently within a module.
+check-quote-consistency=no
+
+# This flag controls whether the implicit-str-concat should generate a warning
+# on implicit string concatenation in sequences defined over several lines.
+check-str-concat-over-line-jumps=no
+
+
+[TYPECHECK]
+
+# List of decorators that produce context managers, such as
+# contextlib.contextmanager. Add to this list to register other decorators that
+# produce valid context managers.
+contextmanager-decorators=contextlib.contextmanager
+
+# List of members which are set dynamically and missed by pylint inference
+# system, and so shouldn't trigger E1101 when accessed. Python regular
+# expressions are accepted.
+generated-members=
+
+# Tells whether to warn about missing members when the owner of the attribute
+# is inferred to be None.
+ignore-none=yes
+
+# This flag controls whether pylint should warn about no-member and similar
+# checks whenever an opaque object is returned when inferring. The inference
+# can return multiple potential results while evaluating a Python object, but
+# some branches might not be evaluated, which results in partial inference. In
+# that case, it might be useful to still emit no-member and other checks for
+# the rest of the inferred objects.
+ignore-on-opaque-inference=yes
+
+# List of symbolic message names to ignore for Mixin members.
+ignored-checks-for-mixins=no-member,
+                          not-async-context-manager,
+                          not-context-manager,
+                          attribute-defined-outside-init
+
+# List of class names for which member attributes should not be checked (useful
+# for classes with dynamically set attributes). This supports the use of
+# qualified names.
+ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
+
+# Show a hint with possible names when a member name was not found. The aspect
+# of finding the hint is based on edit distance.
+missing-member-hint=yes
+
+# The minimum edit distance a name should have in order to be considered a
+# similar match for a missing member name.
+missing-member-hint-distance=1
+
+# The total number of similar names that should be taken in consideration when
+# showing a hint for a missing member.
+missing-member-max-choices=1
+
+# Regex pattern to define which classes are considered mixins.
+mixin-class-rgx=.*[Mm]ixin
+
+# List of decorators that change the signature of a decorated function.
+signature-mutators=
+
+
+[VARIABLES]
+
+# List of additional names supposed to be defined in builtins. Remember that
+# you should avoid defining new builtins when possible.
+additional-builtins=
+
+# Tells whether unused global variables should be treated as a violation.
+allow-global-unused-variables=yes
+
+# List of names allowed to shadow builtins
+allowed-redefined-builtins=
+
+# List of strings which can identify a callback function by name. A callback
+# name must start or end with one of those strings.
+callbacks=cb_,
+          _cb
+
+# A regular expression matching the name of dummy variables (i.e. expected to
+# not be used).
+dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
+
+# Argument names that match this expression will be ignored.
+ignored-argument-names=_.*|^ignored_|^unused_
+
+# Tells whether we should check for unused import in __init__ files.
+init-import=no
+
+# List of qualified module names which can have objects that can redefine
+# builtins.
+redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io

.travis.yml

@@ -1,20 +0,0 @@
-language: python
-python:
- - "3.5"
- - "3.5-dev"
- - "3.6"
- - "3.6-dev"
- - "3.7-dev"
- - "nightly"
-install:
- - pip install pylint requests
-script:
- - python3 -m pylint -r n -d bad-whitespace,missing-docstring,too-many-arguments,locally-disabled,line-too-long,too-many-public-methods,too-many-lines,too-many-instance-attributes,too-many-locals,too-many-branches,too-many-statements instaloader
-deploy:
-  provider: pypi
-  user: aandergr
-  on:
-    tags: true
-    python: 3.6
-  password:
-    secure: KBsATtXNnEzqr0X2ALdyx2zaYj2EUfK8YjP4jEOilBELpJ4evHYXVgRkn9CKXGsD7LKbWmR3gz2rr8NZ6U42pHw0fpQHizz91MpJSKbNY7RZZh4UVWMDh81izeglchWAF+m3Wdpxpo/GBDbrZIbRQFhPgB+kjSzJo3uiCS+yVXtGG+6csJHzd0WkJjsVHz+XSdxyOiLy/qFF3nmY6ZrVrgw6WbX2YLbx9je6DxK3YRrnBqgMgH7Ui2xWWbgJCk3KKpQTMiMKnTa9mdeMJAd26YMCz1gG/X7ck6VNly/6bs5+Zq7/bU0dSolhPBhNthyccYdS0IeGPdPF3YqatblJ4fLGTfHn8oJs3/T11yIW9pvOT3UftEynPOUX+en2Qnb/3WfgwUfVts8i8DAYgEFLGEzbF5XMGy4gwAD/bZ0JW0pmgAG2PZsdOokAt+ydlNj/uxq1GFW2doZbul6k4t03p1mik6itYscvuhKoEm8Yw6wr1P9aDvMR6FFpSu4gDt2XrKmRmWkSXxpNeCfZujTjyAnISxLVQGSY7BG5K0Z2HbDtqqNz9nj4WgNGzZmngiRofCevWR+vPIDUDW7NCX03csfcFS9RRllBO/BrL7NVUV+Qp9dfTQYp+8SqYLWgHtDoXdgyMREjJaAmbx2u/TYTY8bADZ5F3vGUpv2j5hP3NTU=

AUTHORS.md

@@ -0,0 +1,9 @@
+Authors
+=======
+
+Instaloader is written by
+
+- Alexander Graf (@aandergr)
+- André Koch-Kramer (@Thammus)
+- Lars Lindqvist (@e5150)
+- ... and many more, see https://github.com/instaloader/instaloader/graphs/contributors

CODE_OF_CONDUCT.md

@@ -0,0 +1,135 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement by opening an
+issue or contacting one or more of the project maintainers.
+
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
+
+Community Impact Guidelines were inspired by 
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available 
+at [https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations
+

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2016 Alexander Graf and André Koch-Kramer.
+Copyright (c) 2016-2020 Alexander Graf and André Koch-Kramer.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

Pipfile

@@ -0,0 +1,23 @@
+[[source]]
+name = "pypi"
+url = "https://pypi.org/simple"
+verify_ssl = true
+
+[dev-packages]
+pylint = "*"
+mypy = "*"
+sphinx = "~=2.3"
+pyinstaller = "*"
+pefile = "*"
+pywin32-ctypes = "*"
+psutil = "*"
+typing-extensions = "*"
+types-requests = "*"
+typed-ast = "*"
+jinja2 = "~=3.0.1"
+wrapt = "*"  # Needed for Python<3.11
+tomli = "*"  # Needed for Python<3.11
+dill = "*"  # Needed for Python<3.11
+
+[packages]
+requests = "*"

Pipfile.lock

@@ -0,0 +1,798 @@
+{
+    "_meta": {
+        "hash": {
+            "sha256": "847399ef6d835f48014fd3ba091234cf789a5cdf1430d65fe25b40544702a970"
+        },
+        "pipfile-spec": 6,
+        "requires": {},
+        "sources": [
+            {
+                "name": "pypi",
+                "url": "https://pypi.org/simple",
+                "verify_ssl": true
+            }
+        ]
+    },
+    "default": {
+        "certifi": {
+            "hashes": [
+                "sha256:35824b4c3a97115964b408844d64aa14db1cc518f6562e8d7261699d1350a9e3",
+                "sha256:4ad3232f5e926d6718ec31cfc1fcadfde020920e278684144551c91769c7bc18"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==2022.12.7"
+        },
+        "charset-normalizer": {
+            "hashes": [
+                "sha256:00d3ffdaafe92a5dc603cb9bd5111aaa36dfa187c8285c543be562e61b755f6b",
+                "sha256:024e606be3ed92216e2b6952ed859d86b4cfa52cd5bc5f050e7dc28f9b43ec42",
+                "sha256:0298eafff88c99982a4cf66ba2efa1128e4ddaca0b05eec4c456bbc7db691d8d",
+                "sha256:02a51034802cbf38db3f89c66fb5d2ec57e6fe7ef2f4a44d070a593c3688667b",
+                "sha256:083c8d17153ecb403e5e1eb76a7ef4babfc2c48d58899c98fcaa04833e7a2f9a",
+                "sha256:0a11e971ed097d24c534c037d298ad32c6ce81a45736d31e0ff0ad37ab437d59",
+                "sha256:0bf2dae5291758b6f84cf923bfaa285632816007db0330002fa1de38bfcb7154",
+                "sha256:0c0a590235ccd933d9892c627dec5bc7511ce6ad6c1011fdf5b11363022746c1",
+                "sha256:0f438ae3532723fb6ead77e7c604be7c8374094ef4ee2c5e03a3a17f1fca256c",
+                "sha256:109487860ef6a328f3eec66f2bf78b0b72400280d8f8ea05f69c51644ba6521a",
+                "sha256:11b53acf2411c3b09e6af37e4b9005cba376c872503c8f28218c7243582df45d",
+                "sha256:12db3b2c533c23ab812c2b25934f60383361f8a376ae272665f8e48b88e8e1c6",
+                "sha256:14e76c0f23218b8f46c4d87018ca2e441535aed3632ca134b10239dfb6dadd6b",
+                "sha256:16a8663d6e281208d78806dbe14ee9903715361cf81f6d4309944e4d1e59ac5b",
+                "sha256:292d5e8ba896bbfd6334b096e34bffb56161c81408d6d036a7dfa6929cff8783",
+                "sha256:2c03cc56021a4bd59be889c2b9257dae13bf55041a3372d3295416f86b295fb5",
+                "sha256:2e396d70bc4ef5325b72b593a72c8979999aa52fb8bcf03f701c1b03e1166918",
+                "sha256:2edb64ee7bf1ed524a1da60cdcd2e1f6e2b4f66ef7c077680739f1641f62f555",
+                "sha256:31a9ddf4718d10ae04d9b18801bd776693487cbb57d74cc3458a7673f6f34639",
+                "sha256:356541bf4381fa35856dafa6a965916e54bed415ad8a24ee6de6e37deccf2786",
+                "sha256:358a7c4cb8ba9b46c453b1dd8d9e431452d5249072e4f56cfda3149f6ab1405e",
+                "sha256:37f8febc8ec50c14f3ec9637505f28e58d4f66752207ea177c1d67df25da5aed",
+                "sha256:39049da0ffb96c8cbb65cbf5c5f3ca3168990adf3551bd1dee10c48fce8ae820",
+                "sha256:39cf9ed17fe3b1bc81f33c9ceb6ce67683ee7526e65fde1447c772afc54a1bb8",
+                "sha256:3ae1de54a77dc0d6d5fcf623290af4266412a7c4be0b1ff7444394f03f5c54e3",
+                "sha256:3b590df687e3c5ee0deef9fc8c547d81986d9a1b56073d82de008744452d6541",
+                "sha256:3e45867f1f2ab0711d60c6c71746ac53537f1684baa699f4f668d4c6f6ce8e14",
+                "sha256:3fc1c4a2ffd64890aebdb3f97e1278b0cc72579a08ca4de8cd2c04799a3a22be",
+                "sha256:4457ea6774b5611f4bed5eaa5df55f70abde42364d498c5134b7ef4c6958e20e",
+                "sha256:44ba614de5361b3e5278e1241fda3dc1838deed864b50a10d7ce92983797fa76",
+                "sha256:4a8fcf28c05c1f6d7e177a9a46a1c52798bfe2ad80681d275b10dcf317deaf0b",
+                "sha256:4b0d02d7102dd0f997580b51edc4cebcf2ab6397a7edf89f1c73b586c614272c",
+                "sha256:502218f52498a36d6bf5ea77081844017bf7982cdbe521ad85e64cabee1b608b",
+                "sha256:503e65837c71b875ecdd733877d852adbc465bd82c768a067badd953bf1bc5a3",
+                "sha256:5995f0164fa7df59db4746112fec3f49c461dd6b31b841873443bdb077c13cfc",
+                "sha256:59e5686dd847347e55dffcc191a96622f016bc0ad89105e24c14e0d6305acbc6",
+                "sha256:601f36512f9e28f029d9481bdaf8e89e5148ac5d89cffd3b05cd533eeb423b59",
+                "sha256:608862a7bf6957f2333fc54ab4399e405baad0163dc9f8d99cb236816db169d4",
+                "sha256:62595ab75873d50d57323a91dd03e6966eb79c41fa834b7a1661ed043b2d404d",
+                "sha256:70990b9c51340e4044cfc394a81f614f3f90d41397104d226f21e66de668730d",
+                "sha256:71140351489970dfe5e60fc621ada3e0f41104a5eddaca47a7acb3c1b851d6d3",
+                "sha256:72966d1b297c741541ca8cf1223ff262a6febe52481af742036a0b296e35fa5a",
+                "sha256:74292fc76c905c0ef095fe11e188a32ebd03bc38f3f3e9bcb85e4e6db177b7ea",
+                "sha256:761e8904c07ad053d285670f36dd94e1b6ab7f16ce62b9805c475b7aa1cffde6",
+                "sha256:772b87914ff1152b92a197ef4ea40efe27a378606c39446ded52c8f80f79702e",
+                "sha256:79909e27e8e4fcc9db4addea88aa63f6423ebb171db091fb4373e3312cb6d603",
+                "sha256:7e189e2e1d3ed2f4aebabd2d5b0f931e883676e51c7624826e0a4e5fe8a0bf24",
+                "sha256:7eb33a30d75562222b64f569c642ff3dc6689e09adda43a082208397f016c39a",
+                "sha256:81d6741ab457d14fdedc215516665050f3822d3e56508921cc7239f8c8e66a58",
+                "sha256:8499ca8f4502af841f68135133d8258f7b32a53a1d594aa98cc52013fff55678",
+                "sha256:84c3990934bae40ea69a82034912ffe5a62c60bbf6ec5bc9691419641d7d5c9a",
+                "sha256:87701167f2a5c930b403e9756fab1d31d4d4da52856143b609e30a1ce7160f3c",
+                "sha256:88600c72ef7587fe1708fd242b385b6ed4b8904976d5da0893e31df8b3480cb6",
+                "sha256:8ac7b6a045b814cf0c47f3623d21ebd88b3e8cf216a14790b455ea7ff0135d18",
+                "sha256:8b8af03d2e37866d023ad0ddea594edefc31e827fee64f8de5611a1dbc373174",
+                "sha256:8c7fe7afa480e3e82eed58e0ca89f751cd14d767638e2550c77a92a9e749c317",
+                "sha256:8eade758719add78ec36dc13201483f8e9b5d940329285edcd5f70c0a9edbd7f",
+                "sha256:911d8a40b2bef5b8bbae2e36a0b103f142ac53557ab421dc16ac4aafee6f53dc",
+                "sha256:93ad6d87ac18e2a90b0fe89df7c65263b9a99a0eb98f0a3d2e079f12a0735837",
+                "sha256:95dea361dd73757c6f1c0a1480ac499952c16ac83f7f5f4f84f0658a01b8ef41",
+                "sha256:9ab77acb98eba3fd2a85cd160851816bfce6871d944d885febf012713f06659c",
+                "sha256:9cb3032517f1627cc012dbc80a8ec976ae76d93ea2b5feaa9d2a5b8882597579",
+                "sha256:9cf4e8ad252f7c38dd1f676b46514f92dc0ebeb0db5552f5f403509705e24753",
+                "sha256:9d9153257a3f70d5f69edf2325357251ed20f772b12e593f3b3377b5f78e7ef8",
+                "sha256:a152f5f33d64a6be73f1d30c9cc82dfc73cec6477ec268e7c6e4c7d23c2d2291",
+                "sha256:a16418ecf1329f71df119e8a65f3aa68004a3f9383821edcb20f0702934d8087",
+                "sha256:a60332922359f920193b1d4826953c507a877b523b2395ad7bc716ddd386d866",
+                "sha256:a8d0fc946c784ff7f7c3742310cc8a57c5c6dc31631269876a88b809dbeff3d3",
+                "sha256:ab5de034a886f616a5668aa5d098af2b5385ed70142090e2a31bcbd0af0fdb3d",
+                "sha256:c22d3fe05ce11d3671297dc8973267daa0f938b93ec716e12e0f6dee81591dc1",
+                "sha256:c2ac1b08635a8cd4e0cbeaf6f5e922085908d48eb05d44c5ae9eabab148512ca",
+                "sha256:c512accbd6ff0270939b9ac214b84fb5ada5f0409c44298361b2f5e13f9aed9e",
+                "sha256:c75ffc45f25324e68ab238cb4b5c0a38cd1c3d7f1fb1f72b5541de469e2247db",
+                "sha256:c95a03c79bbe30eec3ec2b7f076074f4281526724c8685a42872974ef4d36b72",
+                "sha256:cadaeaba78750d58d3cc6ac4d1fd867da6fc73c88156b7a3212a3cd4819d679d",
+                "sha256:cd6056167405314a4dc3c173943f11249fa0f1b204f8b51ed4bde1a9cd1834dc",
+                "sha256:db72b07027db150f468fbada4d85b3b2729a3db39178abf5c543b784c1254539",
+                "sha256:df2c707231459e8a4028eabcd3cfc827befd635b3ef72eada84ab13b52e1574d",
+                "sha256:e62164b50f84e20601c1ff8eb55620d2ad25fb81b59e3cd776a1902527a788af",
+                "sha256:e696f0dd336161fca9adbb846875d40752e6eba585843c768935ba5c9960722b",
+                "sha256:eaa379fcd227ca235d04152ca6704c7cb55564116f8bc52545ff357628e10602",
+                "sha256:ebea339af930f8ca5d7a699b921106c6e29c617fe9606fa7baa043c1cdae326f",
+                "sha256:f4c39b0e3eac288fedc2b43055cfc2ca7a60362d0e5e87a637beac5d801ef478",
+                "sha256:f5057856d21e7586765171eac8b9fc3f7d44ef39425f85dbcccb13b3ebea806c",
+                "sha256:f6f45710b4459401609ebebdbcfb34515da4fc2aa886f95107f556ac69a9147e",
+                "sha256:f97e83fa6c25693c7a35de154681fcc257c1c41b38beb0304b9c4d2d9e164479",
+                "sha256:f9d0c5c045a3ca9bedfc35dca8526798eb91a07aa7a2c0fee134c6c6f321cbd7",
+                "sha256:ff6f3db31555657f3163b15a6b7c6938d08df7adbfc9dd13d9d19edad678f1e8"
+            ],
+            "version": "==3.0.1"
+        },
+        "idna": {
+            "hashes": [
+                "sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4",
+                "sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==3.4"
+        },
+        "requests": {
+            "hashes": [
+                "sha256:64299f4909223da747622c030b781c0d7811e359c37124b4bd368fb8c6518baa",
+                "sha256:98b1b2782e3c6c4904938b84c0eb932721069dfdb9134313beff7c83c2df24bf"
+            ],
+            "index": "pypi",
+            "version": "==2.28.2"
+        },
+        "urllib3": {
+            "hashes": [
+                "sha256:076907bf8fd355cde77728471316625a4d2f7e713c125f51953bb5b3eecf4f72",
+                "sha256:75edcdc2f7d85b137124a6c3c9fc3933cdeaa12ecb9a6a959f22797a0feca7e1"
+            ],
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
+            "version": "==1.26.14"
+        }
+    },
+    "develop": {
+        "alabaster": {
+            "hashes": [
+                "sha256:1ee19aca801bbabb5ba3f5f258e4422dfa86f82f3e9cefb0859b283cdd7f62a3",
+                "sha256:a27a4a084d5e690e16e01e03ad2b2e552c61a65469419b907243193de1a84ae2"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==0.7.13"
+        },
+        "altgraph": {
+            "hashes": [
+                "sha256:ad33358114df7c9416cdb8fa1eaa5852166c505118717021c6a8c7c7abbd03dd",
+                "sha256:c8ac1ca6772207179ed8003ce7687757c04b0b71536f81e2ac5755c6226458fe"
+            ],
+            "version": "==0.17.3"
+        },
+        "astroid": {
+            "hashes": [
+                "sha256:23c718921acab5f08cbbbe9293967f1f8fec40c336d19cd75dc12a9ea31d2eb2",
+                "sha256:bd1aa4f9915c98e8aaebcd4e71930154d4e8c9aaf05d35ac0a63d1956091ae3f"
+            ],
+            "markers": "python_full_version >= '3.7.2'",
+            "version": "==2.14.1"
+        },
+        "babel": {
+            "hashes": [
+                "sha256:1ad3eca1c885218f6dce2ab67291178944f810a10a9b5f3cb8382a5a232b64fe",
+                "sha256:5ef4b3226b0180dedded4229651c8b0e1a3a6a2837d45a073272f313e4cf97f6"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==2.11.0"
+        },
+        "certifi": {
+            "hashes": [
+                "sha256:35824b4c3a97115964b408844d64aa14db1cc518f6562e8d7261699d1350a9e3",
+                "sha256:4ad3232f5e926d6718ec31cfc1fcadfde020920e278684144551c91769c7bc18"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==2022.12.7"
+        },
+        "charset-normalizer": {
+            "hashes": [
+                "sha256:00d3ffdaafe92a5dc603cb9bd5111aaa36dfa187c8285c543be562e61b755f6b",
+                "sha256:024e606be3ed92216e2b6952ed859d86b4cfa52cd5bc5f050e7dc28f9b43ec42",
+                "sha256:0298eafff88c99982a4cf66ba2efa1128e4ddaca0b05eec4c456bbc7db691d8d",
+                "sha256:02a51034802cbf38db3f89c66fb5d2ec57e6fe7ef2f4a44d070a593c3688667b",
+                "sha256:083c8d17153ecb403e5e1eb76a7ef4babfc2c48d58899c98fcaa04833e7a2f9a",
+                "sha256:0a11e971ed097d24c534c037d298ad32c6ce81a45736d31e0ff0ad37ab437d59",
+                "sha256:0bf2dae5291758b6f84cf923bfaa285632816007db0330002fa1de38bfcb7154",
+                "sha256:0c0a590235ccd933d9892c627dec5bc7511ce6ad6c1011fdf5b11363022746c1",
+                "sha256:0f438ae3532723fb6ead77e7c604be7c8374094ef4ee2c5e03a3a17f1fca256c",
+                "sha256:109487860ef6a328f3eec66f2bf78b0b72400280d8f8ea05f69c51644ba6521a",
+                "sha256:11b53acf2411c3b09e6af37e4b9005cba376c872503c8f28218c7243582df45d",
+                "sha256:12db3b2c533c23ab812c2b25934f60383361f8a376ae272665f8e48b88e8e1c6",
+                "sha256:14e76c0f23218b8f46c4d87018ca2e441535aed3632ca134b10239dfb6dadd6b",
+                "sha256:16a8663d6e281208d78806dbe14ee9903715361cf81f6d4309944e4d1e59ac5b",
+                "sha256:292d5e8ba896bbfd6334b096e34bffb56161c81408d6d036a7dfa6929cff8783",
+                "sha256:2c03cc56021a4bd59be889c2b9257dae13bf55041a3372d3295416f86b295fb5",
+                "sha256:2e396d70bc4ef5325b72b593a72c8979999aa52fb8bcf03f701c1b03e1166918",
+                "sha256:2edb64ee7bf1ed524a1da60cdcd2e1f6e2b4f66ef7c077680739f1641f62f555",
+                "sha256:31a9ddf4718d10ae04d9b18801bd776693487cbb57d74cc3458a7673f6f34639",
+                "sha256:356541bf4381fa35856dafa6a965916e54bed415ad8a24ee6de6e37deccf2786",
+                "sha256:358a7c4cb8ba9b46c453b1dd8d9e431452d5249072e4f56cfda3149f6ab1405e",
+                "sha256:37f8febc8ec50c14f3ec9637505f28e58d4f66752207ea177c1d67df25da5aed",
+                "sha256:39049da0ffb96c8cbb65cbf5c5f3ca3168990adf3551bd1dee10c48fce8ae820",
+                "sha256:39cf9ed17fe3b1bc81f33c9ceb6ce67683ee7526e65fde1447c772afc54a1bb8",
+                "sha256:3ae1de54a77dc0d6d5fcf623290af4266412a7c4be0b1ff7444394f03f5c54e3",
+                "sha256:3b590df687e3c5ee0deef9fc8c547d81986d9a1b56073d82de008744452d6541",
+                "sha256:3e45867f1f2ab0711d60c6c71746ac53537f1684baa699f4f668d4c6f6ce8e14",
+                "sha256:3fc1c4a2ffd64890aebdb3f97e1278b0cc72579a08ca4de8cd2c04799a3a22be",
+                "sha256:4457ea6774b5611f4bed5eaa5df55f70abde42364d498c5134b7ef4c6958e20e",
+                "sha256:44ba614de5361b3e5278e1241fda3dc1838deed864b50a10d7ce92983797fa76",
+                "sha256:4a8fcf28c05c1f6d7e177a9a46a1c52798bfe2ad80681d275b10dcf317deaf0b",
+                "sha256:4b0d02d7102dd0f997580b51edc4cebcf2ab6397a7edf89f1c73b586c614272c",
+                "sha256:502218f52498a36d6bf5ea77081844017bf7982cdbe521ad85e64cabee1b608b",
+                "sha256:503e65837c71b875ecdd733877d852adbc465bd82c768a067badd953bf1bc5a3",
+                "sha256:5995f0164fa7df59db4746112fec3f49c461dd6b31b841873443bdb077c13cfc",
+                "sha256:59e5686dd847347e55dffcc191a96622f016bc0ad89105e24c14e0d6305acbc6",
+                "sha256:601f36512f9e28f029d9481bdaf8e89e5148ac5d89cffd3b05cd533eeb423b59",
+                "sha256:608862a7bf6957f2333fc54ab4399e405baad0163dc9f8d99cb236816db169d4",
+                "sha256:62595ab75873d50d57323a91dd03e6966eb79c41fa834b7a1661ed043b2d404d",
+                "sha256:70990b9c51340e4044cfc394a81f614f3f90d41397104d226f21e66de668730d",
+                "sha256:71140351489970dfe5e60fc621ada3e0f41104a5eddaca47a7acb3c1b851d6d3",
+                "sha256:72966d1b297c741541ca8cf1223ff262a6febe52481af742036a0b296e35fa5a",
+                "sha256:74292fc76c905c0ef095fe11e188a32ebd03bc38f3f3e9bcb85e4e6db177b7ea",
+                "sha256:761e8904c07ad053d285670f36dd94e1b6ab7f16ce62b9805c475b7aa1cffde6",
+                "sha256:772b87914ff1152b92a197ef4ea40efe27a378606c39446ded52c8f80f79702e",
+                "sha256:79909e27e8e4fcc9db4addea88aa63f6423ebb171db091fb4373e3312cb6d603",
+                "sha256:7e189e2e1d3ed2f4aebabd2d5b0f931e883676e51c7624826e0a4e5fe8a0bf24",
+                "sha256:7eb33a30d75562222b64f569c642ff3dc6689e09adda43a082208397f016c39a",
+                "sha256:81d6741ab457d14fdedc215516665050f3822d3e56508921cc7239f8c8e66a58",
+                "sha256:8499ca8f4502af841f68135133d8258f7b32a53a1d594aa98cc52013fff55678",
+                "sha256:84c3990934bae40ea69a82034912ffe5a62c60bbf6ec5bc9691419641d7d5c9a",
+                "sha256:87701167f2a5c930b403e9756fab1d31d4d4da52856143b609e30a1ce7160f3c",
+                "sha256:88600c72ef7587fe1708fd242b385b6ed4b8904976d5da0893e31df8b3480cb6",
+                "sha256:8ac7b6a045b814cf0c47f3623d21ebd88b3e8cf216a14790b455ea7ff0135d18",
+                "sha256:8b8af03d2e37866d023ad0ddea594edefc31e827fee64f8de5611a1dbc373174",
+                "sha256:8c7fe7afa480e3e82eed58e0ca89f751cd14d767638e2550c77a92a9e749c317",
+                "sha256:8eade758719add78ec36dc13201483f8e9b5d940329285edcd5f70c0a9edbd7f",
+                "sha256:911d8a40b2bef5b8bbae2e36a0b103f142ac53557ab421dc16ac4aafee6f53dc",
+                "sha256:93ad6d87ac18e2a90b0fe89df7c65263b9a99a0eb98f0a3d2e079f12a0735837",
+                "sha256:95dea361dd73757c6f1c0a1480ac499952c16ac83f7f5f4f84f0658a01b8ef41",
+                "sha256:9ab77acb98eba3fd2a85cd160851816bfce6871d944d885febf012713f06659c",
+                "sha256:9cb3032517f1627cc012dbc80a8ec976ae76d93ea2b5feaa9d2a5b8882597579",
+                "sha256:9cf4e8ad252f7c38dd1f676b46514f92dc0ebeb0db5552f5f403509705e24753",
+                "sha256:9d9153257a3f70d5f69edf2325357251ed20f772b12e593f3b3377b5f78e7ef8",
+                "sha256:a152f5f33d64a6be73f1d30c9cc82dfc73cec6477ec268e7c6e4c7d23c2d2291",
+                "sha256:a16418ecf1329f71df119e8a65f3aa68004a3f9383821edcb20f0702934d8087",
+                "sha256:a60332922359f920193b1d4826953c507a877b523b2395ad7bc716ddd386d866",
+                "sha256:a8d0fc946c784ff7f7c3742310cc8a57c5c6dc31631269876a88b809dbeff3d3",
+                "sha256:ab5de034a886f616a5668aa5d098af2b5385ed70142090e2a31bcbd0af0fdb3d",
+                "sha256:c22d3fe05ce11d3671297dc8973267daa0f938b93ec716e12e0f6dee81591dc1",
+                "sha256:c2ac1b08635a8cd4e0cbeaf6f5e922085908d48eb05d44c5ae9eabab148512ca",
+                "sha256:c512accbd6ff0270939b9ac214b84fb5ada5f0409c44298361b2f5e13f9aed9e",
+                "sha256:c75ffc45f25324e68ab238cb4b5c0a38cd1c3d7f1fb1f72b5541de469e2247db",
+                "sha256:c95a03c79bbe30eec3ec2b7f076074f4281526724c8685a42872974ef4d36b72",
+                "sha256:cadaeaba78750d58d3cc6ac4d1fd867da6fc73c88156b7a3212a3cd4819d679d",
+                "sha256:cd6056167405314a4dc3c173943f11249fa0f1b204f8b51ed4bde1a9cd1834dc",
+                "sha256:db72b07027db150f468fbada4d85b3b2729a3db39178abf5c543b784c1254539",
+                "sha256:df2c707231459e8a4028eabcd3cfc827befd635b3ef72eada84ab13b52e1574d",
+                "sha256:e62164b50f84e20601c1ff8eb55620d2ad25fb81b59e3cd776a1902527a788af",
+                "sha256:e696f0dd336161fca9adbb846875d40752e6eba585843c768935ba5c9960722b",
+                "sha256:eaa379fcd227ca235d04152ca6704c7cb55564116f8bc52545ff357628e10602",
+                "sha256:ebea339af930f8ca5d7a699b921106c6e29c617fe9606fa7baa043c1cdae326f",
+                "sha256:f4c39b0e3eac288fedc2b43055cfc2ca7a60362d0e5e87a637beac5d801ef478",
+                "sha256:f5057856d21e7586765171eac8b9fc3f7d44ef39425f85dbcccb13b3ebea806c",
+                "sha256:f6f45710b4459401609ebebdbcfb34515da4fc2aa886f95107f556ac69a9147e",
+                "sha256:f97e83fa6c25693c7a35de154681fcc257c1c41b38beb0304b9c4d2d9e164479",
+                "sha256:f9d0c5c045a3ca9bedfc35dca8526798eb91a07aa7a2c0fee134c6c6f321cbd7",
+                "sha256:ff6f3db31555657f3163b15a6b7c6938d08df7adbfc9dd13d9d19edad678f1e8"
+            ],
+            "version": "==3.0.1"
+        },
+        "dill": {
+            "hashes": [
+                "sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0",
+                "sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373"
+            ],
+            "index": "pypi",
+            "version": "==0.3.6"
+        },
+        "docutils": {
+            "hashes": [
+                "sha256:686577d2e4c32380bb50cbb22f575ed742d58168cee37e99117a854bcd88f125",
+                "sha256:cf316c8370a737a022b72b56874f6602acf974a37a9fba42ec2876387549fc61"
+            ],
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
+            "version": "==0.17.1"
+        },
+        "idna": {
+            "hashes": [
+                "sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4",
+                "sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==3.4"
+        },
+        "imagesize": {
+            "hashes": [
+                "sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b",
+                "sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a"
+            ],
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+            "version": "==1.4.1"
+        },
+        "isort": {
+            "hashes": [
+                "sha256:8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504",
+                "sha256:f84c2818376e66cf843d497486ea8fed8700b340f308f076c6fb1229dff318b6"
+            ],
+            "markers": "python_full_version >= '3.8.0'",
+            "version": "==5.12.0"
+        },
+        "jinja2": {
+            "hashes": [
+                "sha256:077ce6014f7b40d03b47d1f1ca4b0fc8328a692bd284016f806ed0eaca390ad8",
+                "sha256:611bb273cd68f3b993fabdc4064fc858c5b47a973cb5aa7999ec1ba405c87cd7"
+            ],
+            "index": "pypi",
+            "version": "==3.0.3"
+        },
+        "lazy-object-proxy": {
+            "hashes": [
+                "sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382",
+                "sha256:0a891e4e41b54fd5b8313b96399f8b0e173bbbfc03c7631f01efbe29bb0bcf82",
+                "sha256:189bbd5d41ae7a498397287c408617fe5c48633e7755287b21d741f7db2706a9",
+                "sha256:18b78ec83edbbeb69efdc0e9c1cb41a3b1b1ed11ddd8ded602464c3fc6020494",
+                "sha256:1aa3de4088c89a1b69f8ec0dcc169aa725b0ff017899ac568fe44ddc1396df46",
+                "sha256:212774e4dfa851e74d393a2370871e174d7ff0ebc980907723bb67d25c8a7c30",
+                "sha256:2d0daa332786cf3bb49e10dc6a17a52f6a8f9601b4cf5c295a4f85854d61de63",
+                "sha256:5f83ac4d83ef0ab017683d715ed356e30dd48a93746309c8f3517e1287523ef4",
+                "sha256:659fb5809fa4629b8a1ac5106f669cfc7bef26fbb389dda53b3e010d1ac4ebae",
+                "sha256:660c94ea760b3ce47d1855a30984c78327500493d396eac4dfd8bd82041b22be",
+                "sha256:66a3de4a3ec06cd8af3f61b8e1ec67614fbb7c995d02fa224813cb7afefee701",
+                "sha256:721532711daa7db0d8b779b0bb0318fa87af1c10d7fe5e52ef30f8eff254d0cd",
+                "sha256:7322c3d6f1766d4ef1e51a465f47955f1e8123caee67dd641e67d539a534d006",
+                "sha256:79a31b086e7e68b24b99b23d57723ef7e2c6d81ed21007b6281ebcd1688acb0a",
+                "sha256:81fc4d08b062b535d95c9ea70dbe8a335c45c04029878e62d744bdced5141586",
+                "sha256:8fa02eaab317b1e9e03f69aab1f91e120e7899b392c4fc19807a8278a07a97e8",
+                "sha256:9090d8e53235aa280fc9239a86ae3ea8ac58eff66a705fa6aa2ec4968b95c821",
+                "sha256:946d27deaff6cf8452ed0dba83ba38839a87f4f7a9732e8f9fd4107b21e6ff07",
+                "sha256:9990d8e71b9f6488e91ad25f322898c136b008d87bf852ff65391b004da5e17b",
+                "sha256:9cd077f3d04a58e83d04b20e334f678c2b0ff9879b9375ed107d5d07ff160171",
+                "sha256:9e7551208b2aded9c1447453ee366f1c4070602b3d932ace044715d89666899b",
+                "sha256:9f5fa4a61ce2438267163891961cfd5e32ec97a2c444e5b842d574251ade27d2",
+                "sha256:b40387277b0ed2d0602b8293b94d7257e17d1479e257b4de114ea11a8cb7f2d7",
+                "sha256:bfb38f9ffb53b942f2b5954e0f610f1e721ccebe9cce9025a38c8ccf4a5183a4",
+                "sha256:cbf9b082426036e19c6924a9ce90c740a9861e2bdc27a4834fd0a910742ac1e8",
+                "sha256:d9e25ef10a39e8afe59a5c348a4dbf29b4868ab76269f81ce1674494e2565a6e",
+                "sha256:db1c1722726f47e10e0b5fdbf15ac3b8adb58c091d12b3ab713965795036985f",
+                "sha256:e7c21c95cae3c05c14aafffe2865bbd5e377cfc1348c4f7751d9dc9a48ca4bda",
+                "sha256:e8c6cfb338b133fbdbc5cfaa10fe3c6aeea827db80c978dbd13bc9dd8526b7d4",
+                "sha256:ea806fd4c37bf7e7ad82537b0757999264d5f70c45468447bb2b91afdbe73a6e",
+                "sha256:edd20c5a55acb67c7ed471fa2b5fb66cb17f61430b7a6b9c3b4a1e40293b1671",
+                "sha256:f0117049dd1d5635bbff65444496c90e0baa48ea405125c088e93d9cf4525b11",
+                "sha256:f0705c376533ed2a9e5e97aacdbfe04cecd71e0aa84c7c0595d02ef93b6e4455",
+                "sha256:f12ad7126ae0c98d601a7ee504c1122bcef553d1d5e0c3bfa77b16b3968d2734",
+                "sha256:f2457189d8257dd41ae9b434ba33298aec198e30adf2dcdaaa3a28b9994f6adb",
+                "sha256:f699ac1c768270c9e384e4cbd268d6e67aebcfae6cd623b4d7c3bfde5a35db59"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==1.9.0"
+        },
+        "markupsafe": {
+            "hashes": [
+                "sha256:0576fe974b40a400449768941d5d0858cc624e3249dfd1e0c33674e5c7ca7aed",
+                "sha256:085fd3201e7b12809f9e6e9bc1e5c96a368c8523fad5afb02afe3c051ae4afcc",
+                "sha256:090376d812fb6ac5f171e5938e82e7f2d7adc2b629101cec0db8b267815c85e2",
+                "sha256:0b462104ba25f1ac006fdab8b6a01ebbfbce9ed37fd37fd4acd70c67c973e460",
+                "sha256:137678c63c977754abe9086a3ec011e8fd985ab90631145dfb9294ad09c102a7",
+                "sha256:1bea30e9bf331f3fef67e0a3877b2288593c98a21ccb2cf29b74c581a4eb3af0",
+                "sha256:22152d00bf4a9c7c83960521fc558f55a1adbc0631fbb00a9471e097b19d72e1",
+                "sha256:22731d79ed2eb25059ae3df1dfc9cb1546691cc41f4e3130fe6bfbc3ecbbecfa",
+                "sha256:2298c859cfc5463f1b64bd55cb3e602528db6fa0f3cfd568d3605c50678f8f03",
+                "sha256:28057e985dace2f478e042eaa15606c7efccb700797660629da387eb289b9323",
+                "sha256:2e7821bffe00aa6bd07a23913b7f4e01328c3d5cc0b40b36c0bd81d362faeb65",
+                "sha256:2ec4f2d48ae59bbb9d1f9d7efb9236ab81429a764dedca114f5fdabbc3788013",
+                "sha256:340bea174e9761308703ae988e982005aedf427de816d1afe98147668cc03036",
+                "sha256:40627dcf047dadb22cd25ea7ecfe9cbf3bbbad0482ee5920b582f3809c97654f",
+                "sha256:40dfd3fefbef579ee058f139733ac336312663c6706d1163b82b3003fb1925c4",
+                "sha256:4cf06cdc1dda95223e9d2d3c58d3b178aa5dacb35ee7e3bbac10e4e1faacb419",
+                "sha256:50c42830a633fa0cf9e7d27664637532791bfc31c731a87b202d2d8ac40c3ea2",
+                "sha256:55f44b440d491028addb3b88f72207d71eeebfb7b5dbf0643f7c023ae1fba619",
+                "sha256:608e7073dfa9e38a85d38474c082d4281f4ce276ac0010224eaba11e929dd53a",
+                "sha256:63ba06c9941e46fa389d389644e2d8225e0e3e5ebcc4ff1ea8506dce646f8c8a",
+                "sha256:65608c35bfb8a76763f37036547f7adfd09270fbdbf96608be2bead319728fcd",
+                "sha256:665a36ae6f8f20a4676b53224e33d456a6f5a72657d9c83c2aa00765072f31f7",
+                "sha256:6d6607f98fcf17e534162f0709aaad3ab7a96032723d8ac8750ffe17ae5a0666",
+                "sha256:7313ce6a199651c4ed9d7e4cfb4aa56fe923b1adf9af3b420ee14e6d9a73df65",
+                "sha256:7668b52e102d0ed87cb082380a7e2e1e78737ddecdde129acadb0eccc5423859",
+                "sha256:7df70907e00c970c60b9ef2938d894a9381f38e6b9db73c5be35e59d92e06625",
+                "sha256:7e007132af78ea9df29495dbf7b5824cb71648d7133cf7848a2a5dd00d36f9ff",
+                "sha256:835fb5e38fd89328e9c81067fd642b3593c33e1e17e2fdbf77f5676abb14a156",
+                "sha256:8bca7e26c1dd751236cfb0c6c72d4ad61d986e9a41bbf76cb445f69488b2a2bd",
+                "sha256:8db032bf0ce9022a8e41a22598eefc802314e81b879ae093f36ce9ddf39ab1ba",
+                "sha256:99625a92da8229df6d44335e6fcc558a5037dd0a760e11d84be2260e6f37002f",
+                "sha256:9cad97ab29dfc3f0249b483412c85c8ef4766d96cdf9dcf5a1e3caa3f3661cf1",
+                "sha256:a4abaec6ca3ad8660690236d11bfe28dfd707778e2442b45addd2f086d6ef094",
+                "sha256:a6e40afa7f45939ca356f348c8e23048e02cb109ced1eb8420961b2f40fb373a",
+                "sha256:a6f2fcca746e8d5910e18782f976489939d54a91f9411c32051b4aab2bd7c513",
+                "sha256:a806db027852538d2ad7555b203300173dd1b77ba116de92da9afbc3a3be3eed",
+                "sha256:abcabc8c2b26036d62d4c746381a6f7cf60aafcc653198ad678306986b09450d",
+                "sha256:b8526c6d437855442cdd3d87eede9c425c4445ea011ca38d937db299382e6fa3",
+                "sha256:bb06feb762bade6bf3c8b844462274db0c76acc95c52abe8dbed28ae3d44a147",
+                "sha256:c0a33bc9f02c2b17c3ea382f91b4db0e6cde90b63b296422a939886a7a80de1c",
+                "sha256:c4a549890a45f57f1ebf99c067a4ad0cb423a05544accaf2b065246827ed9603",
+                "sha256:ca244fa73f50a800cf8c3ebf7fd93149ec37f5cb9596aa8873ae2c1d23498601",
+                "sha256:cf877ab4ed6e302ec1d04952ca358b381a882fbd9d1b07cccbfd61783561f98a",
+                "sha256:d9d971ec1e79906046aa3ca266de79eac42f1dbf3612a05dc9368125952bd1a1",
+                "sha256:da25303d91526aac3672ee6d49a2f3db2d9502a4a60b55519feb1a4c7714e07d",
+                "sha256:e55e40ff0cc8cc5c07996915ad367fa47da6b3fc091fdadca7f5403239c5fec3",
+                "sha256:f03a532d7dee1bed20bc4884194a16160a2de9ffc6354b3878ec9682bb623c54",
+                "sha256:f1cd098434e83e656abf198f103a8207a8187c0fc110306691a2e94a78d0abb2",
+                "sha256:f2bfb563d0211ce16b63c7cb9395d2c682a23187f54c3d79bfec33e6705473c6",
+                "sha256:f8ffb705ffcf5ddd0e80b65ddf7bed7ee4f5a441ea7d3419e861a12eaf41af58"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==2.1.2"
+        },
+        "mccabe": {
+            "hashes": [
+                "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325",
+                "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==0.7.0"
+        },
+        "mypy": {
+            "hashes": [
+                "sha256:01b1b9e1ed40544ef486fa8ac022232ccc57109f379611633ede8e71630d07d2",
+                "sha256:0ab090d9240d6b4e99e1fa998c2d0aa5b29fc0fb06bd30e7ad6183c95fa07593",
+                "sha256:14d776869a3e6c89c17eb943100f7868f677703c8a4e00b3803918f86aafbc52",
+                "sha256:1ace23f6bb4aec4604b86c4843276e8fa548d667dbbd0cb83a3ae14b18b2db6c",
+                "sha256:2efa963bdddb27cb4a0d42545cd137a8d2b883bd181bbc4525b568ef6eca258f",
+                "sha256:2f6ac8c87e046dc18c7d1d7f6653a66787a4555085b056fe2d599f1f1a2a2d21",
+                "sha256:3ae4c7a99e5153496243146a3baf33b9beff714464ca386b5f62daad601d87af",
+                "sha256:3cfad08f16a9c6611e6143485a93de0e1e13f48cfb90bcad7d5fde1c0cec3d36",
+                "sha256:4e5175026618c178dfba6188228b845b64131034ab3ba52acaffa8f6c361f805",
+                "sha256:50979d5efff8d4135d9db293c6cb2c42260e70fb010cbc697b1311a4d7a39ddb",
+                "sha256:5cd187d92b6939617f1168a4fe68f68add749902c010e66fe574c165c742ed88",
+                "sha256:5cfca124f0ac6707747544c127880893ad72a656e136adc935c8600740b21ff5",
+                "sha256:5e398652d005a198a7f3c132426b33c6b85d98aa7dc852137a2a3be8890c4072",
+                "sha256:67cced7f15654710386e5c10b96608f1ee3d5c94ca1da5a2aad5889793a824c1",
+                "sha256:7306edca1c6f1b5fa0bc9aa645e6ac8393014fa82d0fa180d0ebc990ebe15964",
+                "sha256:7cc2c01dfc5a3cbddfa6c13f530ef3b95292f926329929001d45e124342cd6b7",
+                "sha256:87edfaf344c9401942883fad030909116aa77b0fa7e6e8e1c5407e14549afe9a",
+                "sha256:8845125d0b7c57838a10fd8925b0f5f709d0e08568ce587cc862aacce453e3dd",
+                "sha256:92024447a339400ea00ac228369cd242e988dd775640755fa4ac0c126e49bb74",
+                "sha256:a86b794e8a56ada65c573183756eac8ac5b8d3d59daf9d5ebd72ecdbb7867a43",
+                "sha256:bb2782a036d9eb6b5a6efcdda0986774bf798beef86a62da86cb73e2a10b423d",
+                "sha256:be78077064d016bc1b639c2cbcc5be945b47b4261a4f4b7d8923f6c69c5c9457",
+                "sha256:c7cf862aef988b5fbaa17764ad1d21b4831436701c7d2b653156a9497d92c83c",
+                "sha256:e0626db16705ab9f7fa6c249c017c887baf20738ce7f9129da162bb3075fc1af",
+                "sha256:f34495079c8d9da05b183f9f7daec2878280c2ad7cc81da686ef0b484cea2ecf",
+                "sha256:fe523fcbd52c05040c7bee370d66fee8373c5972171e4fbc323153433198592d"
+            ],
+            "index": "pypi",
+            "version": "==1.0.0"
+        },
+        "mypy-extensions": {
+            "hashes": [
+                "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d",
+                "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.0.0"
+        },
+        "packaging": {
+            "hashes": [
+                "sha256:714ac14496c3e68c99c29b00845f7a2b85f3bb6f1078fd9f72fd20f0570002b2",
+                "sha256:b6ad297f8907de0fa2fe1ccbd26fdaf387f5f47c7275fedf8cce89f99446cf97"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==23.0"
+        },
+        "pefile": {
+            "hashes": [
+                "sha256:82e6114004b3d6911c77c3953e3838654b04511b8b66e8583db70c65998017dc",
+                "sha256:da185cd2af68c08a6cd4481f7325ed600a88f6a813bad9dea07ab3ef73d8d8d6"
+            ],
+            "index": "pypi",
+            "version": "==2023.2.7"
+        },
+        "platformdirs": {
+            "hashes": [
+                "sha256:8a1228abb1ef82d788f74139988b137e78692984ec7b08eaa6c65f1723af28f9",
+                "sha256:b1d5eb14f221506f50d6604a561f4c5786d9e80355219694a1b244bcd96f4567"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==3.0.0"
+        },
+        "psutil": {
+            "hashes": [
+                "sha256:149555f59a69b33f056ba1c4eb22bb7bf24332ce631c44a319cec09f876aaeff",
+                "sha256:16653106f3b59386ffe10e0bad3bb6299e169d5327d3f187614b1cb8f24cf2e1",
+                "sha256:3d7f9739eb435d4b1338944abe23f49584bde5395f27487d2ee25ad9a8774a62",
+                "sha256:3ff89f9b835100a825b14c2808a106b6fdcc4b15483141482a12c725e7f78549",
+                "sha256:54c0d3d8e0078b7666984e11b12b88af2db11d11249a8ac8920dd5ef68a66e08",
+                "sha256:54d5b184728298f2ca8567bf83c422b706200bcbbfafdc06718264f9393cfeb7",
+                "sha256:6001c809253a29599bc0dfd5179d9f8a5779f9dffea1da0f13c53ee568115e1e",
+                "sha256:68908971daf802203f3d37e78d3f8831b6d1014864d7a85937941bb35f09aefe",
+                "sha256:6b92c532979bafc2df23ddc785ed116fced1f492ad90a6830cf24f4d1ea27d24",
+                "sha256:852dd5d9f8a47169fe62fd4a971aa07859476c2ba22c2254d4a1baa4e10b95ad",
+                "sha256:9120cd39dca5c5e1c54b59a41d205023d436799b1c8c4d3ff71af18535728e94",
+                "sha256:c1ca331af862803a42677c120aff8a814a804e09832f166f226bfd22b56feee8",
+                "sha256:efeae04f9516907be44904cc7ce08defb6b665128992a56957abc9b61dca94b7",
+                "sha256:fd8522436a6ada7b4aad6638662966de0d61d241cb821239b2ae7013d41a43d4"
+            ],
+            "index": "pypi",
+            "version": "==5.9.4"
+        },
+        "pygments": {
+            "hashes": [
+                "sha256:b3ed06a9e8ac9a9aae5a6f5dbe78a8a58655d17b43b93c078f094ddc476ae297",
+                "sha256:fa7bd7bd2771287c0de303af8bfdfc731f51bd2c6a47ab69d117138893b82717"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==2.14.0"
+        },
+        "pyinstaller": {
+            "hashes": [
+                "sha256:0e5953937d35f0b37543cc6915dacaf3239bcbdf3fd3ecbb7866645468a16775",
+                "sha256:0f80e2403e76630ad3392c71f09c1a4284e8d8a8a99fb55ff3a0aba0e06300ed",
+                "sha256:1ac3f09b838710c43e34b0a7ad003bd168a754b0b786c561b47baf1af9104354",
+                "sha256:28a8a0da656493aa32d9665e2f6f84775da0f23174859ed8facaa4226fe77a17",
+                "sha256:2c1dd9d11cfc48bab61eeb06de69a3d1ad742bbb2ef14716965ca0333dd43a5b",
+                "sha256:3180b9bf22263380adc5e2ee051b7c21463292877215bbe70c9155dc76f4b966",
+                "sha256:3e51e18a16dec0414079762843cf892a5d70749ad56ca7b3c7b5f8367dc50b1e",
+                "sha256:9b47c10fbefac6f6493266f8b1689109b2b14efa9142dbd2cd7549226a4568b7",
+                "sha256:9cdb8ee8622ee8d2c6cd67f001b610019d4371a8bf3f7850562640ce786894d7",
+                "sha256:b967ae71ab7b05e18608dbb4518da5afa54f0835927cb7a5ce52ab8fffed03b6",
+                "sha256:dfc12e92fe10ae645dd0dd1fcfa4cd7677b2e96119e3cd4980d742e09bb78925",
+                "sha256:f35f06d48faea0ad738429c009941059beebaa306e9d9ead95f1df4b441de2aa"
+            ],
+            "index": "pypi",
+            "version": "==5.7.0"
+        },
+        "pyinstaller-hooks-contrib": {
+            "hashes": [
+                "sha256:55c1def8066d0279d06cd67eea30c12ffcdb961a5edeeaf361adac0164baef30",
+                "sha256:73fd4051dc1620f3ae9643291cd9e2f47bfed582ade2eb05e3247ecab4a4f5f3"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==2022.15"
+        },
+        "pylint": {
+            "hashes": [
+                "sha256:bad9d7c36037f6043a1e848a43004dfd5ea5ceb05815d713ba56ca4503a9fe37",
+                "sha256:ffe7fa536bb38ba35006a7c8a6d2efbfdd3d95bbf21199cad31f76b1c50aaf30"
+            ],
+            "index": "pypi",
+            "version": "==2.16.1"
+        },
+        "pytz": {
+            "hashes": [
+                "sha256:01a0681c4b9684a28304615eba55d1ab31ae00bf68ec157ec3708a8182dbbcd0",
+                "sha256:78f4f37d8198e0627c5f1143240bb0206b8691d8d7ac6d78fee88b78733f8c4a"
+            ],
+            "version": "==2022.7.1"
+        },
+        "pywin32-ctypes": {
+            "hashes": [
+                "sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942",
+                "sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98"
+            ],
+            "index": "pypi",
+            "version": "==0.2.0"
+        },
+        "requests": {
+            "hashes": [
+                "sha256:64299f4909223da747622c030b781c0d7811e359c37124b4bd368fb8c6518baa",
+                "sha256:98b1b2782e3c6c4904938b84c0eb932721069dfdb9134313beff7c83c2df24bf"
+            ],
+            "index": "pypi",
+            "version": "==2.28.2"
+        },
+        "setuptools": {
+            "hashes": [
+                "sha256:16ccf598aab3b506593c17378473978908a2734d7336755a8769b480906bec1c",
+                "sha256:b440ee5f7e607bb8c9de15259dba2583dd41a38879a7abc1d43a71c59524da48"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==67.2.0"
+        },
+        "snowballstemmer": {
+            "hashes": [
+                "sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1",
+                "sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a"
+            ],
+            "version": "==2.2.0"
+        },
+        "sphinx": {
+            "hashes": [
+                "sha256:02d7e9dc5f30caa42a682b26de408b755a55c7b07f356a30a3b6300bf7d4740e",
+                "sha256:b00394e90463e7482c4cf59e7db1c8604baeca1468abfc062904dedc1cea6fcc"
+            ],
+            "index": "pypi",
+            "version": "==2.4.5"
+        },
+        "sphinxcontrib-applehelp": {
+            "hashes": [
+                "sha256:29d341f67fb0f6f586b23ad80e072c8e6ad0b48417db2bde114a4c9746feb228",
+                "sha256:828f867945bbe39817c210a1abfd1bc4895c8b73fcaade56d45357a348a07d7e"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==1.0.4"
+        },
+        "sphinxcontrib-devhelp": {
+            "hashes": [
+                "sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e",
+                "sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.0.2"
+        },
+        "sphinxcontrib-htmlhelp": {
+            "hashes": [
+                "sha256:0cbdd302815330058422b98a113195c9249825d681e18f11e8b1f78a2f11efff",
+                "sha256:c38cb46dccf316c79de6e5515e1770414b797162b23cd3d06e67020e1d2a6903"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==2.0.1"
+        },
+        "sphinxcontrib-jsmath": {
+            "hashes": [
+                "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178",
+                "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.0.1"
+        },
+        "sphinxcontrib-qthelp": {
+            "hashes": [
+                "sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72",
+                "sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.0.3"
+        },
+        "sphinxcontrib-serializinghtml": {
+            "hashes": [
+                "sha256:352a9a00ae864471d3a7ead8d7d79f5fc0b57e8b3f95e9867eb9eb28999b92fd",
+                "sha256:aa5f6de5dfdf809ef505c4895e51ef5c9eac17d0f287933eb49ec495280b6952"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.1.5"
+        },
+        "tomli": {
+            "hashes": [
+                "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc",
+                "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"
+            ],
+            "index": "pypi",
+            "version": "==2.0.1"
+        },
+        "tomlkit": {
+            "hashes": [
+                "sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b",
+                "sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==0.11.6"
+        },
+        "typed-ast": {
+            "hashes": [
+                "sha256:0261195c2062caf107831e92a76764c81227dae162c4f75192c0d489faf751a2",
+                "sha256:0fdbcf2fef0ca421a3f5912555804296f0b0960f0418c440f5d6d3abb549f3e1",
+                "sha256:183afdf0ec5b1b211724dfef3d2cad2d767cbefac291f24d69b00546c1837fb6",
+                "sha256:211260621ab1cd7324e0798d6be953d00b74e0428382991adfddb352252f1d62",
+                "sha256:267e3f78697a6c00c689c03db4876dd1efdfea2f251a5ad6555e82a26847b4ac",
+                "sha256:2efae9db7a8c05ad5547d522e7dbe62c83d838d3906a3716d1478b6c1d61388d",
+                "sha256:370788a63915e82fd6f212865a596a0fefcbb7d408bbbb13dea723d971ed8bdc",
+                "sha256:39e21ceb7388e4bb37f4c679d72707ed46c2fbf2a5609b8b8ebc4b067d977df2",
+                "sha256:3e123d878ba170397916557d31c8f589951e353cc95fb7f24f6bb69adc1a8a97",
+                "sha256:4879da6c9b73443f97e731b617184a596ac1235fe91f98d279a7af36c796da35",
+                "sha256:4e964b4ff86550a7a7d56345c7864b18f403f5bd7380edf44a3c1fb4ee7ac6c6",
+                "sha256:639c5f0b21776605dd6c9dbe592d5228f021404dafd377e2b7ac046b0349b1a1",
+                "sha256:669dd0c4167f6f2cd9f57041e03c3c2ebf9063d0757dc89f79ba1daa2bfca9d4",
+                "sha256:6778e1b2f81dfc7bc58e4b259363b83d2e509a65198e85d5700dfae4c6c8ff1c",
+                "sha256:683407d92dc953c8a7347119596f0b0e6c55eb98ebebd9b23437501b28dcbb8e",
+                "sha256:79b1e0869db7c830ba6a981d58711c88b6677506e648496b1f64ac7d15633aec",
+                "sha256:7d5d014b7daa8b0bf2eaef684295acae12b036d79f54178b92a2b6a56f92278f",
+                "sha256:98f80dee3c03455e92796b58b98ff6ca0b2a6f652120c263efdba4d6c5e58f72",
+                "sha256:a94d55d142c9265f4ea46fab70977a1944ecae359ae867397757d836ea5a3f47",
+                "sha256:a9916d2bb8865f973824fb47436fa45e1ebf2efd920f2b9f99342cb7fab93f72",
+                "sha256:c542eeda69212fa10a7ada75e668876fdec5f856cd3d06829e6aa64ad17c8dfe",
+                "sha256:cf4afcfac006ece570e32d6fa90ab74a17245b83dfd6655a6f68568098345ff6",
+                "sha256:ebd9d7f80ccf7a82ac5f88c521115cc55d84e35bf8b446fcd7836eb6b98929a3",
+                "sha256:ed855bbe3eb3715fca349c80174cfcfd699c2f9de574d40527b8429acae23a66"
+            ],
+            "index": "pypi",
+            "version": "==1.5.4"
+        },
+        "types-requests": {
+            "hashes": [
+                "sha256:dbc2933635860e553ffc59f5e264264981358baffe6342b925e3eb8261f866ee",
+                "sha256:fd530aab3fc4f05ee36406af168f0836e6f00f1ee51a0b96b7311f82cb675230"
+            ],
+            "index": "pypi",
+            "version": "==2.28.11.12"
+        },
+        "types-urllib3": {
+            "hashes": [
+                "sha256:5630e578246d170d91ebe3901788cd28d53c4e044dc2e2488e3b0d55fb6895d8",
+                "sha256:e8f25c8bb85cde658c72ee931e56e7abd28803c26032441eea9ff4a4df2b0c31"
+            ],
+            "version": "==1.26.25.5"
+        },
+        "typing-extensions": {
+            "hashes": [
+                "sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa",
+                "sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e"
+            ],
+            "index": "pypi",
+            "version": "==4.4.0"
+        },
+        "urllib3": {
+            "hashes": [
+                "sha256:076907bf8fd355cde77728471316625a4d2f7e713c125f51953bb5b3eecf4f72",
+                "sha256:75edcdc2f7d85b137124a6c3c9fc3933cdeaa12ecb9a6a959f22797a0feca7e1"
+            ],
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
+            "version": "==1.26.14"
+        },
+        "wrapt": {
+            "hashes": [
+                "sha256:00b6d4ea20a906c0ca56d84f93065b398ab74b927a7a3dbd470f6fc503f95dc3",
+                "sha256:01c205616a89d09827986bc4e859bcabd64f5a0662a7fe95e0d359424e0e071b",
+                "sha256:02b41b633c6261feff8ddd8d11c711df6842aba629fdd3da10249a53211a72c4",
+                "sha256:07f7a7d0f388028b2df1d916e94bbb40624c59b48ecc6cbc232546706fac74c2",
+                "sha256:11871514607b15cfeb87c547a49bca19fde402f32e2b1c24a632506c0a756656",
+                "sha256:1b376b3f4896e7930f1f772ac4b064ac12598d1c38d04907e696cc4d794b43d3",
+                "sha256:21ac0156c4b089b330b7666db40feee30a5d52634cc4560e1905d6529a3897ff",
+                "sha256:257fd78c513e0fb5cdbe058c27a0624c9884e735bbd131935fd49e9fe719d310",
+                "sha256:2b39d38039a1fdad98c87279b48bc5dce2c0ca0d73483b12cb72aa9609278e8a",
+                "sha256:2cf71233a0ed05ccdabe209c606fe0bac7379fdcf687f39b944420d2a09fdb57",
+                "sha256:2fe803deacd09a233e4762a1adcea5db5d31e6be577a43352936179d14d90069",
+                "sha256:3232822c7d98d23895ccc443bbdf57c7412c5a65996c30442ebe6ed3df335383",
+                "sha256:34aa51c45f28ba7f12accd624225e2b1e5a3a45206aa191f6f9aac931d9d56fe",
+                "sha256:36f582d0c6bc99d5f39cd3ac2a9062e57f3cf606ade29a0a0d6b323462f4dd87",
+                "sha256:380a85cf89e0e69b7cfbe2ea9f765f004ff419f34194018a6827ac0e3edfed4d",
+                "sha256:40e7bc81c9e2b2734ea4bc1aceb8a8f0ceaac7c5299bc5d69e37c44d9081d43b",
+                "sha256:43ca3bbbe97af00f49efb06e352eae40434ca9d915906f77def219b88e85d907",
+                "sha256:4fcc4649dc762cddacd193e6b55bc02edca674067f5f98166d7713b193932b7f",
+                "sha256:5a0f54ce2c092aaf439813735584b9537cad479575a09892b8352fea5e988dc0",
+                "sha256:5a9a0d155deafd9448baff28c08e150d9b24ff010e899311ddd63c45c2445e28",
+                "sha256:5b02d65b9ccf0ef6c34cba6cf5bf2aab1bb2f49c6090bafeecc9cd81ad4ea1c1",
+                "sha256:60db23fa423575eeb65ea430cee741acb7c26a1365d103f7b0f6ec412b893853",
+                "sha256:642c2e7a804fcf18c222e1060df25fc210b9c58db7c91416fb055897fc27e8cc",
+                "sha256:6a9a25751acb379b466ff6be78a315e2b439d4c94c1e99cb7266d40a537995d3",
+                "sha256:6b1a564e6cb69922c7fe3a678b9f9a3c54e72b469875aa8018f18b4d1dd1adf3",
+                "sha256:6d323e1554b3d22cfc03cd3243b5bb815a51f5249fdcbb86fda4bf62bab9e164",
+                "sha256:6e743de5e9c3d1b7185870f480587b75b1cb604832e380d64f9504a0535912d1",
+                "sha256:709fe01086a55cf79d20f741f39325018f4df051ef39fe921b1ebe780a66184c",
+                "sha256:7b7c050ae976e286906dd3f26009e117eb000fb2cf3533398c5ad9ccc86867b1",
+                "sha256:7d2872609603cb35ca513d7404a94d6d608fc13211563571117046c9d2bcc3d7",
+                "sha256:7ef58fb89674095bfc57c4069e95d7a31cfdc0939e2a579882ac7d55aadfd2a1",
+                "sha256:80bb5c256f1415f747011dc3604b59bc1f91c6e7150bd7db03b19170ee06b320",
+                "sha256:81b19725065dcb43df02b37e03278c011a09e49757287dca60c5aecdd5a0b8ed",
+                "sha256:833b58d5d0b7e5b9832869f039203389ac7cbf01765639c7309fd50ef619e0b1",
+                "sha256:88bd7b6bd70a5b6803c1abf6bca012f7ed963e58c68d76ee20b9d751c74a3248",
+                "sha256:8ad85f7f4e20964db4daadcab70b47ab05c7c1cf2a7c1e51087bfaa83831854c",
+                "sha256:8c0ce1e99116d5ab21355d8ebe53d9460366704ea38ae4d9f6933188f327b456",
+                "sha256:8d649d616e5c6a678b26d15ece345354f7c2286acd6db868e65fcc5ff7c24a77",
+                "sha256:903500616422a40a98a5a3c4ff4ed9d0066f3b4c951fa286018ecdf0750194ef",
+                "sha256:9736af4641846491aedb3c3f56b9bc5568d92b0692303b5a305301a95dfd38b1",
+                "sha256:988635d122aaf2bdcef9e795435662bcd65b02f4f4c1ae37fbee7401c440b3a7",
+                "sha256:9cca3c2cdadb362116235fdbd411735de4328c61425b0aa9f872fd76d02c4e86",
+                "sha256:9e0fd32e0148dd5dea6af5fee42beb949098564cc23211a88d799e434255a1f4",
+                "sha256:9f3e6f9e05148ff90002b884fbc2a86bd303ae847e472f44ecc06c2cd2fcdb2d",
+                "sha256:a85d2b46be66a71bedde836d9e41859879cc54a2a04fad1191eb50c2066f6e9d",
+                "sha256:a9a52172be0b5aae932bef82a79ec0a0ce87288c7d132946d645eba03f0ad8a8",
+                "sha256:aa31fdcc33fef9eb2552cbcbfee7773d5a6792c137b359e82879c101e98584c5",
+                "sha256:b014c23646a467558be7da3d6b9fa409b2c567d2110599b7cf9a0c5992b3b471",
+                "sha256:b21bb4c09ffabfa0e85e3a6b623e19b80e7acd709b9f91452b8297ace2a8ab00",
+                "sha256:b5901a312f4d14c59918c221323068fad0540e34324925c8475263841dbdfe68",
+                "sha256:b9b7a708dd92306328117d8c4b62e2194d00c365f18eff11a9b53c6f923b01e3",
+                "sha256:d1967f46ea8f2db647c786e78d8cc7e4313dbd1b0aca360592d8027b8508e24d",
+                "sha256:d52a25136894c63de15a35bc0bdc5adb4b0e173b9c0d07a2be9d3ca64a332735",
+                "sha256:d77c85fedff92cf788face9bfa3ebaa364448ebb1d765302e9af11bf449ca36d",
+                "sha256:d79d7d5dc8a32b7093e81e97dad755127ff77bcc899e845f41bf71747af0c569",
+                "sha256:dbcda74c67263139358f4d188ae5faae95c30929281bc6866d00573783c422b7",
+                "sha256:ddaea91abf8b0d13443f6dac52e89051a5063c7d014710dcb4d4abb2ff811a59",
+                "sha256:dee0ce50c6a2dd9056c20db781e9c1cfd33e77d2d569f5d1d9321c641bb903d5",
+                "sha256:dee60e1de1898bde3b238f18340eec6148986da0455d8ba7848d50470a7a32fb",
+                "sha256:e2f83e18fe2f4c9e7db597e988f72712c0c3676d337d8b101f6758107c42425b",
+                "sha256:e3fb1677c720409d5f671e39bac6c9e0e422584e5f518bfd50aa4cbbea02433f",
+                "sha256:ee2b1b1769f6707a8a445162ea16dddf74285c3964f605877a20e38545c3c462",
+                "sha256:ee6acae74a2b91865910eef5e7de37dc6895ad96fa23603d1d27ea69df545015",
+                "sha256:ef3f72c9666bba2bab70d2a8b79f2c6d2c1a42a7f7e2b0ec83bb2f9e383950af"
+            ],
+            "index": "pypi",
+            "version": "==1.14.1"
+        }
+    }
+}

README.rst

@@ -1,40 +1,66 @@
-Instaloader
-===========
+.. image:: https://raw.githubusercontent.com/instaloader/instaloader/master/docs/logo_heading.png
 
-Download pictures (or videos) along with their captions and other metadata
-from Instagram.
+.. badges-start
 
-Installation
-------------
+|pypi| |pyversion| |license| |aur| |contributors| |downloads|
 
-Instaloader is written in Python, thus ensure having
-`Python <https://www.python.org/>`__ (at least version 3.5) installed.
+.. |pypi| image:: https://img.shields.io/pypi/v/instaloader.svg
+   :alt: Instaloader PyPI Project Page
+   :target: https://pypi.org/project/instaloader/
 
-If you intend to use this tool under Windows, it is recommended
-to install
-`win-unicode-console <https://pypi.python.org/pypi/win_unicode_console>`__.
+.. |license| image:: https://img.shields.io/github/license/instaloader/instaloader.svg
+   :alt: MIT License
+   :target: https://github.com/instaloader/instaloader/blob/master/LICENSE
 
-If you have `pip <https://pypi.python.org/pypi/pip>`__ installed, you
-may install Instaloader using
+.. |pyversion| image:: https://img.shields.io/pypi/pyversions/instaloader.svg
+   :alt: Supported Python Versions
+
+.. |contributors| image:: https://img.shields.io/github/contributors/instaloader/instaloader.svg
+   :alt: Contributor Count
+   :target: https://github.com/instaloader/instaloader/graphs/contributors
+
+.. |aur| image:: https://img.shields.io/aur/version/instaloader.svg
+   :alt: Arch User Repository Package
+   :target: https://aur.archlinux.org/packages/instaloader/
+
+.. |downloads| image:: https://pepy.tech/badge/instaloader/month
+   :alt: PyPI Download Count
+   :target: https://pepy.tech/project/instaloader
+
+.. badges-end
 
 ::
 
-    pip3 install instaloader
+    $ pip3 install instaloader
 
-Alternatively, to get the most current version of Instaloader from our
-`Git repository <https://github.com/Thammus/instaloader>`__:
+    $ instaloader profile [profile ...]
+
+**Instaloader**
+
+- downloads **public and private profiles, hashtags, user stories,
+  feeds and saved media**,
+
+- downloads **comments, geotags and captions** of each post,
+
+- automatically **detects profile name changes** and renames the target
+  directory accordingly,
+
+- allows **fine-grained customization** of filters and where to store
+  downloaded media,
+
+- automatically **resumes previously-interrupted** download iterations.
 
 ::
 
-    pip3 install git+https://github.com/Thammus/instaloader
+    instaloader [--comments] [--geotags]
+                [--stories] [--highlights] [--tagged] [--igtv]
+                [--login YOUR-USERNAME] [--fast-update]
+                profile | "#hashtag" | :stories | :feed | :saved
 
-(pass ``--upgrade`` to upgrade if Instaloader is already installed)
+`Instaloader Documentation <https://instaloader.github.io/>`__
 
-Instaloader requires
-`requests <https://pypi.python.org/pypi/requests>`__, which
-will be installed automatically, if it is not already installed.
 
-How to automatically download pictures from Instagram
+How to Automatically Download Pictures from Instagram
 -----------------------------------------------------
 
 To **download all pictures and videos of a profile**, as well as the
@@ -53,8 +79,22 @@ To later **update your local copy** of that profiles, you may run
 
     instaloader --fast-update profile [profile ...]
 
-When ``--fast-update`` is given, Instaloader stops when arriving at
-the first already-downloaded picture.
+If ``--fast-update`` is given, Instaloader stops when arriving at the
+first already-downloaded picture.
+
+Alternatively, you can use ``--latest-stamps`` to have Instaloader store
+the time each profile was last downloaded and only download newer media:
+
+::
+
+    instaloader --latest-stamps -- profile [profile ...]
+
+With this option it's possible to move or delete downloaded media and still keep
+the archive updated.
+
+When updating profiles, Instaloader
+automatically **detects profile name changes** and renames the target directory
+accordingly.
 
 Instaloader can also be used to **download private profiles**. To do so,
 invoke it with
@@ -63,237 +103,54 @@ invoke it with
 
     instaloader --login=your_username profile [profile ...]
 
-When invoked like this, it also **stores the session cookies** in a file
-in your temporary directory, which will be reused later when ``--login`` is given. So
-you can download private profiles **non-interactively** when you already
-have a valid session cookie file.
+When logging in, Instaloader **stores the session cookies** in a file in your
+temporary directory, which will be reused later the next time ``--login``
+is given.  So you can download private profiles **non-interactively** when you
+already have a valid session cookie file.
 
-Besides downloading private profiles, being logged in allows to
-**download stories**:
+`Instaloader Documentation <https://instaloader.github.io/basic-usage.html>`__
 
-::
+Contributing
+------------
 
-    instaloader --login=your_username --stories profile [profile ...]
+As an open source project, Instaloader heavily depends on the contributions from
+its community. See
+`contributing <https://instaloader.github.io/contributing.html>`__
+for how you may help Instaloader to become an even greater tool.
 
-You may also download
-**the most recent pictures by hashtag**:
+Supporters
+----------
 
-::
+.. current-sponsors-start
 
-    instaloader "#hashtag"
+| Instaloader is proudly sponsored by
+|  `@rocketapi-io <https://github.com/rocketapi-io>`__
+|  `@socialmethod <https://github.com/socialmethod>`__
 
-This downloads the requested posts into a directory named according to the specified
-hashtag and the filenames correspond to the timestamp of the posts.
-As with all download tasks, this behavior can easily be customized, for example
-encode the poster's profile name in the filenames:
+See `Alex' GitHub Sponsors <https://github.com/sponsors/aandergr>`__ page for
+how you can sponsor the development of Instaloader!
 
-::
+.. current-sponsors-end
 
-    instaloader --filename-pattern={date}_{profile} "#hashtag"
+It is a pleasure for us to share our Instaloader to the world, and we are proud
+to have attracted such an active and motivating community, with so many users
+who share their suggestions and ideas with us. Buying a community-sponsored beer
+or coffee from time to time is very likely to further raise our passion for the
+development of Instaloader.
 
-If you want to **download all followees of a given profile**, call
-
-::
-
-    instaloader --login=your_username @profile
-
-To **download all the pictures from your feed which you have liked**, call
-
-::
-
-    instaloader --login=your_username :feed-liked
-
-or to **download all pictures from your feed**:
-
-::
-
-    instaloader --login=your_username :feed-all
-
-**Download all stories** from the profiles you follow:
-
-::
-
-    instaloader --login=your_username --filename-pattern={date}_{profile} :stories
-
-Advanced Options
-----------------
-
-The following flags can be given to Instaloader to specify how profiles should
-be downloaded.
-
-To get a list of all flags, their abbreviations and their descriptions, you may
-run ``instaloader --help``.
-
-What to Download
-^^^^^^^^^^^^^^^^
-
-Specify a list of profiles or #hashtags. For each of these, Instaloader
-creates a folder and downloads all posts along with the pictures's
-captions and the current **profile picture**. If an already-downloaded profile
-has been renamed, Instaloader automatically **finds it by its unique ID** and
-renames the folder likewise.
-
-Instead of a *profile* or a *#hashtag*, the special targets
-``:feed-all`` (pictures from your feed),
-``:feed-liked`` (pictures from your feed which you liked), and
-``:stories`` (stories of your followees) can be specified.
-
---profile-pic-only         Only download profile picture.
---skip-videos              Do not download videos.
---geotags                  **Download geotags** when available. Geotags are stored as
-                           a text file with the location's name and a Google Maps
-                           link. This requires an additional request to the
-                           Instagram server for each picture, which is why it is
-                           disabled by default.
---comments                 Download and update comments for each post. This
-                           requires an additional request to the Instagram server
-                           for each post, which is why it is disabled by default.
---stories                  Also **download stories** of each profile that is
-                           downloaded. Requires ``--login``.
---stories-only             Rather than downloading regular posts of each
-                           specified profile, only download stories.
-                           Requires ``--login``.
-
-When to Stop Downloading
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-If none of these options are given, Instaloader goes through all pictures
-matching the specified targets.
-
---fast-update              For each target, stop when encountering the first
-                           already-downloaded picture. This flag is recommended
-                           when you use Instaloader to update your personal
-                           Instagram archive.
---count COUNT              Do not attempt to download more than COUNT posts.
-                           Applies only to ``#hashtag``, ``:feed-all`` and ``:feed-liked``.
-
-
-Login (Download Private Profiles)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Instaloader can **login to Instagram**. This allows downloading private
-profiles. To login, pass the ``--login`` option. Your session cookie (not your
-password!) will be saved to a local file to be reused next time you want
-Instaloader to login.
-
---login YOUR-USERNAME      Login name (profile name) for your Instagram account.
---sessionfile SESSIONFILE  Path for loading and storing session key file.
-                           Defaults to a path
-                           within your temporary directory, encoding your local
-                           username and your Instagram profile name.
---password YOUR-PASSWORD   Password for your Instagram account. Without this
-                           option, you'll be prompted for your password
-                           interactively if there is not yet a valid session
-                           file.
-
-How to Download
-^^^^^^^^^^^^^^^
-
---dirname-pattern DIRNAME_PATTERN
-                           Name of directory where to store posts. ``{profile}``
-                           is replaced by the profile name, ``{target}`` is replaced
-                           by the target you specified, i.e. either ``:feed``,
-                           ``#hashtag`` or the profile name. Defaults to ``{target}``.
---filename-pattern FILENAME_PATTERN
-                           Prefix of filenames. Posts are stored in the
-                           directory whose pattern is given with ``--dirname-pattern``.
-                           ``{profile}`` is replaced by the profile name,
-                           ``{target}`` is replaced by the target you specified, i.e.
-                           either ``:feed``, ``#hashtag`` or the profile name. Also, the
-                           fields ``{date}`` and ``{shortcode}`` can be specified.
-                           Defaults to ``{date:%Y-%m-%d_%H-%M-%S}``.
---user-agent USER_AGENT    User Agent to use for HTTP requests. Per default,
-                           Instaloader pretends being Chrome/51.
---no-sleep                 Do not sleep between requests to Instagram's servers.
-                           This makes downloading faster, but may be suspicious.
-
-Miscellaneous Options
-^^^^^^^^^^^^^^^^^^^^^
-
---shorter-output           Do not display captions while downloading.
---quiet                    Disable user interaction, i.e. do not print messages
-                           (except errors) and fail if login credentials are
-                           needed but not given. This makes Instaloader
-                           **suitable as a cron job**.
-
-Usage as Python module
-----------------------
-
-You may also use parts of Instaloader as library to do other interesting
-things.
-
-For example, to get a list of all followees and a list of all followers of a profile, do
-
-.. code:: python
-
-    import instaloader
-
-    # Get instance
-    loader = instaloader.Instaloader()
-
-    # Login
-    loader.interactive_login(USERNAME)
-
-    # Retrieve followees
-    followees = loader.get_followees(PROFILE)
-    print(PROFILE + " follows these profiles:")
-    for f in followees:
-        print("\t%s\t%s" % (f['username'], f['full_name']))
-
-    # Retrieve followers
-    followers = loader.get_followers(PROFILE)
-    print("Followers of " + PROFILE + ":")
-    for f in followers:
-        print("\t%s\t%s" % (f['username'], f['full_name']))
-
-Then, you may download all pictures of all followees with
-
-.. code:: python
-
-    for f in followees:
-        try:
-            loader.download(f['username'])
-        except instaloader.NonfatalException:
-            pass
-
-You could also download your last 20 liked pics with
-
-.. code:: python
-
-    loader.download_feed_pics(max_count=20, fast_update=True,
-                             filter_func=lambda node:
-                                   not node["likes"]["viewer_has_liked"] if "likes" in node else not node["viewer_has_liked"])
-
-To download the last 20 pictures with hashtag #cat, do
-
-.. code:: python
-
-    loader.download_hashtag('cat', max_count=20)
-
-If logged in, Instaloader is also able to download user stories:
-
-.. code:: python
-
-    loader.download_stories()
-
-Each Instagram profile has its own unique ID which stays unmodified even
-if a user changes his/her username. To get said ID, given the profile's
-name, you may call
-
-.. code:: python
-
-    loader.get_id_by_username(PROFILE_NAME)
-
-``get_followees()`` also returns unique IDs for all loaded followees. To
-get the current username of a profile, given this unique ID
-``get_username_by_id()`` can be used. For example:
-
-.. code:: python
-
-    loader.get_username_by_id(followees[0]['id'])
+| For Donations, we provide GitHub Sponsors page, a PayPal.Me link and a Bitcoin address.
+|  GitHub Sponsors: `Sponsor @aandergr on GitHub Sponsors <https://github.com/sponsors/aandergr>`__
+|  PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
+|  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
 
 Disclaimer
 ----------
 
-This code is in no way affiliated with, authorized, maintained or endorsed by Instagram or any of its affiliates or
-subsidiaries. This is an independent and unofficial project. Use at your own risk.
+.. disclaimer-start
+
+Instaloader is in no way affiliated with, authorized, maintained or endorsed by Instagram or any of its affiliates or
+subsidiaries. This is an independent and unofficial project. Use at your own risk.
+
+Instaloader is licensed under an MIT license. Refer to ``LICENSE`` file for more information.
+
+.. disclaimer-end

deploy/arch/.SRCINFO.template

@@ -0,0 +1,15 @@
+pkgbase = instaloader
+	pkgdesc = Command line tool to download pictures, videos and metadata from Instagram
+	pkgver = {{version}}
+	pkgrel = 1
+	url = https://instaloader.github.io/
+	arch = any
+	license = MIT
+	makedepends = python-setuptools
+	depends = python>=3.8
+	depends = python-requests>=2.4
+	options = !emptydirs
+	source = instaloader-{{version}}.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v{{version}}
+	sha512sums = {{hash}}
+
+pkgname = instaloader

deploy/arch/PKGBUILD.template

@@ -0,0 +1,27 @@
+# Maintainer: André Koch-Kramer <koch-kramer@web.de>
+
+pkgname=instaloader
+pkgver={{version}}
+pkgrel=1
+pkgdesc="Command line tool to download pictures, videos and metadata from Instagram"
+arch=('any')
+url="https://instaloader.github.io/"
+license=('MIT')
+groups=()
+depends=('python>=3.8' 'python-requests>=2.4')
+makedepends=('python-setuptools')
+options=('!emptydirs')
+source=($pkgname-$pkgver.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v$pkgver)
+sha512sums=('{{hash}}')
+
+build() {
+	cd "$srcdir/$pkgname-$pkgver"
+	python setup.py clean --all
+	python setup.py build
+}
+
+package() {
+	cd "$srcdir/$pkgname-$pkgver"
+	install -Dm644 LICENSE "$pkgdir"/usr/share/licenses/$pkgname/LICENSE
+	python setup.py install --root "$pkgdir/" --skip-build
+}

deploy/arch/deploy.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+VERSION=${1:1}
+
+# do not deploy pre-releases
+echo $VERSION | grep -qe "[abc]" && exit 0
+
+cd $(dirname $0)
+
+# decrypt and add ssh key
+openssl aes-256-cbc -K $AUR_KEY -iv $AUR_IV -in id_rsa_AUR.enc -out /tmp/AUR_openssh -d
+eval "$(ssh-agent -s)"
+chmod 600 /tmp/AUR_openssh
+ssh-add /tmp/AUR_openssh
+mkdir -p ~/.ssh
+ssh-keyscan -t rsa aur.archlinux.org >> ~/.ssh/known_hosts
+
+# clone and modify AUR repo
+git clone --depth 1 ssh://aur@aur.archlinux.org/instaloader.git
+
+curl -sSfOJ https://codeload.github.com/instaloader/instaloader/tar.gz/$1
+HASH=$(sha512sum instaloader-$VERSION.tar.gz | cut -f1 -d " ")
+sed -e "s/{{version}}/$VERSION/g" -e "s/{{hash}}/$HASH/g" PKGBUILD.template > instaloader/PKGBUILD
+sed -e "s/{{version}}/$VERSION/g" -e "s/{{hash}}/$HASH/g" .SRCINFO.template > instaloader/.SRCINFO
+
+# commit and push changes
+cd instaloader
+git config user.email "koch-kramer@web.de"
+git config user.name "André Koch-Kramer"
+git add .
+git commit -m "Release of version $VERSION"
+git push

deploy/windows/create_exe.py

@@ -0,0 +1,71 @@
+import hashlib
+import os
+import re
+import shutil
+import subprocess
+import sys
+
+# copy the required files into repo root
+shutil.copy('docs/favicon.ico', '.')
+shutil.copy('deploy/windows/instaloader.spec', '.')
+
+code = """
+import contextlib
+import psutil
+import subprocess
+
+def __main():
+    with contextlib.suppress(AttributeError, psutil.Error):
+        if psutil.Process().parent().parent().name() == "explorer.exe":
+            subprocess.Popen("powershell -NoExit -Command \\\"& '{0}'\\\"".format(sys.argv[0]))
+            return
+    main()
+
+if __name__ == "__main__":
+    __main()
+"""
+
+with open('instaloader/__main__.py', 'r') as f:
+    # adjust imports for changed file structure
+    regex = re.compile(r'from (?:(\.[^ ]+ )|\.( ))import')
+    lines = [regex.sub(r'from instaloader\1\2import', line) for line in f.readlines()]
+
+    # insert code for magic exe behavior
+    index = lines.index('if __name__ == "__main__":\n')
+    code_lines = [cl + '\n' for cl in code.splitlines()]
+    for i, code_line in enumerate(code_lines):
+        if i + index < len(lines):
+            lines[i + index] = code_line
+        else:
+            lines.extend(code_lines[i:])
+            break
+
+with open('__main__.py', 'w+') as f:
+    f.writelines(lines)
+
+# install dependencies and invoke PyInstaller
+commands = ["pip install pipenv==2023.2.4",
+            f"pipenv --python {sys.version_info.major}.{sys.version_info.minor} sync --dev",
+            "pipenv run pyinstaller --log-level=DEBUG instaloader.spec"]
+
+for command in commands:
+    print()
+    print('#' * (len(command) + 6))
+    print('## {} ##'.format(command))
+    print('#' * (len(command) + 6))
+    print(flush=True)
+    err = subprocess.Popen(command).wait()
+    if err != 0:
+        sys.exit(err)
+
+# calculate and store MD5 hash for created executable
+hash_md5 = hashlib.md5()
+with open('dist/instaloader.exe', 'rb') as f:
+    for chunk in iter(lambda: f.read(4096), b''):
+        hash_md5.update(chunk)
+
+with open('dist/instaloader.exe.md5', 'w+') as f:
+    f.write('{} *instaloader.exe\n'.format(hash_md5.hexdigest()))
+
+# Create ZIP file
+shutil.make_archive('instaloader-{}-windows-standalone'.format(os.getenv('VERSION_TAG')), 'zip', 'dist')

deploy/windows/instaloader.spec

@@ -0,0 +1,28 @@
+# -*- mode: python -*-
+
+block_cipher = None
+
+
+a = Analysis(['__main__.py'],
+             binaries=[],
+             datas=[],
+             hiddenimports=[],
+             hookspath=[],
+             runtime_hooks=[],
+             excludes=[],
+             win_no_prefer_redirects=False,
+             win_private_assemblies=False,
+             cipher=block_cipher)
+pyz = PYZ(a.pure, a.zipped_data,
+             cipher=block_cipher)
+exe = EXE(pyz,
+          a.scripts,
+          a.binaries,
+          a.zipfiles,
+          a.datas,
+          name='instaloader',
+          debug=False,
+          strip=False,
+          upx=False,
+          runtime_tmpdir=None,
+          console=True , icon='favicon.ico')

docs/.gitignore

@@ -0,0 +1 @@
+_build/

docs/Makefile

@@ -0,0 +1,231 @@
+# Makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = sphinx-build
+PAPER         =
+BUILDDIR      = _build
+
+# Internal variables.
+PAPEROPT_a4     = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+.PHONY: help
+help:
+	@echo "Please use \`make <target>' where <target> is one of"
+	@echo "  html       to make standalone HTML files"
+	@echo "  dirhtml    to make HTML files named index.html in directories"
+	@echo "  singlehtml to make a single large HTML file"
+	@echo "  pickle     to make pickle files"
+	@echo "  json       to make JSON files"
+	@echo "  htmlhelp   to make HTML files and a HTML help project"
+	@echo "  qthelp     to make HTML files and a qthelp project"
+	@echo "  applehelp  to make an Apple Help Book"
+	@echo "  devhelp    to make HTML files and a Devhelp project"
+	@echo "  epub       to make an epub"
+	@echo "  epub3      to make an epub3"
+	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
+	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
+	@echo "  latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
+	@echo "  text       to make text files"
+	@echo "  man        to make manual pages"
+	@echo "  texinfo    to make Texinfo files"
+	@echo "  info       to make Texinfo files and run them through makeinfo"
+	@echo "  gettext    to make PO message catalogs"
+	@echo "  changes    to make an overview of all changed/added/deprecated items"
+	@echo "  xml        to make Docutils-native XML files"
+	@echo "  pseudoxml  to make pseudoxml-XML files for display purposes"
+	@echo "  linkcheck  to check all external links for integrity"
+	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"
+	@echo "  coverage   to run coverage check of the documentation (if enabled)"
+	@echo "  dummy      to check syntax errors of document sources"
+
+.PHONY: clean
+clean:
+	rm -rf $(BUILDDIR)/*
+
+.PHONY: html
+html:
+	LC_ALL=C $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
+
+.PHONY: dirhtml
+dirhtml:
+	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
+	@echo
+	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
+
+.PHONY: singlehtml
+singlehtml:
+	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
+	@echo
+	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
+
+.PHONY: pickle
+pickle:
+	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
+	@echo
+	@echo "Build finished; now you can process the pickle files."
+
+.PHONY: json
+json:
+	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
+	@echo
+	@echo "Build finished; now you can process the JSON files."
+
+.PHONY: htmlhelp
+htmlhelp:
+	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
+	@echo
+	@echo "Build finished; now you can run HTML Help Workshop with the" \
+	      ".hhp project file in $(BUILDDIR)/htmlhelp."
+
+.PHONY: qthelp
+qthelp:
+	$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
+	@echo
+	@echo "Build finished; now you can run "qcollectiongenerator" with the" \
+	      ".qhcp project file in $(BUILDDIR)/qthelp, like this:"
+	@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Instaloader.qhcp"
+	@echo "To view the help file:"
+	@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Instaloader.qhc"
+
+.PHONY: applehelp
+applehelp:
+	$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
+	@echo
+	@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
+	@echo "N.B. You won't be able to view it unless you put it in" \
+	      "~/Library/Documentation/Help or install it in your application" \
+	      "bundle."
+
+.PHONY: devhelp
+devhelp:
+	$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
+	@echo
+	@echo "Build finished."
+	@echo "To view the help file:"
+	@echo "# mkdir -p $$HOME/.local/share/devhelp/Instaloader"
+	@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Instaloader"
+	@echo "# devhelp"
+
+.PHONY: epub
+epub:
+	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
+	@echo
+	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
+
+.PHONY: epub3
+epub3:
+	$(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3
+	@echo
+	@echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3."
+
+.PHONY: latex
+latex:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo
+	@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
+	@echo "Run \`make' in that directory to run these through (pdf)latex" \
+	      "(use \`make latexpdf' here to do that automatically)."
+
+.PHONY: latexpdf
+latexpdf:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo "Running LaTeX files through pdflatex..."
+	$(MAKE) -C $(BUILDDIR)/latex all-pdf
+	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+.PHONY: latexpdfja
+latexpdfja:
+	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
+	@echo "Running LaTeX files through platex and dvipdfmx..."
+	$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
+	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
+
+.PHONY: text
+text:
+	$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
+	@echo
+	@echo "Build finished. The text files are in $(BUILDDIR)/text."
+
+.PHONY: man
+man:
+	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
+	@echo
+	@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
+
+.PHONY: texinfo
+texinfo:
+	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+	@echo
+	@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
+	@echo "Run \`make' in that directory to run these through makeinfo" \
+	      "(use \`make info' here to do that automatically)."
+
+.PHONY: info
+info:
+	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
+	@echo "Running Texinfo files through makeinfo..."
+	make -C $(BUILDDIR)/texinfo info
+	@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
+
+.PHONY: gettext
+gettext:
+	$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
+	@echo
+	@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
+
+.PHONY: changes
+changes:
+	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
+	@echo
+	@echo "The overview file is in $(BUILDDIR)/changes."
+
+.PHONY: linkcheck
+linkcheck:
+	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
+	@echo
+	@echo "Link check complete; look for any errors in the above output " \
+	      "or in $(BUILDDIR)/linkcheck/output.txt."
+
+.PHONY: doctest
+doctest:
+	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
+	@echo "Testing of doctests in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/doctest/output.txt."
+
+.PHONY: coverage
+coverage:
+	$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
+	@echo "Testing of coverage in the sources finished, look at the " \
+	      "results in $(BUILDDIR)/coverage/python.txt."
+
+.PHONY: xml
+xml:
+	$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
+	@echo
+	@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
+
+.PHONY: pseudoxml
+pseudoxml:
+	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
+	@echo
+	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+
+.PHONY: dummy
+dummy:
+	$(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy
+	@echo
+	@echo "Build finished. Dummy builder generates no files."
+
+favicon.ico: logo.svg
+	convert -density 384 -background transparent -define icon:auto-resize -colors 256 logo.svg favicon.ico
+
+logo.png: logo.svg
+	inkscape -z -e logo.png -w 100 -h 100 logo.svg

docs/README.md

@@ -0,0 +1,15 @@
+# Instaloader Documentation
+
+This directory contains sources for the Instaloader documentation.
+
+The documentation is hosted at:
+https://instaloader.github.io/
+
+## Build
+
+The documentation is created with [Sphinx](http://www.sphinx-doc.org/). To build it, use
+
+```
+pip3 install -r requirements.txt
+make html
+```

docs/_templates/layout.html

@@ -0,0 +1,83 @@
+{% extends "!layout.html" %}
+{% set html5_doctype = True %}
+{% set script_files = script_files + ['_static/instaloaderdoc.js'] %}
+{% block htmltitle %}
+  {% if pagename == "index" %}
+    <title>{{ title|striptags|e }}</title>
+  {% else %}
+    <title>{{ title|striptags|e }}{{ titlesuffix }}</title>
+  {% endif %}
+{% endblock %}
+{% block extrahead %}
+  {{ super() }}
+  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
+  <meta name="google-site-verification" content="FNUsDdkOJM9obm9QEdrNBVd9AtAzHzA-aaSMCUrS6C8"/>
+  {% if pagename == "index" %}
+    <link rel="canonical" href="https://instaloader.github.io/"/>
+  {% else %}
+    <link rel="canonical" href="https://instaloader.github.io/{{ pagename }}.html"/>
+  {% endif %}
+{% endblock %}
+{% block header %}
+  <nav class="navbar navbar-light navbar-expand bg-light border-bottom">
+    <a class="navbar-brand" href="{{ pathto(master_doc) }}">
+      <img src="{{ pathto('_static/' + logo, 1) }}" width="30" height="30" class="d-inline-block align-top" alt="logo">
+      <span class="ml-3">Instaloader</span>
+    </a>
+    <ul class="navbar-nav mr-auto">
+      <li class="nav-item ml-0 ml-sm-2">
+        <a class="nav-link"
+           href="https://github.com/instaloader/instaloader/releases/tag/v{{ current_release }}"
+           title="Released on {{ current_release_date }}" data-toggle="tooltip">
+          v<b>{{ current_release }}</b>
+        </a>
+      </li>
+    </ul>
+    <ul class="navbar-nav ml-auto d-none d-sm-flex">
+      <li class="nav-item">
+        <a class="nav-link" href="https://github.com/instaloader/instaloader/issues">
+          Issues
+        </a>
+      </li>
+      <li class="nav-item">
+        <a class="nav-link" href="https://github.com/instaloader/instaloader/releases">
+          Releases
+        </a>
+      </li>
+      <li class="nav-item">
+        <a class="nav-link" href="https://github.com/instaloader/instaloader">
+          GitHub
+        </a>
+      </li>
+    </ul>
+    <button class="d-inline-block d-md-none btn btn-outline-primary btn-sm" type="button"
+            data-toggle="collapse" data-target="#instaloaderdocSidebar" aria-controls="instaloaderdocSidebar"
+            aria-expanded="false" aria-label="Toggle navigation">
+      <span class="navbar-toggler-icon"></span>
+    </button>
+  </nav>
+{% endblock %}
+{% block relbar1 %}{% endblock %}
+{% block relbar2 %}{% endblock %}
+{% block footer %}{% endblock %}
+{% block content %}
+  <div class="container-fluid">
+    <div class="row">
+      <div class="instaloaderdoc-sidebar collapse d-md-block col-md-3 col-xl-2 bg-light"
+           id="instaloaderdocSidebar">
+        <nav role="navigation" aria-label="Table of Contents">
+          <form action="{{ pathto('search') }}" method="get" role="search" class="border-bottom mr-2 pb-2 mb-2 px-1">
+            <label class="sr-only" for="searchQueryInput">Search</label>
+            <input type="search" name="q" id="searchQueryInput" class="form-control my-2"
+                   placeholder="Search..." aria-label="Search">
+          </form>
+          <h4>Contents</h4>
+          {{ toctree(maxdepth=-1) }}
+        </nav>
+      </div>
+      <main class="col-12 col-md-9 col-xl-10 pt-2" role="main">
+        {% block body %}{% endblock %}
+      </main>
+    </div>
+  </div>
+{% endblock %}

docs/_templates/page.html

@@ -0,0 +1,10 @@
+{% extends "!page.html" %}
+{% block body %}
+{{ super() }}
+{% if next and prev %}
+<h2>Next Section</h2>
+<p>
+  <a class="reference internal" href="{{ next.link|e }}">{{ next.title }}</a>
+</p>
+{% endif %}
+{% endblock %}

docs/_templates/search.html

@@ -0,0 +1,28 @@
+{% extends "!search.html" %}
+
+{% block body %}
+  <div id="fallback" class="admonition warning">
+    <script type="text/javascript">$('#fallback').hide();</script>
+    <p>
+      {% trans %}Please activate JavaScript to enable the search
+        functionality.{% endtrans %}
+    </p>
+  </div>
+  {% if search_performed %}
+    <h1>{{ _('Search Results') }}</h1>
+    {% if not search_results %}
+      <p>{{ _('Your search did not match any documents. Please make sure that all words are spelled correctly and that you\'ve selected enough categories.') }}</p>
+    {% endif %}
+  {% endif %}
+  <div id="search-results">
+    {% if search_results %}
+      <ul>
+        {% for href, caption, context in search_results %}
+          <li><a href="{{ pathto(item.href) }}">{{ caption }}</a>
+            <div class="context">{{ context|e }}</div>
+          </li>
+        {% endfor %}
+      </ul>
+    {% endif %}
+  </div>
+{% endblock %}

docs/as-module.rst

@@ -0,0 +1,107 @@
+.. meta::
+   :description:
+      Documentation of Instaloader module, a powerful and intuitive
+      Python library to download Instagram media and metadata.
+
+.. _python-module-instaloader:
+
+Python Module :mod:`instaloader`
+--------------------------------
+
+.. module:: instaloader
+
+.. highlight:: python
+
+Instaloader exposes its internally used methods and structures as a Python
+module, making it a powerful and intuitive Python API for Instagram,
+allowing to further customize obtaining media and metadata.
+
+Start with getting an instance of :class:`Instaloader`::
+
+    import instaloader
+
+    # Get instance
+    L = instaloader.Instaloader()
+
+    # Optionally, login or load session
+    L.login(USER, PASSWORD)        # (login)
+    L.interactive_login(USER)      # (ask password on terminal)
+    L.load_session_from_file(USER) # (load session created w/
+                                   #  `instaloader -l USERNAME`)
+
+:mod:`instaloader` provides the :class:`Post` structure, which represents a
+picture, video or sidecar (set of multiple pictures/videos) posted in a user's
+profile. :class:`Instaloader` provides methods to iterate over Posts from a
+certain source::
+
+    for post in instaloader.Hashtag.from_name(L.context, 'cat').get_posts():
+        # post is an instance of instaloader.Post
+        L.download_post(post, target='#cat')
+
+:class:`Post` instances can be created with:
+
+- :func:`Post.from_shortcode`
+   Use a Post shortcode (part of the Post URL,
+   ``https://www.instagram.com/p/SHORTCODE/``) to create a Post
+   object::
+
+      post = Post.from_shortcode(L.context, SHORTCODE)
+
+- :meth:`Profile.get_posts`
+   All media of a :class:`Profile`.
+
+- :meth:`Profile.get_saved_posts`
+   Media that the user marked as saved (:class:`Profile` must be own profile
+   for this to work)
+
+- :meth:`Instaloader.get_feed_posts`
+   Media in the user's feed.
+
+- :meth:`Instaloader.get_explore_posts`
+   Media that is suggested by Instagram to explore.
+
+- :meth:`Hashtag.get_posts`
+   Media associated with given hashtag.
+
+With the :class:`Profile` class, Instaloader also makes it easy to access
+metadata of a Profile. :class:`Profile` instances can be created with:
+
+- :meth:`Profile.from_username`::
+
+     profile = Profile.from_username(L.context, USERNAME)
+
+- :meth:`Profile.from_id`
+   given its User ID. This allows to easily lookup a Profile's username given
+   its ID::
+
+      Profile.from_id(L.context, USERID).username
+
+- :meth:`Profile.get_followees`
+   Profiles that are followed by given user.
+
+- :meth:`Profile.get_followers`
+   Profiles that follow given user.
+
+- :attr:`Post.owner_profile`, :attr:`Story.owner_profile`, :attr:`StoryItem.owner_profile`
+   Owner profile of particular object.
+
+- :meth:`Post.get_likes`
+   Profiles that liked a given :class:`Post`
+
+- :attr:`PostComment.owner`
+   Profile of a Post comment.
+
+For a list of a few code examples that use the Instaloader module, see
+:ref:`codesnippets`.
+
+The reference of the classes and functions provided by the :mod:`instaloader` module is
+divided into the following subsections:
+
+.. toctree::
+   :maxdepth: 2
+
+   module/instaloader
+   module/structures
+   module/nodeiterator
+   module/instaloadercontext
+   module/exceptions

docs/basic-usage.rst

@@ -0,0 +1,349 @@
+.. meta::
+   :description:
+      How to download pictures from Instagram. Description of basic
+      usage of Instaloader, free tool to download photos from public
+      and private profiles, hashtags, stories, feeds, saved media, and
+      their metadata, comments and captions.
+
+.. _download-pictures-from-instagram:
+
+Download Pictures from Instagram
+---------------------------------
+
+.. highlight:: none
+
+Here we describe how to use Instaloader to download pictures from Instagram. If
+you do not have Instaloader installed yet, see :ref:`install`.
+
+.. NOTE that Section "Basic Usage" is duplicated in README.rst.
+
+Basic Usage
+^^^^^^^^^^^
+
+To **download all pictures and videos of a profile**, as well as the
+**profile picture**, do
+
+::
+
+    instaloader profile [profile ...]
+
+where ``profile`` is the name of a profile you want to download. Instead
+of only one profile, you may also specify a list of profiles.
+
+To later **update your local copy** of that profiles, you may run
+
+::
+
+    instaloader --fast-update profile [profile ...]
+
+If :option:`--fast-update` is given, Instaloader stops when arriving at the
+first already-downloaded picture.
+
+Alternatively, you can use :option:`--latest-stamps` to have Instaloader store
+the time each profile was last downloaded and only download newer media:
+
+::
+
+    instaloader --latest-stamps -- profile [profile ...]
+
+With this option it's possible to move or delete downloaded media and still keep
+the archive updated.
+
+When updating profiles, Instaloader
+automatically **detects profile name changes** and renames the target directory
+accordingly.
+
+Instaloader can also be used to **download private profiles**. To do so,
+invoke it with
+
+::
+
+    instaloader --login=your_username profile [profile ...]
+
+When logging in, Instaloader **stores the session cookies** in a file in your
+home directory, which will be reused later the next time :option:`--login`
+is given.  So you can download private profiles **non-interactively** when you
+already have a valid session cookie file.
+
+.. _what-to-download:
+
+What to Download
+^^^^^^^^^^^^^^^^
+
+.. targets-start
+
+Instaloader supports the following targets:
+
+- ``profile``
+   Public profile, or private profile with :option:`--login`.
+
+   If an already-downloaded profile has been renamed, Instaloader automatically
+   finds it by its unique ID and renames the folder accordingly.
+
+   Besides the profile's posts, its current profile picture is downloaded. For
+   each profile you download,
+
+   - :option:`--stories`
+      instructs Instaloader to also **download the user's stories**,
+
+   - :option:`--highlights`
+      to **download the highlights of that profile**,
+
+   - :option:`--tagged`
+      to **download posts where the user is tagged**, and
+
+   - :option:`--igtv`
+      to **download IGTV videos**.
+
+- ``"#hashtag"``
+   Posts with a certain **hashtag** (the quotes are usually necessary).
+
+- ``%location id``
+   Posts tagged with a given location; the location ID is the numerical ID
+   Instagram labels a location with (e.g.
+   \https://www.instagram.com/explore/locations/**362629379**/plymouth-naval-memorial/).
+   Requires :option:`--login`.
+
+   .. versionadded:: 4.2
+
+- ``:stories``
+   The currently-visible **stories** of your followees (requires
+   :option:`--login`).
+
+- ``:feed``
+   Your **feed** (requires :option:`--login`).
+
+- ``:saved``
+   Posts which are marked as **saved** (requires :option:`--login`).
+
+- ``@profile``
+   All profiles that are followed by ``profile``, i.e. the *followees* of
+   ``profile`` (requires :option:`--login`).
+
+- ``-post``
+   Replace **post** with the post's shortcode to download single post. Must be preceded by ``--`` in
+   the argument list to not be mistaken as an option flag::
+
+    instaloader -- -B_K4CykAOtf
+
+
+
+   .. versionadded:: 4.1
+
+.. targets-end
+
+Instaloader goes through all media matching the specified targets and
+downloads the pictures and videos and their captions. You can specify
+
+- :option:`--comments`
+   also **download comments** of each post,
+
+- :option:`--geotags`
+   **download geotags** of each post and save them as
+   Google Maps link (requires :option:`--login`),
+
+For a reference of all supported command line options, see
+:ref:`command-line-options`.
+
+.. _filename-specification:
+
+Filename Specification
+^^^^^^^^^^^^^^^^^^^^^^
+
+For each target, Instaloader creates a directory named after the target,
+i.e. ``profile``, ``#hashtag``, ``%location id``, ``:feed``, etc. and therein saves the
+posts in files named after the post's timestamp.
+
+:option:`--dirname-pattern` allows to configure the directory name of each
+target. The default is ``--dirname-pattern={target}``. In the dirname
+pattern, the token ``{target}`` is replaced by the target name, and
+``{profile}`` is replaced by the owner of the post which is downloaded.
+
+:option:`--filename-pattern` configures the path of the post and story's files relative
+to the target directory that is specified with :option:`--dirname-pattern`.
+The default is ``--filename-pattern={date_utc}_UTC``.
+The tokens ``{target}`` and ``{profile}`` are replaced like in the
+dirname pattern.
+
+:option:`--title-pattern` is similar to :option:`--filename-pattern`, but for profile
+pics, hashtag profile pics, and highlight covers. The default is
+``{date_utc}_UTC_{typename}`` if :option:`--dirname-pattern` contains ``{target}`` or
+``{profile}``, or ``{target}_{date_utc}_UTC_{typename}`` if it does not. Some tokens
+are not supported for this option, see below for details.
+
+The following tokens are defined for usage with
+:option:`--filename-pattern` and :option:`--title-pattern`:
+
+- ``{target}``
+   Target name (as given in Instaloader command line)
+
+- ``{profile}`` (same as ``{owner_username}``)
+   Owner of the Post / StoryItem / ProfilePic. For hashtag profile pics and
+   highlight covers, equivalent to ``{target}``.
+
+- ``{owner_id}``
+   Unique integer ID of owner profile. For hashtag profile pics, equivalent to
+   ``{target}``.
+
+- ``{shortcode}``
+   Shortcode (identifier string). Not available for :option:`--title-pattern`.
+
+- ``{mediaid}``
+   Integer representation of shortcode. Not available for :option:`--title-pattern`.
+
+- ``{filename}``
+   Instagram's internal filename.
+
+- ``{date_utc}`` (same as ``{date}``)
+   Creation time in UTC timezone.
+   `strftime()-style formatting options <https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>`__
+   are supported as format specifier. The default date format specifier used by
+   Instaloader is::
+
+      {date_utc:%Y-%m-%d_%H-%M-%S}
+
+- ``{typename}``
+   Type of media being saved, such as GraphImage, GraphStoryVideo, profile_pic,
+   etc.
+
+For example, encode the poster's profile name in the filenames with::
+
+    instaloader --filename-pattern={date_utc}_UTC_{profile} "#hashtag"
+
+As another example, you may instruct Instaloader to store posts in a
+``PROFILE/YEAR/SHORTCODE.jpg`` directory structure::
+
+    instaloader --dirname-pattern={profile} --filename-pattern={date_utc:%Y}/{shortcode} <target> ...
+
+.. _filter-posts:
+
+Filter Posts
+^^^^^^^^^^^^
+
+.. py:currentmodule:: instaloader
+
+The options :option:`--post-filter` and :option:`--storyitem-filter`
+allow to specify criteria that posts or story items have to
+meet to be downloaded. If not given, all posts are downloaded.
+
+The filter string must be a
+`Python boolean expression <https://docs.python.org/3/reference/expressions.html#boolean-operations>`__
+where the attributes from :class:`Post` or
+:class:`StoryItem` respectively are defined.
+
+Id est, the following attributes can be used with both
+:option:`--post-filter` and :option:`--storyitem-filter`:
+
+- :attr:`~Post.owner_username` (str), :attr:`~Post.owner_id` (int)
+   Owner profile username / user ID.
+
+- :attr:`~Post.date_utc` (datetime), :attr:`~Post.date_local` (datetime)
+   Creation timestamp. Since :class:`~datetime.datetime` objects can be created
+   inside filter strings, this easily allows filtering by creation date. E.g.::
+
+      instaloader --post-filter="date_utc <= datetime(2018, 5, 31)" target
+
+- :attr:`~Post.is_video` (bool)
+   Whether Post/StoryItem is a video. For example, you may skip videos::
+
+      instaloader --post-filter="not is_video" target
+
+   This is not the same as :option:`--no-videos` and
+   :option:`--no-video-thumbnails`, since sidecar posts (posts that contain
+   multiple pictures/videos in one post) have this attribute set to False.
+
+As :option:`--post-filter`, the following attributes can be used additionally:
+
+- :attr:`~Post.viewer_has_liked` (bool)
+   Whether user (with :option:`--login`) has liked given post. To download the
+   pictures from your feed that you have liked::
+
+      instaloader --login=your_username --post-filter=viewer_has_liked :feed
+
+- :attr:`~Post.likes` (int), :attr:`~Post.comments` (int)
+   Likes count / comments count. You might only want to download posts that
+   were either liked by yourself or by many others::
+
+      instaloader --login=your_username --post-filter="likes>100 or viewer_has_liked" profile
+
+- :attr:`~Post.caption_hashtags` (list of str) / :attr:`~Post.caption_mentions` (list of str)
+   ``#hashtags`` or ``@mentions`` (lowercased) in the Post's caption. For example, to
+   download posts of kittens that are cute::
+
+       instaloader --post-filter="'cute' in caption_hashtags" "#kitten"
+
+- :attr:`~Post.tagged_users` (list of str)
+   Lowercased usernames that are tagged in the Post.
+
+For :option:`--storyitem-filter`, the following additional attributes are
+defined:
+
+- :attr:`~StoryItem.expiring_utc` (datetime) / :attr:`~StoryItem.expiring_local` (datetime)
+   Timestamp when StoryItem will get unavailable.
+
+.. _metadata-text-files:
+
+Metadata Text Files
+^^^^^^^^^^^^^^^^^^^
+
+Unless :option:`--no-captions` is given, Instaloader creates a ``.txt`` file
+along with each post where the Post's caption is saved.
+
+You can customize what metadata to save for each Post or StoryItem with
+:option:`--post-metadata-txt` and :option:`--storyitem-metadata-txt`. The
+default is ``--post-metadata-txt={caption}`` and no storyitem metadata txt.
+These strings are formatted similar as the path patterns described in :ref:`filename-specification` and
+the result is saved in text files, unless it is empty.
+
+Specifying these options multiple times results in output having multiple lines,
+in the order they were given to Instaloader.
+
+The field names are evaluated to :class:`Post` or :class:`StoryItem` attributes,
+and as such, the same fields are supported as in :ref:`filename-specification`
+and :ref:`filter-posts`.
+
+For example, to save the current number of likes for each post, rather than
+the post's caption::
+
+   instaloader --post-metadata-txt="{likes} likes." <target>
+
+Note that with this feature, it is possible to easily and fastly extract
+additional metadata of already-downloaded posts, by reimporting their JSON
+files. Say, you now also want to export the number of comments the Posts had
+when they were downloaded::
+
+   instaloader --post-metadata-txt="{likes} likes, {comments} comments." <target>/*.json.xz
+
+.. _instaloader-as-cronjob:
+
+Instaloader as Cronjob
+^^^^^^^^^^^^^^^^^^^^^^
+
+Instaloader is suitable for running as a cronjob to periodically update your
+personal Instagram archive. The :option:`--quiet` option disables user
+interactions and logging of non-error messages. To non-interactively use
+Instaloader logged-in, create a session file::
+
+   instaloader --login=your_username
+
+Then use the same username in your cronjob to load the session and download
+the given targets::
+
+   instaloader --login=your_username --quiet target [...]
+
+Instaloader saves the session file to
+``~/.config/instaloader/session-YOUR-USERNAME``. See
+:option:`--sessionfile` option for how to override this path.
+
+Programming Instaloader
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If your task cannot be done with the command line interface of Instaloader,
+consider taking a look at the :ref:`python-module-instaloader`.
+Instaloader exposes its internally used methods and structures, making it a
+powerful and intuitive Python API for Instagram, allowing to further customize
+obtaining media and metadata.
+
+Also see :ref:`codesnippets`, where we collect a few example scripts that use
+Instaloader for simple tasks that cannot be done with the command line
+interface.

docs/cli-options.rst

@@ -0,0 +1,346 @@
+.. _command-line-options:
+
+Command Line Options
+====================
+
+Instaloader is invoked with::
+
+   $ instaloader [options] target [target ...]
+
+where ``target`` is a ``profile``, a ``"#hashtag"``, ``@profile`` (all profiles
+that *profile* is following), ``%location ID``, or if logged in ``:feed`` (pictures from your
+feed), ``:stories`` (stories of your followees) or ``:saved`` (collection of
+posts marked as saved).
+
+Here we explain the additional options that can be given to Instaloader to
+customize its behavior.  For an
+introduction on how to use Instaloader, see
+:ref:`download-pictures-from-instagram`.
+
+To get a list of all flags, their abbreviations and
+their descriptions, you may also run::
+
+   instaloader --help
+
+Targets
+^^^^^^^
+
+Specify a list of targets. For each of these, Instaloader creates a folder and
+stores all posts along with the pictures's captions there.
+
+.. include:: basic-usage.rst
+   :start-after: targets-start
+   :end-before: targets-end
+
+- ``filename.json[.xz]``
+   Re-Download the given object
+
+- ``+args.txt``
+   Read targets (and options) from given text file. See :option:`+args.txt`.
+
+What to Download of each Post
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. option:: --no-pictures
+
+   Do not download post pictures. Cannot be used together with
+   :option:`--fast-update`. Implies :option:`--no-video-thumbnails`, does not
+   imply :option:`--no-videos`.
+
+   .. versionadded:: 4.1
+
+.. option:: --no-videos, -V
+
+   Do not download videos.
+
+.. option:: --no-video-thumbnails
+
+   Do not download thumbnails of videos.
+
+.. option:: --geotags, -G
+
+   Download geotags when available. Geotags are stored as a text file with
+   the location's name and a Google Maps link. This requires an additional
+   request to the Instagram server for each picture. Requires :option:`--login`.
+
+.. option:: --comments, -C
+
+   Download and update comments for each post. This requires an additional
+   request to the Instagram server for each post, which is why it is disabled by
+   default.
+
+.. option:: --no-captions
+
+   Do not create txt files.
+
+.. option:: --post-metadata-txt
+
+   Template to write in txt file for each Post. See :ref:`metadata-text-files`.
+
+.. option:: --storyitem-metadata-txt
+
+   Template to write in txt file for each StoryItem. See
+   :ref:`metadata-text-files`.
+
+.. option:: --slide
+
+   Download only selected images of a sidecar. You can select single images using their
+   index in the sidecar starting with the leftmost or you can specify a range of images
+   with the following syntax: ``start_index-end_index``. Example:
+   ``--slide 1`` will select only the first image, ``--slide last`` only the last one and ``--slide 1-3`` will select only
+   the first three images.
+
+   .. versionadded:: 4.6
+
+.. option:: --no-metadata-json
+
+   Do not create a JSON file containing the metadata of each post.
+
+.. option:: --no-compress-json
+
+   Do not xz compress JSON files, rather create pretty formatted JSONs.
+
+What to Download of each Profile
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. option:: --no-posts
+
+   Do not download regular posts.
+
+   .. versionadded:: 4.1
+
+.. option:: --no-profile-pic
+
+   Do not download profile picture.
+
+.. option:: --stories, -s
+
+   Also download stories of each profile that is downloaded. Requires
+   :option:`--login`.
+
+.. option:: --highlights
+
+   Also download highlights of each profile that is downloaded. Requires
+   :option:`--login`.
+
+   .. versionadded:: 4.1
+
+.. option:: --tagged
+
+   Also download posts where each profile is tagged.
+
+   .. versionadded:: 4.1
+
+.. option:: --igtv
+
+   Also download IGTV videos.
+
+   .. versionadded:: 4.3
+
+Which Posts to Download
+^^^^^^^^^^^^^^^^^^^^^^^
+
+.. option:: --fast-update, -F
+
+   For each target, stop when encountering the first already-downloaded picture.
+   This flag is recommended when you use Instaloader to update your personal
+   Instagram archive.
+
+.. option:: --latest-stamps [STAMPSFILE]
+
+   Works similarly to :option:`--fast-update`, but instead of relying on already
+   downloaded media, the time each profile was downloaded is stored, and only
+   media newer than the last download is fetched. This allows updating your
+   personal Instagram archive while emptying the target directories.
+
+   Only works for media associated with a specific profile, and that is returned
+   in chronological order: profile posts, profile stories, profile IGTV posts
+   and profile tagged posts.
+
+   By default, the information is stored in
+   ``~/.config/instaloader/latest-stamps.ini``, but you can specify an
+   alternative location.
+
+   .. versionadded:: 4.8
+
+.. option:: --post-filter filter, --only-if filter
+
+   Expression that, if given, must evaluate to True for each post to be
+   downloaded.  Must be a syntactically valid Python expression. Variables are
+   evaluated to :class:`instaloader.Post` attributes.  Example:
+   ``--post-filter=viewer_has_liked``. See :ref:`filter-posts` for more
+   examples.
+
+.. option:: --storyitem-filter filter
+
+   Expression that, if given, must evaluate to True for each storyitem to be
+   downloaded.  Must be a syntactically valid Python expression. Variables are
+   evaluated to :class:`instaloader.StoryItem` attributes.
+   See :ref:`filter-posts` for more examples.
+
+.. option:: --count COUNT, -c
+
+   Do not attempt to download more than COUNT posts.  Applies to
+   ``#hashtag``, ``%location_id``, ``:feed``, and ``:saved``.
+
+
+Login (Download Private Profiles)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Instaloader can login to Instagram. This allows downloading private
+profiles. To login, pass the :option:`--login` option. Your session cookie (not your
+password!) will be saved to a local file to be reused next time you want
+Instaloader to login.
+
+.. option:: --login YOUR-USERNAME, -l YOUR-USERNAME
+
+   Login name (profile name) for your Instagram account.
+
+.. option:: --sessionfile SESSIONFILE, -f SESSIONFILE
+
+   Path for loading and storing session key file.  Defaults to
+   ``~/.config/instaloader/session-YOUR-USERNAME``.
+
+.. option:: --password YOUR-PASSWORD, -p YOUR-PASSWORD
+
+   Password for your Instagram account.  Without this option, you'll be prompted
+   for your password interactively if there is not yet a valid session file.
+
+   .. warning:: Using :option:`--password` option is discouraged for security
+      reasons.  Enter your password interactively when asked, or use the
+      sessionfile feature (:option:`--sessionfile` to customize path).
+
+How to Download
+^^^^^^^^^^^^^^^
+
+.. option:: --dirname-pattern DIRNAME_PATTERN
+
+   Name of directory where to store posts. ``{profile}`` is replaced by the
+   profile name, ``{target}`` is replaced by the target you specified, i.e.
+   either ``:feed``, ``#hashtag`` or the profile name. Defaults to ``{target}``.
+   See :ref:`filename-specification`.
+
+.. option:: --filename-pattern FILENAME_PATTERN
+
+   Prefix of filenames for posts and stories, relative to the directory given with
+   :option:`--dirname-pattern`. ``{profile}`` is replaced by the profile name,
+   ``{target}`` is replaced by the target you specified, i.e.  either ``:feed``,
+   ``#hashtag`` or the profile name. Defaults to ``{date_utc}_UTC``.
+   See :ref:`filename-specification` for a list of supported tokens.
+
+.. option:: --title-pattern TITLE_PATTERN
+
+   Prefix of filenames for profile pics, hashtag profile pics, and highlight
+   covers, relative to the directory given with :option:`--dirname-pattern`.
+   Defaults to ``{date_utc}_UTC_{typename}`` if :option:`--dirname-pattern`
+   contains ``{target}`` or ``{profile}``, otherwise defaults to
+   ``{target}_{date_utc}_UTC_{typename}``.
+   See :ref:`filename-specification` for a list of supported tokens.
+
+   .. versionadded:: 4.8
+
+.. option:: --sanitize-paths
+
+   Force sanitization of paths so that the resulting file and directory names
+   are valid on both Windows and Unix.
+
+   .. versionadded:: 4.9
+
+.. option:: --resume-prefix prefix
+
+   For many targets, Instaloader is capable of resuming a previously-aborted
+   download loop.  To do so, it creates a JSON file within the target directory
+   when interrupted.  This option controls the prefix for filenames that are
+   used to save the information to resume an interrupted download.  The default
+   prefix is ``iterator``.
+
+   Resuming an interrupted download is supported for the following targets:
+    - Profile posts,
+    - Profile IGTV posts (:option:`--igtv`),
+    - Profile tagged posts (:option:`--tagged`),
+    - Saved posts (``:saved``),
+    - Hashtags.
+
+   This feature is enabled by default for targets where it is supported;
+   :option:`--resume-prefix` only changes the name of the iterator files.
+
+   To turn this feature off, use :option:`--no-resume`.
+
+   JSON files with resume information are always compressed, regardless of
+   :option:`--no-compress-json`.
+
+   .. versionadded:: 4.5
+
+.. option:: --no-resume
+
+   Do not resume a previously-aborted download iteration, and do not save such
+   information when interrupted.
+
+   .. versionadded:: 4.5
+
+.. option:: --user-agent USER_AGENT
+
+   User Agent to use for HTTP requests. Per default, Instaloader pretends being
+   Chrome/92 on Linux.
+
+.. option:: --max-connection-attempts N
+
+   Maximum number of connection attempts until a request is aborted. Defaults
+   to ``3``. If a connection fails, it can be manually skipped by hitting
+   :kbd:`Control-c`. Set this to ``0`` to retry infinitely.
+
+.. option:: --request-timeout N
+
+   Seconds to wait before timing out a connection request. Defaults to 300.
+
+   .. versionadded:: 4.3
+
+   .. versionchanged:: 4.6
+      Enabled this option by default with a timeout of 300 seconds.
+
+.. option:: --abort-on STATUS_CODE_LIST
+
+   Comma-separated list of HTTP status codes that cause Instaloader to abort,
+   bypassing all retry logic.
+
+   For example, with ``--abort-on=302,400,429``, Instaloader will stop if a
+   request is responded with a 302 redirect, a Bad Request error, or a Too Many
+   Requests error.
+
+   .. versionadded:: 4.7
+
+.. option:: --no-iphone
+
+   Do not attempt to download iPhone version of images and videos.
+
+   .. versionadded:: 4.8
+
+Miscellaneous Options
+^^^^^^^^^^^^^^^^^^^^^
+
+.. option:: --quiet, -q
+
+   Disable user interaction, i.e. do not print messages (except errors) and fail
+   if login credentials are needed but not given.
+   This is handy for running :ref:`instaloader-as-cronjob`.
+
+.. option:: +args.txt
+
+   Read arguments from file `args.txt`, a shortcut to provide arguments from
+   file rather than command-line. This provides a convenient way to hide login
+   info from CLI, and can also be used to simplify management of long arguments.
+   You can provide more than one file at once, e.g.: ``+args1.txt +args2.txt``.
+
+   .. note::
+
+      Text file should separate arguments with line breaks.
+
+   `args.txt` example::
+
+      --login=MYUSERNAME
+      --password=MYPASSWORD
+      --fast-update
+      profile1
+      profile2
+
+   .. versionadded:: 4.1

docs/codesnippets.rst

@@ -0,0 +1,124 @@
+.. _codesnippets:
+
+Advanced Instaloader Examples
+=============================
+
+.. currentmodule:: instaloader
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+Here we present code examples that use the :ref:`python-module-instaloader` for
+more advanced Instagram downloading or metadata mining than what is possible
+with the Instaloader command line interface.
+
+The scripts presented here can be downloaded from our source tree:
+`instaloader/docs/codesnippets/ <https://github.com/instaloader/instaloader/tree/master/docs/codesnippets>`__
+
+.. For each code snippet:
+   - title
+   - brief description of what it does / motivation / how it works
+   - code, or link to code
+   - link to discussion issue
+   - link used methods
+
+Download Posts in a Specific Period
+-----------------------------------
+
+To only download Instagram pictures (and metadata) that are within a specific
+period, you can simply use :func:`~itertools.dropwhile` and
+:func:`~itertools.takewhile` from :mod:`itertools` on a generator that returns
+Posts in **exact chronological order**, such as :meth:`Profile.get_posts`.
+
+.. literalinclude:: codesnippets/121_since_until.py
+
+See also :class:`Post`, :meth:`Instaloader.download_post`.
+
+Discussed in :issue:`121`.
+
+The code example with :func:`~itertools.dropwhile` and
+:func:`~itertools.takewhile` makes the assumption that the post iterator returns
+posts in exact chronological order.  As discussed in :issue:`666`, the following
+approach fits for an **almost chronological order**, where up to *k* older posts
+are inserted into an otherwise chronological order, such as a Hashtag feed.
+
+.. literalinclude:: codesnippets/666_historical_hashtag_data.py
+
+Likes of a Profile / Ghost Followers
+------------------------------------
+
+To obtain a list of your inactive followers, i.e. followers that did not like
+any of your pictures, you can use this approach.
+
+.. literalinclude:: codesnippets/120_ghost_followers.py
+
+See also :meth:`Profile.get_posts`, :meth:`Post.get_likes`,
+:meth:`Profile.get_followers`, :meth:`Instaloader.load_session_from_file`,
+:meth:`Profile.from_username`.
+
+Discussed in :issue:`120`.
+
+Track Deleted Posts
+-------------------
+
+This script uses Instaloader to obtain a list of currently-online Instagram and
+compares it with the set of posts that you already have downloaded.  It outputs
+a list of posts which are online but not offline (i.e. not yet downloaded) and a
+list of posts which are offline but not online (i.e. deleted in the profile).
+
+.. literalinclude:: codesnippets/56_track_deleted.py
+
+See also :func:`load_structure_from_file`, :meth:`Profile.from_username`,
+:meth:`Profile.get_posts`, :class:`Post`.
+
+Discussed in :issue:`56`.
+
+Only one Post per User
+----------------------
+
+To download only the single most recent post per user within a hashtag feed,
+this snippet uses a :class:`set` that contains the users of whom a post has
+already been downloaded. For each post, it checks whether the post's creator is
+already contained in that set. If not, the post is downloaded from Instagram and
+the user is added to that set.
+
+.. literalinclude:: codesnippets/113_only_one_per_user.py
+
+See also :class:`Post`, :meth:`Instaloader.download_post`,
+:attr:`Post.owner_profile`, :class:`Profile`.
+
+Discussed in :issue:`113`.
+
+Top X Posts of User
+-------------------
+
+With Instaloader, it is easy to download the few most-liked pictures of a user.
+
+.. literalinclude:: codesnippets/194_top_x_of_user.py
+
+Discussed in :issue:`194`.
+
+Metadata JSON Files
+-------------------
+
+The JSON files Instaloader saves along with each Post contain all the metadata
+that has been retrieved from Instagram while downloading the picture and
+associated required information.
+
+With `jq <https://stedolan.github.io/jq/>`__, a command-line JSON processor, the
+metadata can be easily post-processed. For example, Instaloader's JSON files can
+be pretty-formatted with:
+
+.. code-block:: none
+
+   xzcat 2018-05-13_11-18-45_UTC.json.xz | jq .node
+
+However, Instaloader tries to do as few metadata requests as possible, so,
+depending on how Instaloader has been invoked, it may occur that these files do
+not contain the complete available metadata structure. Nevertheless, the file
+can be loaded into Instaloader with :func:`load_structure_from_file` and the
+required metadata then be accessed via the :class:`Post` or :class:`Profile`
+attributes, which trigger an Instagram request if that particular information is
+not present in the JSON file.

docs/codesnippets/113_only_one_per_user.py

@@ -0,0 +1,14 @@
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = instaloader.Hashtag.from_name(L.context, 'urbanphotography').get_posts()
+
+users = set()
+
+for post in posts:
+    if not post.owner_profile in users:
+        L.download_post(post, '#urbanphotography')
+        users.add(post.owner_profile)
+    else:
+        print("{} from {} skipped.".format(post, post.owner_profile))

docs/codesnippets/120_ghost_followers.py

@@ -0,0 +1,27 @@
+import instaloader
+
+L = instaloader.Instaloader()
+
+USER = "your_account"
+PROFILE = USER
+
+# Load session previously saved with `instaloader -l USERNAME`:
+L.load_session_from_file(USER)
+
+profile = instaloader.Profile.from_username(L.context, PROFILE)
+
+likes = set()
+print("Fetching likes of all posts of profile {}.".format(profile.username))
+for post in profile.get_posts():
+    print(post)
+    likes = likes | set(post.get_likes())
+
+print("Fetching followers of profile {}.".format(profile.username))
+followers = set(profile.get_followers())
+
+ghosts = followers - likes
+
+print("Storing ghosts into file.")
+with open("inactive-users.txt", 'w') as f:
+    for ghost in ghosts:
+        print(ghost.username, file=f)

docs/codesnippets/121_since_until.py

@@ -0,0 +1,15 @@
+from datetime import datetime
+from itertools import dropwhile, takewhile
+
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = instaloader.Profile.from_username(L.context, "instagram").get_posts()
+
+SINCE = datetime(2015, 5, 1)
+UNTIL = datetime(2015, 3, 1)
+
+for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
+    print(post.date)
+    L.download_post(post, "instagram")

docs/codesnippets/194_top_x_of_user.py

@@ -0,0 +1,17 @@
+from itertools import islice
+from math import ceil
+
+from instaloader import Instaloader, Profile
+
+PROFILE = ...        # profile to download from
+X_percentage = 10    # percentage of posts that should be downloaded
+
+L = Instaloader()
+
+profile = Profile.from_username(L.context, PROFILE)
+posts_sorted_by_likes = sorted(profile.get_posts(),
+                               key=lambda p: p.likes + p.comments,
+                               reverse=True)
+
+for post in islice(posts_sorted_by_likes, ceil(profile.mediacount * X_percentage / 100)):
+    L.download_post(post, PROFILE)

docs/codesnippets/56_track_deleted.py

@@ -0,0 +1,34 @@
+from glob import glob
+from sys import argv
+from os import chdir
+
+from instaloader import Instaloader, Post, Profile, load_structure_from_file
+
+# Instaloader instantiation - you may pass additional arguments to the constructor here
+L = Instaloader()
+
+# If desired, load session previously saved with `instaloader -l USERNAME`:
+#L.load_session_from_file(USERNAME)
+
+try:
+    TARGET = argv[1]
+except IndexError:
+    raise SystemExit("Pass profile name as argument!")
+
+# Obtain set of posts that are on hard disk
+chdir(TARGET)
+offline_posts = set(filter(lambda s: isinstance(s, Post),
+                           (load_structure_from_file(L.context, file)
+                            for file in (glob('*.json.xz') + glob('*.json')))))
+
+# Obtain set of posts that are currently online
+post_iterator = Profile.from_username(L.context, TARGET).get_posts()
+online_posts = set(post_iterator)
+
+if online_posts - offline_posts:
+    print("Not yet downloaded posts:")
+    print(" ".join(str(p) for p in (online_posts - offline_posts)))
+
+if offline_posts - online_posts:
+    print("Deleted posts:")
+    print(" ".join(str(p) for p in (offline_posts - online_posts)))

docs/codesnippets/615_import_firefox_session.py

@@ -0,0 +1,53 @@
+from argparse import ArgumentParser
+from glob import glob
+from os.path import expanduser
+from platform import system
+from sqlite3 import OperationalError, connect
+
+try:
+    from instaloader import ConnectionException, Instaloader
+except ModuleNotFoundError:
+    raise SystemExit("Instaloader not found.\n  pip install [--user] instaloader")
+
+
+def get_cookiefile():
+    default_cookiefile = {
+        "Windows": "~/AppData/Roaming/Mozilla/Firefox/Profiles/*/cookies.sqlite",
+        "Darwin": "~/Library/Application Support/Firefox/Profiles/*/cookies.sqlite",
+    }.get(system(), "~/.mozilla/firefox/*/cookies.sqlite")
+    cookiefiles = glob(expanduser(default_cookiefile))
+    if not cookiefiles:
+        raise SystemExit("No Firefox cookies.sqlite file found. Use -c COOKIEFILE.")
+    return cookiefiles[0]
+
+
+def import_session(cookiefile, sessionfile):
+    print("Using cookies from {}.".format(cookiefile))
+    conn = connect(f"file:{cookiefile}?immutable=1", uri=True)
+    try:
+        cookie_data = conn.execute(
+            "SELECT name, value FROM moz_cookies WHERE baseDomain='instagram.com'"
+        )
+    except OperationalError:
+        cookie_data = conn.execute(
+            "SELECT name, value FROM moz_cookies WHERE host LIKE '%instagram.com'"
+        )
+    instaloader = Instaloader(max_connection_attempts=1)
+    instaloader.context._session.cookies.update(cookie_data)
+    username = instaloader.test_login()
+    if not username:
+        raise SystemExit("Not logged in. Are you logged in successfully in Firefox?")
+    print("Imported session cookie for {}.".format(username))
+    instaloader.context.username = username
+    instaloader.save_session_to_file(sessionfile)
+
+
+if __name__ == "__main__":
+    p = ArgumentParser()
+    p.add_argument("-c", "--cookiefile")
+    p.add_argument("-f", "--sessionfile")
+    args = p.parse_args()
+    try:
+        import_session(args.cookiefile or get_cookiefile(), args.sessionfile)
+    except (ConnectionException, OperationalError) as e:
+        raise SystemExit("Cookie import failed: {}".format(e))

docs/codesnippets/666_historical_hashtag_data.py

@@ -0,0 +1,31 @@
+from datetime import datetime
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = instaloader.Hashtag.from_name(L.context, "urbanphotography").get_posts()
+
+SINCE = datetime(2020, 5, 10)  # further from today, inclusive
+UNTIL = datetime(2020, 5, 11)  # closer to today, not inclusive
+
+k = 0  # initiate k
+#k_list = []  # uncomment this to tune k
+
+for post in posts:
+    postdate = post.date
+
+    if postdate > UNTIL:
+        continue
+    elif postdate <= SINCE:
+        k += 1
+        if k == 50:
+            break
+        else:
+            continue
+    else:
+        L.download_post(post, "#urbanphotography")
+        # if you want to tune k, uncomment below to get your k max
+        #k_list.append(k)
+        k = 0  # set k to 0
+
+#max(k_list)

docs/codesnippets/README.md

@@ -0,0 +1,4 @@
+# Advanced Instaloader Examples
+
+This directory contains the scripts that are presented at:
+https://instaloader.github.io/codesnippets.html

docs/conf.py

@@ -0,0 +1,379 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+# Instaloader documentation build configuration file, created by
+# sphinx-quickstart on Fri Aug 25 15:38:26 2017.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import platform
+import subprocess
+import sys
+import typing
+
+sys.path.insert(0, os.path.abspath('..'))
+
+import docs.sphinx_autodoc_typehints as sphinx_autodoc_typehints
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.githubpages',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.extlinks'
+]
+
+autodoc_default_flags = ['show-inheritance', 'members', 'undoc-members']
+
+autodoc_member_order = 'bysource'
+
+intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
+                       'requests': ('https://requests.readthedocs.io/en/latest/', None)}
+
+nitpick_ignore = [('py:class', 'typing.Tuple')]
+
+current_release = subprocess.check_output(["git", "describe", "--abbrev=0"]).decode("ascii")[1:-1]
+date_format = "%e %b %Y" if platform.system() != "Windows" else "%d %b %Y"
+current_release_date = subprocess.check_output(
+    ["git", "log", "-1", "--tags", "--format=%ad", "--date=format:"+date_format]).decode("ascii")[:-1]
+
+html_context = {'current_release': current_release, 'current_release_date': current_release_date}
+
+extlinks = {
+    'issue': (
+        'https://github.com/instaloader/instaloader/issues/%s',
+        'Issue #'
+    ),
+    'example': (
+        'https://raw.githubusercontent.com/instaloader/instaloader/master/docs/codesnippets/%s',
+        'Example '
+    ),
+}
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The encoding of source files.
+#
+# source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = 'Instaloader'
+copyright = '2017-2020, Alexander Graf and Andre Koch-Kramer'
+author = 'Alexander Graf and Andre Koch-Kramer'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+#version = '3.0'
+# The full version, including alpha/beta/rc tags.
+#release = '3.0rc0'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = "en"
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#
+# today = ''
+#
+# Else, today_fmt is used as the format for a strftime call.
+#
+# today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This patterns also effect to html_static_path and html_extra_path
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#
+# default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#
+# add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#
+add_module_names = False
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#
+# show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'fruity'
+
+# A list of ignored prefixes for module index sorting.
+# modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+# keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'basic'
+
+# Add any paths that contain custom themes here, relative to this directory.
+# html_theme_path = []
+
+# The name for this set of Sphinx documents.
+# "<project> v<release> documentation" by default.
+#
+# html_title = 'Instaloader v3.0rc0'
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#
+# html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#
+html_logo = "logo.png"
+
+# The name of an image file (relative to this directory) to use as a favicon of
+# the docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#
+html_favicon = "favicon.ico"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#
+# html_extra_path = []
+
+# If not None, a 'Last updated on:' timestamp is inserted at every page
+# bottom, using the given strftime format.
+# The empty string is equivalent to '%b %d, %Y'.
+#
+# html_last_updated_fmt = None
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#
+# html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#
+#html_sidebars = {'**': ["relations.html", "links.html"] }
+html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#
+# html_additional_pages = {}
+
+# If false, no module index is generated.
+#
+html_domain_indices = False
+
+# If false, no index is generated.
+#
+html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#
+# html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#
+html_show_sourcelink = False
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#
+html_show_sphinx = False
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#
+html_show_copyright = False
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#
+# html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+# html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+#   'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
+#   'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr', 'zh'
+#
+# html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# 'ja' uses this config value.
+# 'zh' user can custom change `jieba` dictionary path.
+#
+# html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#
+# html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Instaloaderdoc'
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+     # The paper size ('letterpaper' or 'a4paper').
+     #
+     # 'papersize': 'letterpaper',
+
+     # The font size ('10pt', '11pt' or '12pt').
+     #
+     # 'pointsize': '10pt',
+
+     # Additional stuff for the LaTeX preamble.
+     #
+     # 'preamble': '',
+
+     # Latex figure (float) alignment
+     #
+     # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+    (master_doc, 'Instaloader.tex', 'Instaloader Documentation',
+     'Alexander Graf, André Koch-Kramer', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#
+# latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#
+# latex_use_parts = False
+
+# If true, show page references after internal links.
+#
+# latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#
+# latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#
+# latex_appendices = []
+
+# It false, will not define \strong, \code, 	itleref, \crossref ... but only
+# \sphinxstrong, ..., \sphinxtitleref, ... To help avoid clash with user added
+# packages.
+#
+# latex_keep_old_macro_names = True
+
+# If false, no module index is generated.
+#
+# latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'instaloader', 'Instaloader Documentation',
+     [author], 1)
+]
+
+# If true, show URL addresses after external links.
+#
+# man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (master_doc, 'Instaloader', 'Instaloader Documentation',
+     author, 'Instaloader', 'One line description of project.',
+     'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#
+# texinfo_appendices = []
+
+# If false, no module index is generated.
+#
+# texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#
+# texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#
+# texinfo_no_detailmenu = False
+
+
+def setup(app):
+    typing.TYPE_CHECKING = True
+    app.add_stylesheet("instaloaderdoc.css")
+    app.connect('autodoc-process-signature', sphinx_autodoc_typehints.process_signature)
+    app.connect('autodoc-process-docstring', sphinx_autodoc_typehints.process_docstring)

docs/contributing.rst

@@ -0,0 +1,140 @@
+.. _contributing:
+
+Contributing to Instaloader
+===========================
+
+As an open source project, Instaloader heavily depends on the contributions from
+its community.  In this document, we advise on how you may help Instaloader to
+become an even greater tool.
+
+Instaloader's development is organized on 
+`GitHub <https://github.com/instaloader/instaloader>`__, where Issues and Pull
+Requests are discussed.
+
+Contributing to an open source project is an unmatchable opportunity to improve
+and manifest programming skills. For issues that are good starting points to get
+involved in Instaloader development, as they do not require deep insight into
+Instaloader's code base, have a look at our
+`Issues labelled 'good first issue' <https://github.com/instaloader/instaloader/contribute>`__.
+
+This document covers several ways of how you can contribute back to Instaloader.
+
+Answering Questions
+-------------------
+
+The easiest way to help out is to answer questions. If you are interested in
+answering questions regarding Instaloader, good places to begin are
+
+- `Questions tagged 'instaloader' on Stack Overflow <https://stackoverflow.com/questions/tagged/instaloader>`__,
+- `Instaloader Issues labeled 'question' <https://github.com/instaloader/instaloader/issues?q=is%3Aissue+is%3Aopen+label%3Aquestion>`__.
+
+Reporting Bugs
+--------------
+
+If you encounter a bug, do not hesitate to report it in our
+`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. When
+reporting a problem, please keep the following in mind:
+
+- Ensure you **use the latest version** of Instaloader. The currently-installed
+  version can be found out with ``instaloader --version``.
+
+- Check whether there is a valid solution in our :ref:`troubleshooting` section.
+
+- Briefly **check whether the bug has already been reported**. If you find an
+  issue reporting the same bug you encountered, comment there rather than
+  opening a new issue. However, if unsure, please create a new issue.
+
+- State **how the bug can be reproduced**, i.e. how Instaloader was invoked
+  when the problem occurred (of course, you may anonymize profile names etc.).
+
+- Include all **error messages and tracebacks** in the report.
+
+- Even if it seems obvious, describe **which behavior you expected**
+  instead of what actually happened.
+
+- If we have closed an issue apparently inadvertently or inappropriately, please
+  let us know.
+
+Writing Code or Improving the Documentation
+-------------------------------------------
+
+Improvements of the Instaloader source or its documentation can be proposed as a
+`Pull Request <https://github.com/instaloader/instaloader/pulls>`__.
+
+- Please base your Pull Request on
+
+  - ``master``, which will be released with the next minor release, if it is
+
+    - a bug fix that does not require extensive testing,
+    - an improvement to the documentation,
+
+  - ``upcoming/v4.X``, if it is
+
+    - a new feature,
+    - a bug fix that does require thorough testing before being released to a
+      final version.
+
+- All Pull Requests are analyzed by `Pylint <https://www.pylint.org/>`__ for
+  error and syntax checking of the source and
+  `Mypy <https://github.com/python/mypy>`__ for type checking. You can run them
+  locally with::
+
+     pylint instaloader
+     mypy -m instaloader
+
+- Improvements to the documentation are very welcome. The documentation is
+  created with `Sphinx <https://www.sphinx-doc.org/en/2.0/>`__, version 2,
+  and can be build locally using::
+
+     make -C docs html
+
+- Feel free to create an issue to make sure someone from the Instaloader team
+  agrees that the change might be an improvement, or if you want to discuss
+  your basic proposal, before working on a pull request.
+
+Proposing Features
+------------------
+
+.. goal-start
+
+Instaloader's goal is to mimic the browser's behavior to access the data that
+is available through the Instagram web interface, pack this data into complete
+and easily-(re)usable python objects, and provide a user interface for the most
+common downloading and metadata collection tasks, without changing any of the
+online data.
+
+.. goal-end
+
+Prior spending effort on implementing a new feature, it might be appropriate to
+clarify how it could fit into the project's scope or discuss implementation
+details. If you feel the need to do so, please create a "feature suggestion".
+
+- Instaloader already has plenty of features. **Check the documentation**
+  beforehand to ensure your desired suggestion is not already implemented.
+
+- Briefly **ensure that your idea has not already been suggested**. If you find
+  an issue suggesting the same or a similar feature, share your thoughts in a
+  comment there, instead of opening a new issue.
+
+- **Motivate the feature**, i.e.
+
+  - Provide us a **use case of the feature**: How could the user
+    invoke the new function? Which problem would it solve? If new information is
+    obtained, how would it be further processed?
+
+  - Describe already-working **alternatives of the feature** and how they
+    compare to your proposed feature.
+
+  - Briefly describe how your suggested feature **conforms with Instaloader's
+    project goal**.
+
+- Explain your **solution ideas**. Describe your ideas on how the feature could
+  be implemented and the underlying problem could be solved. Also **describe
+  alternatives** that you have considered.
+
+Sponsoring
+----------
+
+.. include:: sponsors.rst
+   :start-after: donations-start
+   :end-before: donations-end

docs/index.rst

@@ -0,0 +1,106 @@
+.. meta::
+   :description:
+      Free command line tool to download photos from Instagram.
+      Scrapes public and private profiles, hashtags, stories, feeds,
+      saved media, and their metadata, comments and captions.
+      Written in Python.
+
+.. title:: Instaloader — Download Instagram Photos and Metadata
+
+Instaloader
+===========
+
+.. highlight:: none
+
+**Instaloader** is a tool to download pictures (or videos) along with
+their captions and other metadata from Instagram.
+
+.. include:: ../README.rst
+   :start-after: badges-start
+   :end-before: badges-end
+
+With `Python <https://www.python.org/>`__ installed, do::
+
+    $ pip3 install instaloader
+
+    $ instaloader profile [profile ...]
+
+See :ref:`install` for more options on how to install Instaloader.
+
+**Instaloader**
+
+- downloads **public and private profiles, hashtags, user stories,
+  feeds and saved media**,
+
+- downloads **comments, geotags and captions** of each post,
+
+- automatically **detects profile name changes** and renames the target
+  directory accordingly,
+
+- allows **fine-grained customization** of filters and where to store
+  downloaded media,
+
+- automatically **resumes previously-interrupted** download iterations,
+
+- is free `open source <https://github.com/instaloader/instaloader>`__
+  software written in Python.
+
+::
+
+    instaloader [--comments] [--geotags]
+                [--stories] [--highlights] [--tagged] [--igtv]
+                [--login YOUR-USERNAME] [--fast-update]
+                profile | "#hashtag" | %location_id |
+                :stories | :feed | :saved
+
+See :ref:`download-pictures-from-instagram` for a detailed introduction on how
+to use Instaloader to download pictures from Instagram.
+
+Instaloader Documentation
+-------------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   installation
+   basic-usage
+   cli-options
+   as-module
+   codesnippets
+   troubleshooting
+   contributing
+
+Useful Links
+------------
+
+- `Git Repository (on GitHub) <https://github.com/instaloader/instaloader>`__
+- `PyPI Project Page <https://pypi.org/project/instaloader/>`__
+- `Issue Tracker / Bug Tracker <https://github.com/instaloader/instaloader/issues>`__
+- `Version History <https://github.com/instaloader/instaloader/releases>`__
+
+Contributing
+------------
+
+As an open source project, Instaloader heavily depends on the contributions from
+its community. See :ref:`contributing` for how you may help Instaloader to
+become an even greater tool.
+
+Supporters
+----------
+
+.. include:: sponsors.rst
+   :start-after: donations-start
+   :end-before: donations-end
+
+Disclaimer
+----------
+
+.. include:: ../README.rst
+   :start-after: disclaimer-start
+   :end-before: disclaimer-end
+
+..
+   * :ref:`genindex`
+   * :ref:`modindex`
+   * :ref:`search`
+

docs/installation.rst

@@ -0,0 +1,41 @@
+.. _install:
+
+Install Instaloader
+===================
+
+.. highlight:: none
+
+To **install Instaloader**,
+
+#. Ensure that you have `Python <https://www.python.org/>`__, at least
+   version 3.8, and `pip <https://pypi.python.org/pypi/pip>`__
+   installed.
+
+#. Then, install Instaloader using::
+
+      pip3 install instaloader
+
+To **upgrade Instaloader** to its current version, do::
+
+   pip3 install --upgrade instaloader
+
+
+**Alternative methods** for installing Instaloader:
+
+- If you do not want to use pip, even though it is highly recommended,
+  and prefer to **install Instaloader manually**,
+  `Download the Source <https://github.com/instaloader/instaloader/releases/latest>`__,
+  extract the Zip or Tarball and run ``setup.py`` from there.
+
+- On **Arch Linux**, you may install Instaloader using the
+  `Instaloader AUR package <https://aur.archlinux.org/packages/instaloader/>`__.
+
+- On **Windows 10**, you may download the standalone executable from the
+  `current release page <https://github.com/instaloader/instaloader/releases/latest>`__.
+
+- On **Android**, you can use Instaloader with `Termux <https://play.google.com/store/apps/details?id=com.termux>`__
+  after typing ``pkg install python`` and ``pip3 install instaloader``.
+
+- To test the most current **pre-release** version of Instaloader::
+
+     pip3 install --pre instaloader

docs/module/exceptions.rst

@@ -0,0 +1,51 @@
+Exceptions
+^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. currentmodule:: instaloader.exceptions
+
+.. autoexception:: InstaloaderException
+   :no-show-inheritance:
+
+.. autoexception:: ConnectionException
+
+.. currentmodule:: instaloader
+
+.. autoexception:: QueryReturnedBadRequestException
+
+.. autoexception:: QueryReturnedForbiddenException
+
+.. autoexception:: ProfileNotExistsException
+
+.. autoexception:: ProfileHasNoPicsException
+
+.. autoexception:: PrivateProfileNotFollowedException
+
+.. autoexception:: LoginRequiredException
+
+.. autoexception:: TwoFactorAuthRequiredException
+
+   .. versionadded:: 4.2
+
+.. autoexception:: InvalidArgumentException
+
+.. autoexception:: BadResponseException
+
+.. autoexception:: BadCredentialsException
+
+.. autoexception:: PostChangedException
+
+.. autoexception:: QueryReturnedNotFoundException
+
+   .. versionchanged:: 4.3
+      QueryReturnedNotFoundException now inherits ConnectionException
+      to retry on 404 errors.
+
+
+.. autoexception:: TooManyRequestsException
+
+.. autoexception:: AbortDownloadException

docs/module/instaloader.rst

@@ -0,0 +1,10 @@
+``Instaloader`` (Main Class)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. autoclass:: Instaloader
+   :no-show-inheritance:

docs/module/instaloadercontext.rst

@@ -0,0 +1,24 @@
+``InstaloaderContext`` (Low-level functions)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+``InstaloaderContext``
+""""""""""""""""""""""
+
+.. autoclass:: InstaloaderContext
+   :no-show-inheritance:
+
+``RateController``
+""""""""""""""""""
+
+.. autoclass:: RateController
+   :no-show-inheritance:
+
+   .. versionadded:: 4.5

docs/module/nodeiterator.rst

@@ -0,0 +1,38 @@
+Resumable Iterations
+^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+For many download targets, Instaloader is able to resume a
+previously-interrupted iteration. It provides an interruptible
+Iterator :class:`NodeIterator` and a context manager
+:func:`resumable_iteration`, which we both present here.
+
+.. versionadded:: 4.5
+
+``NodeIterator``
+""""""""""""""""
+
+.. autoclass:: NodeIterator
+   :no-show-inheritance:
+
+.. autoclass:: FrozenNodeIterator
+   :no-show-inheritance:
+
+   A serializable representation of a :class:`NodeIterator` instance, saving
+   its iteration state.
+
+   It can be serialized and deserialized with :func:`save_structure_to_file`
+   and :func:`load_structure_from_file`, as well as with :mod:`json` and
+   :mod:`pickle` thanks to being a :class:`~typing.NamedTuple`.
+
+``resumable_iteration``
+"""""""""""""""""""""""
+
+.. autofunction:: resumable_iteration

docs/module/structures.rst

@@ -0,0 +1,92 @@
+Instagram Structures
+^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+Posts
+"""""
+
+.. autoclass:: Post
+   :no-show-inheritance:
+
+Additionally, the following trivial structures are defined:
+
+.. autoclass:: PostSidecarNode
+   :no-show-inheritance:
+
+.. autoclass:: PostComment
+   :no-show-inheritance:
+
+.. autoclass:: PostCommentAnswer
+   :no-show-inheritance:
+
+.. autoclass:: PostLocation
+   :no-show-inheritance:
+
+User Stories
+""""""""""""
+
+.. autoclass:: Story
+   :no-show-inheritance:
+
+.. autoclass:: StoryItem
+   :no-show-inheritance:
+
+Highlights
+""""""""""
+
+.. autoclass:: Highlight
+   :no-show-inheritance:
+   :inherited-members:
+
+   Bases: :class:`Story`
+
+   .. versionadded:: 4.1
+
+Profiles
+""""""""
+
+.. autoclass:: Profile
+   :no-show-inheritance:
+
+Hashtags
+""""""""
+
+.. autoclass:: Hashtag
+   :no-show-inheritance:
+
+   .. versionadded:: 4.4
+
+TopSearchResults
+""""""""""""""""
+
+.. autoclass:: TopSearchResults
+   :no-show-inheritance:
+
+   .. versionadded:: 4.3
+
+Loading and Saving
+""""""""""""""""""
+
+:class:`Post`, :class:`StoryItem`, :class:`Profile`, :class:`Hashtag` and
+:class:`FrozenNodeIterator` can be saved and loaded to/from JSON files.
+
+.. autofunction:: load_structure
+
+.. autofunction:: load_structure_from_file
+
+.. autofunction:: get_json_structure
+
+.. autofunction:: save_structure_to_file
+
+LatestStamps
+""""""""""""
+
+.. autoclass:: LatestStamps
+   :no-show-inheritance:

docs/requirements.txt

@@ -0,0 +1,2 @@
+requests
+sphinx~=2.3

docs/sphinx_autodoc_typehints.py

@@ -0,0 +1,251 @@
+# sphinx_autodoc_typehints from Nov 20 2018 (version 1.5.2).
+#  https://github.com/agronholm/sphinx-autodoc-typehints
+# (slightly modified to fix class links, maybe related to agronholm/sphinx-autodoc-typehints#38)
+#
+# This is the MIT license: http://www.opensource.org/licenses/mit-license.php
+#
+# Copyright (c) Alex Grönholm
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of this
+# software and associated documentation files (the "Software"), to deal in the Software
+# without restriction, including without limitation the rights to use, copy, modify, merge,
+# publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
+# to whom the Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all copies or
+# substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
+# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
+# FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+# DEALINGS IN THE SOFTWARE.
+#
+
+
+import inspect
+from typing import get_type_hints, TypeVar, Any, AnyStr, Generic, Union
+
+from sphinx.util import logging
+from sphinx.util.inspect import Signature
+
+try:
+    from inspect import unwrap
+except ImportError:
+    def unwrap(func, *, stop=None):
+        """This is the inspect.unwrap() method copied from Python 3.5's standard library."""
+        if stop is None:
+            def _is_wrapper(f):
+                return hasattr(f, '__wrapped__')
+        else:
+            def _is_wrapper(f):
+                return hasattr(f, '__wrapped__') and not stop(f)
+        f = func  # remember the original func for error reporting
+        memo = {id(f)}  # Memoise by id to tolerate non-hashable objects
+        while _is_wrapper(func):
+            func = func.__wrapped__
+            id_func = id(func)
+            if id_func in memo:
+                raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
+            memo.add(id_func)
+        return func
+
+logger = logging.getLogger(__name__)
+
+
+def format_annotation(annotation):
+    if inspect.isclass(annotation) and annotation.__module__ == 'builtins':
+        if annotation.__qualname__ == 'NoneType':
+            return '``None``'
+        else:
+            return ':py:class:`{}`'.format(annotation.__qualname__)
+
+    annotation_cls = annotation if inspect.isclass(annotation) else type(annotation)
+    class_name = None
+    if annotation_cls.__module__ == 'typing':
+        params = None
+        prefix = ':py:class:'
+        module = 'typing'
+        extra = ''
+
+        if inspect.isclass(getattr(annotation, '__origin__', None)):
+            annotation_cls = annotation.__origin__
+            try:
+                if Generic in annotation_cls.mro():
+                    module = annotation_cls.__module__
+            except TypeError:
+                pass  # annotation_cls was either the "type" object or typing.Type
+
+        if annotation is Any:
+            return ':py:data:`~typing.Any`'
+        elif annotation is AnyStr:
+            return ':py:data:`~typing.AnyStr`'
+        elif isinstance(annotation, TypeVar):
+            return '\\%r' % annotation
+        elif (annotation is Union or getattr(annotation, '__origin__', None) is Union or
+              hasattr(annotation, '__union_params__')):
+            prefix = ':py:data:'
+            class_name = 'Union'
+            if hasattr(annotation, '__union_params__'):
+                params = annotation.__union_params__
+            elif hasattr(annotation, '__args__'):
+                params = annotation.__args__
+
+            if params and len(params) == 2 and (hasattr(params[1], '__qualname__') and
+                                                params[1].__qualname__ == 'NoneType'):
+                class_name = 'Optional'
+                params = (params[0],)
+        elif annotation_cls.__qualname__ == 'Tuple' and hasattr(annotation, '__tuple_params__'):
+            params = annotation.__tuple_params__
+            if annotation.__tuple_use_ellipsis__:
+                params += (Ellipsis,)
+        elif annotation_cls.__qualname__ == 'Callable':
+            prefix = ':py:data:'
+            arg_annotations = result_annotation = None
+            if hasattr(annotation, '__result__'):
+                arg_annotations = annotation.__args__
+                result_annotation = annotation.__result__
+            elif getattr(annotation, '__args__', None):
+                arg_annotations = annotation.__args__[:-1]
+                result_annotation = annotation.__args__[-1]
+
+            if arg_annotations in (Ellipsis, (Ellipsis,)):
+                params = [Ellipsis, result_annotation]
+            elif arg_annotations is not None:
+                params = [
+                    '\\[{}]'.format(
+                        ', '.join(format_annotation(param) for param in arg_annotations)),
+                    result_annotation
+                ]
+        elif hasattr(annotation, 'type_var'):
+            # Type alias
+            class_name = annotation.name
+            params = (annotation.type_var,)
+        elif getattr(annotation, '__args__', None) is not None:
+            params = annotation.__args__
+        elif hasattr(annotation, '__parameters__'):
+            params = annotation.__parameters__
+
+        if params:
+            extra = '\\[{}]'.format(', '.join(format_annotation(param) for param in params))
+
+        if not class_name:
+            class_name = annotation_cls.__qualname__.title()
+
+        return '{}`~{}.{}`{}'.format(prefix, module, class_name, extra)
+    elif annotation is Ellipsis:
+        return '...'
+    elif inspect.isclass(annotation) or inspect.isclass(getattr(annotation, '__origin__', None)):
+        if not inspect.isclass(annotation):
+            annotation_cls = annotation.__origin__
+
+        extra = ''
+        if Generic in annotation_cls.mro():
+            params = (getattr(annotation, '__parameters__', None) or
+                      getattr(annotation, '__args__', None))
+            extra = '\\[{}]'.format(', '.join(format_annotation(param) for param in params))
+
+        module = annotation.__module__.split('.')[0]    # hack to 'fix' class linking for Instaloader project
+        return ':py:class:`~{}.{}`{}'.format(module, annotation_cls.__qualname__,
+                                             extra)
+
+    return str(annotation)
+
+
+def process_signature(app, what: str, name: str, obj, options, signature, return_annotation):
+    if not callable(obj):
+        return
+
+    if what in ('class', 'exception'):
+        obj = getattr(obj, '__init__', getattr(obj, '__new__', None))
+
+    if not getattr(obj, '__annotations__', None):
+        return
+
+    obj = unwrap(obj)
+    signature = Signature(obj)
+    parameters = [
+        param.replace(annotation=inspect.Parameter.empty)
+        for param in signature.signature.parameters.values()
+    ]
+
+    if parameters:
+        if what in ('class', 'exception'):
+            del parameters[0]
+        elif what == 'method':
+            outer = inspect.getmodule(obj)
+            for clsname in obj.__qualname__.split('.')[:-1]:
+                outer = getattr(outer, clsname)
+
+            method_name = obj.__name__
+            if method_name.startswith("__") and not method_name.endswith("__"):
+                # If the method starts with double underscore (dunder)
+                # Python applies mangling so we need to prepend the class name.
+                # This doesn't happen if it always ends with double underscore.
+                class_name = obj.__qualname__.split('.')[-2]
+                method_name = "_{c}{m}".format(c=class_name, m=method_name)
+
+            method_object = outer.__dict__[method_name]
+            if not isinstance(method_object, (classmethod, staticmethod)):
+                del parameters[0]
+
+    signature.signature = signature.signature.replace(
+        parameters=parameters,
+        return_annotation=inspect.Signature.empty)
+
+    return signature.format_args().replace('\\', '\\\\'), None
+
+
+def process_docstring(app, what, name, obj, options, lines):
+    if isinstance(obj, property):
+        obj = obj.fget
+
+    if callable(obj):
+        if what in ('class', 'exception'):
+            obj = getattr(obj, '__init__')
+
+        obj = unwrap(obj)
+        try:
+            type_hints = get_type_hints(obj)
+        except (AttributeError, TypeError):
+            # Introspecting a slot wrapper will raise TypeError
+            return
+        except NameError as exc:
+            logger.warning('Cannot resolve forward reference in type annotations of "%s": %s',
+                           name, exc)
+            type_hints = obj.__annotations__
+
+        for argname, annotation in type_hints.items():
+            if argname.endswith('_'):
+                argname = '{}\\_'.format(argname[:-1])
+
+            formatted_annotation = format_annotation(annotation)
+
+            if argname == 'return':
+                if what in ('class', 'exception'):
+                    # Don't add return type None from __init__()
+                    continue
+
+                insert_index = len(lines)
+                for i, line in enumerate(lines):
+                    if line.startswith(':rtype:'):
+                        insert_index = None
+                        break
+                    elif line.startswith(':return:') or line.startswith(':returns:'):
+                        insert_index = i
+
+                if insert_index is not None:
+                    if insert_index == len(lines):
+                        # Ensure that :rtype: doesn't get joined with a paragraph of text, which
+                        # prevents it being interpreted.
+                        lines.append('')
+                        insert_index += 1
+
+                    lines.insert(insert_index, ':rtype: {}'.format(formatted_annotation))
+            else:
+                searchfor = ':param {}:'.format(argname)
+                for i, line in enumerate(lines):
+                    if line.startswith(searchfor):
+                        lines.insert(i, ':type {}: {}'.format(argname, formatted_annotation))
+                        break

docs/sponsors.rst

@@ -0,0 +1,23 @@
+.. _sponsors:
+
+Sponsoring the development of Instaloader
+=========================================
+
+.. donations-start
+
+.. include:: ../README.rst
+   :start-after: current-sponsors-start
+   :end-before: current-sponsors-end
+
+It is a pleasure for us to share our Instaloader to the world, and we are proud
+to have attracted such an active and motivating community, with so many users
+who share their suggestions and ideas with us. Buying a community-sponsored beer
+or coffee from time to time is very likely to further raise our passion for the
+development of Instaloader.
+
+| For Donations, we provide GitHub Sponsors page, a PayPal.Me link and a Bitcoin address.
+|  GitHub Sponsors: `Sponsor @aandergr on GitHub Sponsors <https://github.com/sponsors/aandergr>`__
+|  PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
+|  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
+
+.. donations-end

docs/troubleshooting.rst

@@ -0,0 +1,92 @@
+.. _troubleshooting:
+
+Troubleshooting
+===============
+
+.. highlight:: python
+
+429 Too Many Requests
+---------------------
+
+Instaloader has a logic to keep track of its requests to Instagram and to obey
+their rate limits. The rate controller assumes that
+
+- at one time, Instaloader is the only application that consumes requests, i.e.
+  neither the Instagram browser interface, nor a mobile app, nor another
+  Instaloader instance is running in parallel, and
+
+- no requests had been consumed when Instaloader starts.
+
+The latter one implies that restarting or reinstantiating Instaloader often
+within short time is prone to cause a 429.
+
+Since the behavior of the rate controller might change between different
+versions of Instaloader, make sure to use the current version of Instaloader,
+especially when encountering many 429 errors.
+
+If a request is denied with a 429,
+Instaloader retries the request as soon as the temporary ban is assumed to be
+expired. In case the retry continuously fails for some reason, which should not
+happen under normal conditions, consider adjusting the
+:option:`--max-connection-attempts` option.
+
+There have been observations that services, that in their nature offer
+promiscuous IP addresses, such as cloud, VPN and public proxy services, might be
+subject to significantly stricter limits for anonymous access. However,
+logged-in accesses (see :option:`--login`) do not seem to be affected.
+
+Instaloader allows to adjust the rate controlling behavior by overriding
+:class:`instaloader.RateController`.
+
+Too many queries in the last time
+---------------------------------
+
+**"Too many queries in the last time"** is not an error. It is a notice that the
+rate limit has almost been reached, according to Instaloader's own rate
+accounting mechanism.
+
+Instaloader allows to adjust the rate controlling behavior by overriding
+:class:`instaloader.RateController`.
+
+Private but not followed
+------------------------
+
+You have to follow a private account to access most of its associated
+information.
+
+Login error
+-----------
+
+Instaloader's login *should* work fine, both with and without
+Two-Factor-Authentication. It also supports handling the *checkpoint challenge*,
+issued when Instagram suspects authentication activity on your account, by
+pointing the user to an URL to be opened in a browser.
+
+Nevertheless, in :issue:`92`, :issue:`615`, :issue:`1150` and :issue:`1217`,
+users reported problems with
+logging in. We recommend to always keep the session file which Instaloader
+creates when using :option:`--login`. If a session file is present,
+:option:`--login` does not make use of the failure-prone login procedure.
+Also, session files usually do not expire.
+
+If you do not have a session file present, you may use the following script
+(:example:`615_import_firefox_session.py`) to workaround login problems by
+importing the session cookies from Firefox and bypassing Instaloader's login and
+so still use Instaloader's logged-in functionality.
+
+.. literalinclude:: codesnippets/615_import_firefox_session.py
+
+To use this script,
+
+#. Download the script: :example:`615_import_firefox_session.py`,
+
+#. Login to Instagram in Firefox,
+
+#. Execute the snippet, e.g. with ``python 615_import_firefox_session.py``,
+
+#. Then, ``instaloader -l USERNAME`` should work fine.
+
+This script also supports specifying a cookie file path, which may be useful if
+you use multiple Firefox profiles or if your operating system has the directory
+structure differently set up. Also, you can specify an alternative session file
+path.

instaloader/__init__.py

@@ -0,0 +1,22 @@
+"""Download pictures (or videos) along with their captions and other metadata from Instagram."""
+
+
+__version__ = '4.10.1'
+
+
+try:
+    # pylint:disable=wrong-import-position
+    import win_unicode_console  # type: ignore
+except ImportError:
+    pass
+else:
+    win_unicode_console.enable()
+
+from .exceptions import *
+from .instaloader import Instaloader
+from .instaloadercontext import InstaloaderContext, RateController
+from .lateststamps import LatestStamps
+from .nodeiterator import NodeIterator, FrozenNodeIterator, resumable_iteration
+from .structures import (Hashtag, Highlight, Post, PostSidecarNode, PostComment, PostCommentAnswer, PostLocation,
+                         Profile, Story, StoryItem, TopSearchResults, load_structure_from_file, save_structure_to_file,
+                         load_structure, get_json_structure)

instaloader/__main__.py

@@ -0,0 +1,492 @@
+"""Download pictures (or videos) along with their captions and other metadata from Instagram."""
+
+import ast
+import datetime
+import os
+import re
+import sys
+from argparse import ArgumentParser, ArgumentTypeError, SUPPRESS
+from typing import List, Optional
+
+from . import (AbortDownloadException, BadCredentialsException, Instaloader, InstaloaderException,
+               InvalidArgumentException, Post, Profile, ProfileNotExistsException, StoryItem,
+               TwoFactorAuthRequiredException, __version__, load_structure_from_file)
+from .instaloader import (get_default_session_filename, get_default_stamps_filename)
+from .instaloadercontext import default_user_agent
+from .lateststamps import LatestStamps
+
+
+def usage_string():
+    # NOTE: duplicated in README.rst and docs/index.rst
+    argv0 = os.path.basename(sys.argv[0])
+    argv0 = "instaloader" if argv0 == "__main__.py" else argv0
+    return """
+{0} [--comments] [--geotags]
+{2:{1}} [--stories] [--highlights] [--tagged] [--igtv]
+{2:{1}} [--login YOUR-USERNAME] [--fast-update]
+{2:{1}} profile | "#hashtag" | %%location_id | :stories | :feed | :saved
+{0} --help""".format(argv0, len(argv0), '')
+
+
+def http_status_code_list(code_list_str: str) -> List[int]:
+    codes = [int(s) for s in code_list_str.split(',')]
+    for code in codes:
+        if not 100 <= code <= 599:
+            raise ArgumentTypeError("Invalid HTTP status code: {}".format(code))
+    return codes
+
+
+def filterstr_to_filterfunc(filter_str: str, item_type: type):
+    """Takes an --post-filter=... or --storyitem-filter=... filter
+     specification and makes a filter_func Callable out of it."""
+
+    # The filter_str is parsed, then all names occurring in its AST are replaced by loads to post.<name>. A
+    # function Post->bool is returned which evaluates the filter with the post as 'post' in its namespace.
+
+    class TransformFilterAst(ast.NodeTransformer):
+        def visit_Name(self, node: ast.Name):
+            if not isinstance(node.ctx, ast.Load):
+                raise InvalidArgumentException("Invalid filter: Modifying variables ({}) not allowed.".format(node.id))
+            if node.id == "datetime":
+                return node
+            if not hasattr(item_type, node.id):
+                raise InvalidArgumentException("Invalid filter: {} not a {} attribute.".format(node.id,
+                                                                                               item_type.__name__))
+            new_node = ast.Attribute(ast.copy_location(ast.Name('item', ast.Load()), node), node.id,
+                                     ast.copy_location(ast.Load(), node))
+            return ast.copy_location(new_node, node)
+
+    input_filename = '<command line filter parameter>'
+    compiled_filter = compile(TransformFilterAst().visit(ast.parse(filter_str, filename=input_filename, mode='eval')),
+                              filename=input_filename, mode='eval')
+
+    def filterfunc(item) -> bool:
+        # pylint:disable=eval-used
+        return bool(eval(compiled_filter, {'item': item, 'datetime': datetime.datetime}))
+
+    return filterfunc
+
+
+def _main(instaloader: Instaloader, targetlist: List[str],
+          username: Optional[str] = None, password: Optional[str] = None,
+          sessionfile: Optional[str] = None,
+          download_profile_pic: bool = True, download_posts=True,
+          download_stories: bool = False,
+          download_highlights: bool = False,
+          download_tagged: bool = False,
+          download_igtv: bool = False,
+          fast_update: bool = False,
+          latest_stamps_file: Optional[str] = None,
+          max_count: Optional[int] = None, post_filter_str: Optional[str] = None,
+          storyitem_filter_str: Optional[str] = None) -> None:
+    """Download set of profiles, hashtags etc. and handle logging in and session files if desired."""
+    # Parse and generate filter function
+    post_filter = None
+    if post_filter_str is not None:
+        post_filter = filterstr_to_filterfunc(post_filter_str, Post)
+        instaloader.context.log('Only download posts with property "{}".'.format(post_filter_str))
+    storyitem_filter = None
+    if storyitem_filter_str is not None:
+        storyitem_filter = filterstr_to_filterfunc(storyitem_filter_str, StoryItem)
+        instaloader.context.log('Only download storyitems with property "{}".'.format(storyitem_filter_str))
+    latest_stamps = None
+    if latest_stamps_file is not None:
+        latest_stamps = LatestStamps(latest_stamps_file)
+        instaloader.context.log(f"Using latest stamps from {latest_stamps_file}.")
+    # Login, if desired
+    if username is not None:
+        if not re.match(r"^[A-Za-z0-9._]+$", username):
+            instaloader.context.error("Warning: Parameter \"{}\" for --login is not a valid username.".format(username))
+        try:
+            instaloader.load_session_from_file(username, sessionfile)
+        except FileNotFoundError as err:
+            if sessionfile is not None:
+                print(err, file=sys.stderr)
+            instaloader.context.log("Session file does not exist yet - Logging in.")
+        if not instaloader.context.is_logged_in or username != instaloader.test_login():
+            if password is not None:
+                try:
+                    instaloader.login(username, password)
+                except TwoFactorAuthRequiredException:
+                    while True:
+                        try:
+                            code = input("Enter 2FA verification code: ")
+                            instaloader.two_factor_login(code)
+                            break
+                        except BadCredentialsException as err:
+                            print(err, file=sys.stderr)
+                            pass
+            else:
+                instaloader.interactive_login(username)
+        instaloader.context.log("Logged in as %s." % username)
+    # since 4.2.9 login is required for geotags
+    if instaloader.download_geotags and not instaloader.context.is_logged_in:
+        instaloader.context.error("Warning: Use --login to download geotags of posts.")
+    # Try block for KeyboardInterrupt (save session on ^C)
+    profiles = set()
+    anonymous_retry_profiles = set()
+    try:
+        # Generate set of profiles, already downloading non-profile targets
+        for target in targetlist:
+            if (target.endswith('.json') or target.endswith('.json.xz')) and os.path.isfile(target):
+                with instaloader.context.error_catcher(target):
+                    structure = load_structure_from_file(instaloader.context, target)
+                    if isinstance(structure, Post):
+                        if post_filter is not None and not post_filter(structure):
+                            instaloader.context.log("<{} ({}) skipped>".format(structure, target), flush=True)
+                            continue
+                        instaloader.context.log("Downloading {} ({})".format(structure, target))
+                        instaloader.download_post(structure, os.path.dirname(target))
+                    elif isinstance(structure, StoryItem):
+                        if storyitem_filter is not None and not storyitem_filter(structure):
+                            instaloader.context.log("<{} ({}) skipped>".format(structure, target), flush=True)
+                            continue
+                        instaloader.context.log("Attempting to download {} ({})".format(structure, target))
+                        instaloader.download_storyitem(structure, os.path.dirname(target))
+                    elif isinstance(structure, Profile):
+                        raise InvalidArgumentException("Profile JSON are ignored. Pass \"{}\" to download that profile"
+                                                       .format(structure.username))
+                    else:
+                        raise InvalidArgumentException("{} JSON file not supported as target"
+                                                       .format(structure.__class__.__name__))
+                continue
+            # strip '/' characters to be more shell-autocompletion-friendly
+            target = target.rstrip('/')
+            with instaloader.context.error_catcher(target):
+                if re.match(r"^@[A-Za-z0-9._]+$", target):
+                    instaloader.context.log("Retrieving followees of %s..." % target[1:])
+                    profile = Profile.from_username(instaloader.context, target[1:])
+                    for followee in profile.get_followees():
+                        instaloader.save_profile_id(followee)
+                        profiles.add(followee)
+                elif re.match(r"^#\w+$", target):
+                    instaloader.download_hashtag(hashtag=target[1:], max_count=max_count, fast_update=fast_update,
+                                                 post_filter=post_filter,
+                                                 profile_pic=download_profile_pic, posts=download_posts)
+                elif re.match(r"^-[A-Za-z0-9-_]+$", target):
+                    instaloader.download_post(Post.from_shortcode(instaloader.context, target[1:]), target)
+                elif re.match(r"^%[0-9]+$", target):
+                    instaloader.download_location(location=target[1:], max_count=max_count, fast_update=fast_update,
+                                                  post_filter=post_filter)
+                elif target == ":feed":
+                    instaloader.download_feed_posts(fast_update=fast_update, max_count=max_count,
+                                                    post_filter=post_filter)
+                elif target == ":stories":
+                    instaloader.download_stories(fast_update=fast_update, storyitem_filter=storyitem_filter)
+                elif target == ":saved":
+                    instaloader.download_saved_posts(fast_update=fast_update, max_count=max_count,
+                                                     post_filter=post_filter)
+                elif re.match(r"^[A-Za-z0-9._]+$", target):
+                    try:
+                        profile = instaloader.check_profile_id(target, latest_stamps)
+                        if instaloader.context.is_logged_in and profile.has_blocked_viewer:
+                            if download_profile_pic or ((download_posts or download_tagged or download_igtv)
+                                                        and not profile.is_private):
+                                raise ProfileNotExistsException("{} blocked you; But we download her anonymously."
+                                                                .format(target))
+                            else:
+                                instaloader.context.error("{} blocked you.".format(target))
+                        else:
+                            profiles.add(profile)
+                    except ProfileNotExistsException as err:
+                        # Not only our profile.has_blocked_viewer condition raises ProfileNotExistsException,
+                        # check_profile_id() also does, since access to blocked profile may be responded with 404.
+                        if instaloader.context.is_logged_in and (download_profile_pic or download_posts or
+                                                                 download_tagged or download_igtv):
+                            instaloader.context.log(err)
+                            instaloader.context.log("Trying again anonymously, helps in case you are just blocked.")
+                            with instaloader.anonymous_copy() as anonymous_loader:
+                                with instaloader.context.error_catcher():
+                                    anonymous_retry_profiles.add(anonymous_loader.check_profile_id(target,
+                                                                                                   latest_stamps))
+                                    instaloader.context.error("Warning: {} will be downloaded anonymously (\"{}\")."
+                                                              .format(target, err))
+                        else:
+                            raise
+                else:
+                    target_type = {
+                        '#': 'hashtag',
+                        '%': 'location',
+                        '-': 'shortcode',
+                    }.get(target[0], 'username')
+                    raise ProfileNotExistsException('Invalid {} {}'.format(target_type, target))
+        if len(profiles) > 1:
+            instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
+                                                                         ' '.join([p.username for p in profiles])))
+        if instaloader.context.iphone_support and profiles and (download_profile_pic or download_posts) and \
+           not instaloader.context.is_logged_in:
+            instaloader.context.log("Hint: Use --login to download higher-quality versions of pictures.")
+        instaloader.download_profiles(profiles,
+                                      download_profile_pic, download_posts, download_tagged, download_igtv,
+                                      download_highlights, download_stories,
+                                      fast_update, post_filter, storyitem_filter, latest_stamps=latest_stamps)
+        if anonymous_retry_profiles:
+            instaloader.context.log("Downloading anonymously: {}"
+                                    .format(' '.join([p.username for p in anonymous_retry_profiles])))
+            with instaloader.anonymous_copy() as anonymous_loader:
+                anonymous_loader.download_profiles(anonymous_retry_profiles,
+                                                   download_profile_pic, download_posts, download_tagged, download_igtv,
+                                                   fast_update=fast_update, post_filter=post_filter,
+                                                   latest_stamps=latest_stamps)
+    except KeyboardInterrupt:
+        print("\nInterrupted by user.", file=sys.stderr)
+    except AbortDownloadException as exc:
+        print("\nDownload aborted: {}.".format(exc), file=sys.stderr)
+    # Save session if it is useful
+    if instaloader.context.is_logged_in:
+        instaloader.save_session_to_file(sessionfile)
+    # User might be confused if Instaloader does nothing
+    if not targetlist:
+        if instaloader.context.is_logged_in:
+            # Instaloader did at least save a session file
+            instaloader.context.log("No targets were specified, thus nothing has been downloaded.")
+        else:
+            # Instaloader did not do anything
+            instaloader.context.log("usage:" + usage_string())
+
+
+def main():
+    parser = ArgumentParser(description=__doc__, add_help=False, usage=usage_string(),
+                            epilog="The complete documentation can be found at "
+                                   "https://instaloader.github.io/.",
+                            fromfile_prefix_chars='+')
+
+    g_targets = parser.add_argument_group("What to Download",
+                                          "Specify a list of targets. For each of these, Instaloader creates a folder "
+                                          "and downloads all posts. The following targets are supported:")
+    g_targets.add_argument('profile', nargs='*',
+                           help="Download profile. If an already-downloaded profile has been renamed, Instaloader "
+                                "automatically finds it by its unique ID and renames the folder likewise.")
+    g_targets.add_argument('_at_profile', nargs='*', metavar="@profile",
+                           help="Download all followees of profile. Requires --login. "
+                                "Consider using :feed rather than @yourself.")
+    g_targets.add_argument('_hashtag', nargs='*', metavar='"#hashtag"', help="Download #hashtag.")
+    g_targets.add_argument('_location', nargs='*', metavar='%location_id',
+                           help="Download %%location_id. Requires --login.")
+    g_targets.add_argument('_feed', nargs='*', metavar=":feed",
+                           help="Download pictures from your feed. Requires --login.")
+    g_targets.add_argument('_stories', nargs='*', metavar=":stories",
+                           help="Download the stories of your followees. Requires --login.")
+    g_targets.add_argument('_saved', nargs='*', metavar=":saved",
+                           help="Download the posts that you marked as saved. Requires --login.")
+    g_targets.add_argument('_singlepost', nargs='*', metavar="-- -shortcode",
+                           help="Download the post with the given shortcode")
+    g_targets.add_argument('_json', nargs='*', metavar="filename.json[.xz]",
+                           help="Re-Download the given object.")
+    g_targets.add_argument('_fromfile', nargs='*', metavar="+args.txt",
+                           help="Read targets (and options) from given textfile.")
+
+    g_post = parser.add_argument_group("What to Download of each Post")
+
+    g_prof = parser.add_argument_group("What to Download of each Profile")
+
+    g_prof.add_argument('-P', '--profile-pic-only', action='store_true',
+                        help=SUPPRESS)
+    g_prof.add_argument('--no-posts', action='store_true',
+                        help="Do not download regular posts.")
+    g_prof.add_argument('--no-profile-pic', action='store_true',
+                        help='Do not download profile picture.')
+    g_post.add_argument('--slide', action='store',
+                        help='Set what image/interval of a sidecar you want to download.')
+    g_post.add_argument('--no-pictures', action='store_true',
+                        help='Do not download post pictures. Cannot be used together with --fast-update. '
+                             'Implies --no-video-thumbnails, does not imply --no-videos.')
+    g_post.add_argument('-V', '--no-videos', action='store_true',
+                        help='Do not download videos.')
+    g_post.add_argument('--no-video-thumbnails', action='store_true',
+                        help='Do not download thumbnails of videos.')
+    g_post.add_argument('-G', '--geotags', action='store_true',
+                        help='Download geotags when available. Geotags are stored as a '
+                             'text file with the location\'s name and a Google Maps link. '
+                             'This requires an additional request to the Instagram '
+                             'server for each picture. Requires --login.')
+    g_post.add_argument('-C', '--comments', action='store_true',
+                        help='Download and update comments for each post. '
+                             'This requires an additional request to the Instagram '
+                             'server for each post, which is why it is disabled by default.')
+    g_post.add_argument('--no-captions', action='store_true',
+                        help='Do not create txt files.')
+    g_post.add_argument('--post-metadata-txt', action='append',
+                        help='Template to write in txt file for each Post.')
+    g_post.add_argument('--storyitem-metadata-txt', action='append',
+                        help='Template to write in txt file for each StoryItem.')
+    g_post.add_argument('--no-metadata-json', action='store_true',
+                        help='Do not create a JSON file containing the metadata of each post.')
+    g_post.add_argument('--metadata-json', action='store_true',
+                        help=SUPPRESS)
+    g_post.add_argument('--no-compress-json', action='store_true',
+                        help='Do not xz compress JSON files, rather create pretty formatted JSONs.')
+    g_prof.add_argument('-s', '--stories', action='store_true',
+                        help='Also download stories of each profile that is downloaded. Requires --login.')
+    g_prof.add_argument('--stories-only', action='store_true',
+                        help=SUPPRESS)
+    g_prof.add_argument('--highlights', action='store_true',
+                        help='Also download highlights of each profile that is downloaded. Requires --login.')
+    g_prof.add_argument('--tagged', action='store_true',
+                        help='Also download posts where each profile is tagged.')
+    g_prof.add_argument('--igtv', action='store_true',
+                        help='Also download IGTV videos.')
+
+    g_cond = parser.add_argument_group("Which Posts to Download")
+
+    g_cond.add_argument('-F', '--fast-update', action='store_true',
+                        help='For each target, stop when encountering the first already-downloaded picture. This '
+                             'flag is recommended when you use Instaloader to update your personal Instagram archive.')
+    g_cond.add_argument('--latest-stamps', nargs='?', metavar='STAMPSFILE', const=get_default_stamps_filename(),
+                        help='Store the timestamps of latest media scraped for each profile. This allows updating '
+                             'your personal Instagram archive even if you delete the destination directories. '
+                             'If STAMPSFILE is not provided, defaults to ' + get_default_stamps_filename())
+    g_cond.add_argument('--post-filter', '--only-if', metavar='filter',
+                        help='Expression that, if given, must evaluate to True for each post to be downloaded. Must be '
+                             'a syntactically valid python expression. Variables are evaluated to '
+                             'instaloader.Post attributes. Example: --post-filter=viewer_has_liked.')
+    g_cond.add_argument('--storyitem-filter', metavar='filter',
+                        help='Expression that, if given, must evaluate to True for each storyitem to be downloaded. '
+                             'Must be a syntactically valid python expression. Variables are evaluated to '
+                             'instaloader.StoryItem attributes.')
+
+    g_cond.add_argument('-c', '--count',
+                        help='Do not attempt to download more than COUNT posts. '
+                             'Applies to #hashtag, %%location_id, :feed, and :saved.')
+
+    g_login = parser.add_argument_group('Login (Download Private Profiles)',
+                                        'Instaloader can login to Instagram. This allows downloading private profiles. '
+                                        'To login, pass the --login option. Your session cookie (not your password!) '
+                                        'will be saved to a local file to be reused next time you want Instaloader '
+                                        'to login.')
+    g_login.add_argument('-l', '--login', metavar='YOUR-USERNAME',
+                         help='Login name (profile name) for your Instagram account.')
+    g_login.add_argument('-f', '--sessionfile',
+                         help='Path for loading and storing session key file. '
+                              'Defaults to ' + get_default_session_filename("<login_name>"))
+    g_login.add_argument('-p', '--password', metavar='YOUR-PASSWORD',
+                         help='Password for your Instagram account. Without this option, '
+                              'you\'ll be prompted for your password interactively if '
+                              'there is not yet a valid session file.')
+
+    g_how = parser.add_argument_group('How to Download')
+    g_how.add_argument('--dirname-pattern',
+                       help='Name of directory where to store posts. {profile} is replaced by the profile name, '
+                            '{target} is replaced by the target you specified, i.e. either :feed, #hashtag or the '
+                            'profile name. Defaults to \'{target}\'.')
+    g_how.add_argument('--filename-pattern',
+                       help='Prefix of filenames for posts and stories, relative to the directory given with '
+                            '--dirname-pattern. {profile} is replaced by the profile name,'
+                            '{target} is replaced by the target you specified, i.e. either :feed'
+                            '#hashtag or the profile name. Defaults to \'{date_utc}_UTC\'')
+    g_how.add_argument('--title-pattern',
+                       help='Prefix of filenames for profile pics, hashtag profile pics, and highlight covers. '
+                            'Defaults to \'{date_utc}_UTC_{typename}\' if --dirname-pattern contains \'{target}\' '
+                            'or \'{dirname}\', or if --dirname-pattern is not specified. Otherwise defaults to '
+                            '\'{target}_{date_utc}_UTC_{typename}\'.')
+    g_how.add_argument('--resume-prefix', metavar='PREFIX',
+                       help='Prefix for filenames that are used to save the information to resume an interrupted '
+                            'download.')
+    g_how.add_argument('--sanitize-paths', action='store_true',
+                       help='Sanitize paths so that the resulting file and directory names are valid on both '
+                            'Windows and Unix.')
+    g_how.add_argument('--no-resume', action='store_true',
+                       help='Do not resume a previously-aborted download iteration, and do not save such information '
+                            'when interrupted.')
+    g_how.add_argument('--use-aged-resume-files', action='store_true', help=SUPPRESS)
+    g_how.add_argument('--user-agent',
+                       help='User Agent to use for HTTP requests. Defaults to \'{}\'.'.format(default_user_agent()))
+    g_how.add_argument('-S', '--no-sleep', action='store_true', help=SUPPRESS)
+    g_how.add_argument('--max-connection-attempts', metavar='N', type=int, default=3,
+                       help='Maximum number of connection attempts until a request is aborted. Defaults to 3. If a '
+                            'connection fails, it can be manually skipped by hitting CTRL+C. Set this to 0 to retry '
+                            'infinitely.')
+    g_how.add_argument('--commit-mode', action='store_true', help=SUPPRESS)
+    g_how.add_argument('--request-timeout', metavar='N', type=float, default=300.0,
+                       help='Seconds to wait before timing out a connection request. Defaults to 300.')
+    g_how.add_argument('--abort-on', type=http_status_code_list, metavar="STATUS_CODES",
+                       help='Comma-separated list of HTTP status codes that cause Instaloader to abort, bypassing all '
+                            'retry logic.')
+    g_how.add_argument('--no-iphone', action='store_true',
+                        help='Do not attempt to download iPhone version of images and videos.')
+
+    g_misc = parser.add_argument_group('Miscellaneous Options')
+    g_misc.add_argument('-q', '--quiet', action='store_true',
+                        help='Disable user interaction, i.e. do not print messages (except errors) and fail '
+                             'if login credentials are needed but not given. This makes Instaloader suitable as a '
+                             'cron job.')
+    g_misc.add_argument('-h', '--help', action='help', help='Show this help message and exit.')
+    g_misc.add_argument('--version', action='version', help='Show version number and exit.',
+                        version=__version__)
+
+    args = parser.parse_args()
+    try:
+        if args.login is None and (args.stories or args.stories_only):
+            print("--login=USERNAME required to download stories.", file=sys.stderr)
+            args.stories = False
+            if args.stories_only:
+                raise SystemExit(1)
+
+        if ':feed-all' in args.profile or ':feed-liked' in args.profile:
+            raise SystemExit(":feed-all and :feed-liked were removed. Use :feed as target and "
+                             "eventually --post-filter=viewer_has_liked.")
+
+        post_metadata_txt_pattern = '\n'.join(args.post_metadata_txt) if args.post_metadata_txt else None
+        storyitem_metadata_txt_pattern = '\n'.join(args.storyitem_metadata_txt) if args.storyitem_metadata_txt else None
+
+        if args.no_captions:
+            if not (post_metadata_txt_pattern or storyitem_metadata_txt_pattern):
+                post_metadata_txt_pattern = ''
+                storyitem_metadata_txt_pattern = ''
+            else:
+                raise SystemExit("--no-captions and --post-metadata-txt or --storyitem-metadata-txt given; "
+                                 "That contradicts.")
+
+        if args.no_resume and args.resume_prefix:
+            raise SystemExit("--no-resume and --resume-prefix given; That contradicts.")
+        resume_prefix = (args.resume_prefix if args.resume_prefix else 'iterator') if not args.no_resume else None
+
+        if args.no_pictures and args.fast_update:
+            raise SystemExit('--no-pictures and --fast-update cannot be used together.')
+
+        # Determine what to download
+        download_profile_pic = not args.no_profile_pic or args.profile_pic_only
+        download_posts = not (args.no_posts or args.stories_only or args.profile_pic_only)
+        download_stories = args.stories or args.stories_only
+
+        loader = Instaloader(sleep=not args.no_sleep, quiet=args.quiet, user_agent=args.user_agent,
+                             dirname_pattern=args.dirname_pattern, filename_pattern=args.filename_pattern,
+                             download_pictures=not args.no_pictures,
+                             download_videos=not args.no_videos, download_video_thumbnails=not args.no_video_thumbnails,
+                             download_geotags=args.geotags,
+                             download_comments=args.comments, save_metadata=not args.no_metadata_json,
+                             compress_json=not args.no_compress_json,
+                             post_metadata_txt_pattern=post_metadata_txt_pattern,
+                             storyitem_metadata_txt_pattern=storyitem_metadata_txt_pattern,
+                             max_connection_attempts=args.max_connection_attempts,
+                             request_timeout=args.request_timeout,
+                             resume_prefix=resume_prefix,
+                             check_resume_bbd=not args.use_aged_resume_files,
+                             slide=args.slide,
+                             fatal_status_codes=args.abort_on,
+                             iphone_support=not args.no_iphone,
+                             title_pattern=args.title_pattern,
+                             sanitize_paths=args.sanitize_paths)
+        _main(loader,
+              args.profile,
+              username=args.login.lower() if args.login is not None else None,
+              password=args.password,
+              sessionfile=args.sessionfile,
+              download_profile_pic=download_profile_pic,
+              download_posts=download_posts,
+              download_stories=download_stories,
+              download_highlights=args.highlights,
+              download_tagged=args.tagged,
+              download_igtv=args.igtv,
+              fast_update=args.fast_update,
+              latest_stamps_file=args.latest_stamps,
+              max_count=int(args.count) if args.count is not None else None,
+              post_filter_str=args.post_filter,
+              storyitem_filter_str=args.storyitem_filter)
+        loader.close()
+    except InstaloaderException as err:
+        raise SystemExit("Fatal error: %s" % err) from err
+
+
+if __name__ == "__main__":
+    main()

instaloader/exceptions.py

@@ -0,0 +1,80 @@
+class InstaloaderException(Exception):
+    """Base exception for this script.
+
+    :note: This exception should not be raised directly."""
+    pass
+
+
+class QueryReturnedBadRequestException(InstaloaderException):
+    pass
+
+
+class QueryReturnedForbiddenException(InstaloaderException):
+    pass
+
+
+class ProfileNotExistsException(InstaloaderException):
+    pass
+
+
+class ProfileHasNoPicsException(InstaloaderException):
+    """
+    .. deprecated:: 4.2.2
+       Not raised anymore.
+    """
+    pass
+
+
+class PrivateProfileNotFollowedException(InstaloaderException):
+    pass
+
+
+class LoginRequiredException(InstaloaderException):
+    pass
+
+
+class TwoFactorAuthRequiredException(InstaloaderException):
+    pass
+
+
+class InvalidArgumentException(InstaloaderException):
+    pass
+
+
+class BadResponseException(InstaloaderException):
+    pass
+
+
+class BadCredentialsException(InstaloaderException):
+    pass
+
+
+class ConnectionException(InstaloaderException):
+    pass
+
+
+class PostChangedException(InstaloaderException):
+    """.. versionadded:: 4.2.2"""
+    pass
+
+
+class QueryReturnedNotFoundException(ConnectionException):
+    pass
+
+
+class TooManyRequestsException(ConnectionException):
+    pass
+
+class IPhoneSupportDisabledException(InstaloaderException):
+    pass
+
+class AbortDownloadException(Exception):
+    """
+    Exception that is not catched in the error catchers inside the download loop and so aborts the
+    download loop.
+
+    This exception is not a subclass of ``InstaloaderException``.
+
+    .. versionadded:: 4.7
+    """
+    pass

instaloader/instaloadercontext.py

@@ -0,0 +1,815 @@
+import hashlib
+import json
+import os
+import pickle
+import random
+import re
+import shutil
+import sys
+import textwrap
+import time
+import urllib.parse
+import uuid
+from contextlib import contextmanager
+from datetime import datetime, timedelta
+from functools import partial
+from typing import Any, Callable, Dict, Iterator, List, Optional, Union
+
+import requests
+import requests.utils
+
+from .exceptions import *
+
+
+def copy_session(session: requests.Session, request_timeout: Optional[float] = None) -> requests.Session:
+    """Duplicates a requests.Session."""
+    new = requests.Session()
+    new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
+    new.headers = session.headers.copy()  # type: ignore
+    # Override default timeout behavior.
+    # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+    new.request = partial(new.request, timeout=request_timeout)  # type: ignore
+    return new
+
+
+def default_user_agent() -> str:
+    return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \
+           '(KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36'
+
+
+def default_iphone_headers() -> Dict[str, Any]:
+    return {'User-Agent': 'Instagram 273.0.0.16.70 (iPad13,8; iOS 16_3; en_US; en-US; ' \
+                          'scale=2.00; 2048x2732; 452417278) AppleWebKit/420+',
+            'x-ads-opt-out': '1',
+            'x-bloks-is-panorama-enabled': 'true',
+            'x-bloks-version-id': '01507c21540f73e2216b6f62a11a5b5e51aa85491b72475c080da35b1228ddd6',
+            'x-fb-client-ip': 'True',
+            'x-fb-connection-type': 'wifi',
+            'x-fb-http-engine': 'Liger',
+            'x-fb-server-cluster': 'True',
+            'x-fb': '1',
+            'x-ig-abr-connection-speed-kbps': '2',
+            'x-ig-app-id': '124024574287414',
+            'x-ig-app-locale': 'en-US',
+            'x-ig-app-startup-country': 'US',
+            'x-ig-bandwidth-speed-kbps': '0.000',
+            'x-ig-capabilities': '36r/F/8=',
+            'x-ig-connection-speed': '{}kbps'.format(random.randint(1000, 20000)),
+            'x-ig-connection-type': 'WiFi',
+            'x-ig-device-locale': 'en-US',
+            'x-ig-mapped-locale': 'en-US',
+            'x-ig-timezone-offset': str((datetime.now().astimezone().utcoffset() or timedelta(seconds=0)).seconds),
+            'x-ig-www-claim': '0',
+            'x-pigeon-session-id': str(uuid.uuid4()),
+            'x-tigon-is-retry': 'False',
+            'x-whatsapp': '0'}
+
+
+class InstaloaderContext:
+    """Class providing methods for (error) logging and low-level communication with Instagram.
+
+    It is not thought to be instantiated directly, rather :class:`Instaloader` instances maintain a context
+    object.
+
+    For logging, it provides :meth:`log`, :meth:`error`, :meth:`error_catcher`.
+
+    It provides low-level communication routines :meth:`get_json`, :meth:`graphql_query`, :meth:`graphql_node_list`,
+    :meth:`get_and_write_raw` and implements mechanisms for rate controlling and error handling.
+
+    Further, it provides methods for logging in and general session handles, which are used by that routines in
+    class :class:`Instaloader`.
+    """
+
+    def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
+                 max_connection_attempts: int = 3, request_timeout: float = 300.0,
+                 rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None,
+                 fatal_status_codes: Optional[List[int]] = None,
+                 iphone_support: bool = True):
+
+        self.user_agent = user_agent if user_agent is not None else default_user_agent()
+        self.request_timeout = request_timeout
+        self._session = self.get_anonymous_session()
+        self.username = None
+        self.user_id = None
+        self.sleep = sleep
+        self.quiet = quiet
+        self.max_connection_attempts = max_connection_attempts
+        self._graphql_page_length = 50
+        self._root_rhx_gis = None
+        self.two_factor_auth_pending = None
+        self.iphone_support = iphone_support
+        self.iphone_headers = default_iphone_headers()
+
+        # error log, filled with error() and printed at the end of Instaloader.main()
+        self.error_log: List[str] = []
+
+        self._rate_controller = rate_controller(self) if rate_controller is not None else RateController(self)
+
+        # Can be set to True for testing, disables supression of InstaloaderContext._error_catcher
+        self.raise_all_errors = False
+
+        # HTTP status codes that should cause an AbortDownloadException
+        self.fatal_status_codes = fatal_status_codes or []
+
+        # Cache profile from id (mapping from id to Profile)
+        self.profile_id_cache: Dict[int, Any] = dict()
+
+    @contextmanager
+    def anonymous_copy(self):
+        session = self._session
+        username = self.username
+        user_id = self.user_id
+        iphone_headers = self.iphone_headers
+        self._session = self.get_anonymous_session()
+        self.username = None
+        self.user_id = None
+        self.iphone_headers = default_iphone_headers()
+        try:
+            yield self
+        finally:
+            self._session.close()
+            self.username = username
+            self._session = session
+            self.user_id = user_id
+            self.iphone_headers = iphone_headers
+
+    @property
+    def is_logged_in(self) -> bool:
+        """True, if this Instaloader instance is logged in."""
+        return bool(self.username)
+
+    def log(self, *msg, sep='', end='\n', flush=False):
+        """Log a message to stdout that can be suppressed with --quiet."""
+        if not self.quiet:
+            print(*msg, sep=sep, end=end, flush=flush)
+
+    def error(self, msg, repeat_at_end=True):
+        """Log a non-fatal error message to stderr, which is repeated at program termination.
+
+        :param msg: Message to be printed.
+        :param repeat_at_end: Set to false if the message should be printed, but not repeated at program termination."""
+        print(msg, file=sys.stderr)
+        if repeat_at_end:
+            self.error_log.append(msg)
+
+    def close(self):
+        """Print error log and close session"""
+        if self.error_log and not self.quiet:
+            print("\nErrors or warnings occurred:", file=sys.stderr)
+            for err in self.error_log:
+                print(err, file=sys.stderr)
+        self._session.close()
+
+    @contextmanager
+    def error_catcher(self, extra_info: Optional[str] = None):
+        """
+        Context manager to catch, print and record InstaloaderExceptions.
+
+        :param extra_info: String to prefix error message with."""
+        try:
+            yield
+        except InstaloaderException as err:
+            if extra_info:
+                self.error('{}: {}'.format(extra_info, err))
+            else:
+                self.error('{}'.format(err))
+            if self.raise_all_errors:
+                raise
+
+    def _default_http_header(self, empty_session_only: bool = False) -> Dict[str, str]:
+        """Returns default HTTP header we use for requests."""
+        header = {'Accept-Encoding': 'gzip, deflate',
+                  'Accept-Language': 'en-US,en;q=0.8',
+                  'Connection': 'keep-alive',
+                  'Content-Length': '0',
+                  'Host': 'www.instagram.com',
+                  'Origin': 'https://www.instagram.com',
+                  'Referer': 'https://www.instagram.com/',
+                  'User-Agent': self.user_agent,
+                  'X-Instagram-AJAX': '1',
+                  'X-Requested-With': 'XMLHttpRequest'}
+        if empty_session_only:
+            del header['Host']
+            del header['Origin']
+            del header['X-Instagram-AJAX']
+            del header['X-Requested-With']
+        return header
+
+    def get_anonymous_session(self) -> requests.Session:
+        """Returns our default anonymous requests.Session object."""
+        session = requests.Session()
+        session.cookies.update({'sessionid': '', 'mid': '', 'ig_pr': '1',
+                                'ig_vw': '1920', 'csrftoken': '',
+                                's_network': '', 'ds_user_id': ''})
+        session.headers.update(self._default_http_header(empty_session_only=True))
+        # Override default timeout behavior.
+        # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+        session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
+        return session
+
+    def save_session(self):
+        """Not meant to be used directly, use :meth:`Instaloader.save_session`."""
+        return requests.utils.dict_from_cookiejar(self._session.cookies)
+
+    def load_session(self, username, sessiondata):
+        """Not meant to be used directly, use :meth:`Instaloader.load_session`."""
+        session = requests.Session()
+        session.cookies = requests.utils.cookiejar_from_dict(sessiondata)
+        session.headers.update(self._default_http_header())
+        session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']})
+        # Override default timeout behavior.
+        # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+        session.request = partial(session.request, timeout=self.request_timeout)  # type: ignore
+        self._session = session
+        self.username = username
+
+    def save_session_to_file(self, sessionfile):
+        """Not meant to be used directly, use :meth:`Instaloader.save_session_to_file`."""
+        pickle.dump(self.save_session(), sessionfile)
+
+    def load_session_from_file(self, username, sessionfile):
+        """Not meant to be used directly, use :meth:`Instaloader.load_session_from_file`."""
+        self.load_session(username, pickle.load(sessionfile))
+
+    def test_login(self) -> Optional[str]:
+        """Not meant to be used directly, use :meth:`Instaloader.test_login`."""
+        data = self.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})
+        return data["data"]["user"]["username"] if data["data"]["user"] is not None else None
+
+    def login(self, user, passwd):
+        """Not meant to be used directly, use :meth:`Instaloader.login`.
+
+        :raises InvalidArgumentException: If the provided username does not exist.
+        :raises BadCredentialsException: If the provided password is wrong.
+        :raises ConnectionException: If connection to Instagram failed.
+        :raises TwoFactorAuthRequiredException: First step of 2FA login done, now call
+           :meth:`Instaloader.two_factor_login`."""
+        # pylint:disable=import-outside-toplevel
+        import http.client
+        # pylint:disable=protected-access
+        http.client._MAXHEADERS = 200
+        session = requests.Session()
+        session.cookies.update({'sessionid': '', 'mid': '', 'ig_pr': '1',
+                                'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '',
+                                's_network': '', 'ds_user_id': ''})
+        session.headers.update(self._default_http_header())
+        # Override default timeout behavior.
+        # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+        session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
+        csrf_json = self.get_json('accounts/login/', {}, session=session)
+        csrf_token = csrf_json['config']['csrf_token']
+        session.headers.update({'X-CSRFToken': csrf_token})
+        # Not using self.get_json() here, because we need to access csrftoken cookie
+        self.do_sleep()
+        # Workaround credits to pgrimaud.
+        # See: https://github.com/pgrimaud/instagram-user-feed/commit/96ad4cf54d1ad331b337f325c73e664999a6d066
+        enc_password = '#PWD_INSTAGRAM_BROWSER:0:{}:{}'.format(int(datetime.now().timestamp()), passwd)
+        login = session.post('https://www.instagram.com/accounts/login/ajax/',
+                             data={'enc_password': enc_password, 'username': user}, allow_redirects=True)
+        try:
+            resp_json = login.json()
+        except json.decoder.JSONDecodeError as err:
+            raise ConnectionException(
+                "Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason)
+            ) from err
+        if resp_json.get('two_factor_required'):
+            two_factor_session = copy_session(session, self.request_timeout)
+            two_factor_session.headers.update({'X-CSRFToken': csrf_token})
+            two_factor_session.cookies.update({'csrftoken': csrf_token})
+            self.two_factor_auth_pending = (two_factor_session,
+                                            user,
+                                            resp_json['two_factor_info']['two_factor_identifier'])
+            raise TwoFactorAuthRequiredException("Login error: two-factor authentication required.")
+        if resp_json.get('checkpoint_url'):
+            raise ConnectionException("Login: Checkpoint required. Point your browser to "
+                                      "https://www.instagram.com{} - "
+                                      "follow the instructions, then retry.".format(resp_json.get('checkpoint_url')))
+        if resp_json['status'] != 'ok':
+            if 'message' in resp_json:
+                raise ConnectionException("Login error: \"{}\" status, message \"{}\".".format(resp_json['status'],
+                                                                                               resp_json['message']))
+            else:
+                raise ConnectionException("Login error: \"{}\" status.".format(resp_json['status']))
+        if 'authenticated' not in resp_json:
+            # Issue #472
+            if 'message' in resp_json:
+                raise ConnectionException("Login error: Unexpected response, \"{}\".".format(resp_json['message']))
+            else:
+                raise ConnectionException("Login error: Unexpected response, this might indicate a blocked IP.")
+        if not resp_json['authenticated']:
+            if resp_json['user']:
+                # '{"authenticated": false, "user": true, "status": "ok"}'
+                raise BadCredentialsException('Login error: Wrong password.')
+            else:
+                # '{"authenticated": false, "user": false, "status": "ok"}'
+                # Raise InvalidArgumentException rather than BadCredentialException, because BadCredentialException
+                # triggers re-asking of password in Instaloader.interactive_login(), which makes no sense if the
+                # username is invalid.
+                raise InvalidArgumentException('Login error: User {} does not exist.'.format(user))
+        # '{"authenticated": true, "user": true, "userId": ..., "oneTapPrompt": false, "status": "ok"}'
+        session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
+        self._session = session
+        self.username = user
+        self.user_id = resp_json['userId']
+
+    def two_factor_login(self, two_factor_code):
+        """Second step of login if 2FA is enabled.
+        Not meant to be used directly, use :meth:`Instaloader.two_factor_login`.
+
+        :raises InvalidArgumentException: No two-factor authentication pending.
+        :raises BadCredentialsException: 2FA verification code invalid.
+
+        .. versionadded:: 4.2"""
+        if not self.two_factor_auth_pending:
+            raise InvalidArgumentException("No two-factor authentication pending.")
+        (session, user, two_factor_id) = self.two_factor_auth_pending
+
+        login = session.post('https://www.instagram.com/accounts/login/ajax/two_factor/',
+                             data={'username': user, 'verificationCode': two_factor_code, 'identifier': two_factor_id},
+                             allow_redirects=True)
+        resp_json = login.json()
+        if resp_json['status'] != 'ok':
+            if 'message' in resp_json:
+                raise BadCredentialsException("2FA error: {}".format(resp_json['message']))
+            else:
+                raise BadCredentialsException("2FA error: \"{}\" status.".format(resp_json['status']))
+        session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
+        self._session = session
+        self.username = user
+        self.two_factor_auth_pending = None
+
+    def do_sleep(self):
+        """Sleep a short time if self.sleep is set. Called before each request to instagram.com."""
+        if self.sleep:
+            time.sleep(min(random.expovariate(0.6), 15.0))
+
+    def get_json(self, path: str, params: Dict[str, Any], host: str = 'www.instagram.com',
+                 session: Optional[requests.Session] = None, _attempt=1,
+                 response_headers: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
+        """JSON request to Instagram.
+
+        :param path: URL, relative to the given domain which defaults to www.instagram.com/
+        :param params: GET parameters
+        :param host: Domain part of the URL from where to download the requested JSON; defaults to www.instagram.com
+        :param session: Session to use, or None to use self.session
+        :return: Decoded response dictionary
+        :raises QueryReturnedBadRequestException: When the server responds with a 400.
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises ConnectionException: When query repeatedly failed.
+        """
+        is_graphql_query = 'query_hash' in params and 'graphql/query' in path
+        is_iphone_query = host == 'i.instagram.com'
+        is_other_query = not is_graphql_query and host == "www.instagram.com"
+        sess = session if session else self._session
+        try:
+            self.do_sleep()
+            if is_graphql_query:
+                self._rate_controller.wait_before_query(params['query_hash'])
+            if is_iphone_query:
+                self._rate_controller.wait_before_query('iphone')
+            if is_other_query:
+                self._rate_controller.wait_before_query('other')
+            resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False)
+            if resp.status_code in self.fatal_status_codes:
+                redirect = " redirect to {}".format(resp.headers['location']) if 'location' in resp.headers else ""
+                body = ""
+                if resp.headers['Content-Type'].startswith('application/json'):
+                    body = ': ' + resp.text[:500] + ('…' if len(resp.text) > 501 else '')
+                raise AbortDownloadException("Query to https://{}/{} responded with \"{} {}\"{}{}".format(
+                    host, path, resp.status_code, resp.reason, redirect, body
+                ))
+            while resp.is_redirect:
+                redirect_url = resp.headers['location']
+                self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url))
+                if (redirect_url.startswith('https://www.instagram.com/accounts/login') or
+                    redirect_url.startswith('https://i.instagram.com/accounts/login')):
+                    if not self.is_logged_in:
+                        raise LoginRequiredException("Redirected to login page. Use --login.")
+                    raise AbortDownloadException("Redirected to login page. You've been logged out, please wait " +
+                                                 "some time, recreate the session and try again")
+                if redirect_url.startswith('https://{}/'.format(host)):
+                    resp = sess.get(redirect_url if redirect_url.endswith('/') else redirect_url + '/',
+                                    params=params, allow_redirects=False)
+                else:
+                    break
+            if response_headers is not None:
+                response_headers.clear()
+                response_headers.update(resp.headers)
+            if resp.status_code == 400:
+                raise QueryReturnedBadRequestException("400 Bad Request")
+            if resp.status_code == 404:
+                raise QueryReturnedNotFoundException("404 Not Found")
+            if resp.status_code == 429:
+                raise TooManyRequestsException("429 Too Many Requests")
+            if resp.status_code != 200:
+                raise ConnectionException("HTTP error code {}.".format(resp.status_code))
+            is_html_query = not is_graphql_query and not "__a" in params and host == "www.instagram.com"
+            if is_html_query:
+                # Extract JSON from HTML response
+                match = re.search('(?<={"raw":").*?(?<!\\\\)(?=")', resp.text)
+                if match is None:
+                    raise QueryReturnedNotFoundException("Could not find JSON data in html response.")
+                # Unescape escaped JSON string
+                unescaped_string = match.group(0).encode("utf-8").decode("unicode_escape")
+                resp_json = json.loads(unescaped_string)
+                return resp_json
+            else:
+                resp_json = resp.json()
+            if 'status' in resp_json and resp_json['status'] != "ok":
+                if 'message' in resp_json:
+                    raise ConnectionException("Returned \"{}\" status, message \"{}\".".format(resp_json['status'],
+                                                                                               resp_json['message']))
+                else:
+                    raise ConnectionException("Returned \"{}\" status.".format(resp_json['status']))
+            return resp_json
+        except (ConnectionException, json.decoder.JSONDecodeError, requests.exceptions.RequestException) as err:
+            error_string = "JSON Query to {}: {}".format(path, err)
+            if _attempt == self.max_connection_attempts:
+                if isinstance(err, QueryReturnedNotFoundException):
+                    raise QueryReturnedNotFoundException(error_string) from err
+                else:
+                    raise ConnectionException(error_string) from err
+            self.error(error_string + " [retrying; skip with ^C]", repeat_at_end=False)
+            try:
+                if isinstance(err, TooManyRequestsException):
+                    if is_graphql_query:
+                        self._rate_controller.handle_429(params['query_hash'])
+                    if is_iphone_query:
+                        self._rate_controller.handle_429('iphone')
+                    if is_other_query:
+                        self._rate_controller.handle_429('other')
+                return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1,
+                                     response_headers=response_headers)
+            except KeyboardInterrupt:
+                self.error("[skipped by user]", repeat_at_end=False)
+                raise ConnectionException(error_string) from err
+
+    def graphql_query(self, query_hash: str, variables: Dict[str, Any],
+                      referer: Optional[str] = None, rhx_gis: Optional[str] = None) -> Dict[str, Any]:
+        """
+        Do a GraphQL Query.
+
+        :param query_hash: Query identifying hash.
+        :param variables: Variables for the Query.
+        :param referer: HTTP Referer, or None.
+        :param rhx_gis: 'rhx_gis' variable as somewhere returned by Instagram, needed to 'sign' request
+        :return: The server's response dictionary.
+        """
+        with copy_session(self._session, self.request_timeout) as tmpsession:
+            tmpsession.headers.update(self._default_http_header(empty_session_only=True))
+            del tmpsession.headers['Connection']
+            del tmpsession.headers['Content-Length']
+            tmpsession.headers['authority'] = 'www.instagram.com'
+            tmpsession.headers['scheme'] = 'https'
+            tmpsession.headers['accept'] = '*/*'
+            if referer is not None:
+                tmpsession.headers['referer'] = urllib.parse.quote(referer)
+
+            variables_json = json.dumps(variables, separators=(',', ':'))
+
+            if rhx_gis:
+                #self.log("rhx_gis {} query_hash {}".format(rhx_gis, query_hash))
+                values = "{}:{}".format(rhx_gis, variables_json)
+                x_instagram_gis = hashlib.md5(values.encode()).hexdigest()
+                tmpsession.headers['x-instagram-gis'] = x_instagram_gis
+
+            resp_json = self.get_json('graphql/query',
+                                      params={'query_hash': query_hash,
+                                              'variables': variables_json},
+                                      session=tmpsession)
+        if 'status' not in resp_json:
+            self.error("GraphQL response did not contain a \"status\" field.")
+        return resp_json
+
+    def graphql_node_list(self, query_hash: str, query_variables: Dict[str, Any],
+                          query_referer: Optional[str],
+                          edge_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
+                          rhx_gis: Optional[str] = None,
+                          first_data: Optional[Dict[str, Any]] = None) -> Iterator[Dict[str, Any]]:
+        """
+        Retrieve a list of GraphQL nodes.
+
+        .. deprecated:: 4.5
+           Use :class:`NodeIterator` instead, which provides more functionality.
+        """
+
+        def _query():
+            query_variables['first'] = self._graphql_page_length
+            try:
+                return edge_extractor(self.graphql_query(query_hash, query_variables, query_referer, rhx_gis))
+            except QueryReturnedBadRequestException:
+                new_page_length = int(self._graphql_page_length / 2)
+                if new_page_length >= 12:
+                    self._graphql_page_length = new_page_length
+                    self.error("HTTP Error 400 (Bad Request) on GraphQL Query. Retrying with shorter page length.",
+                               repeat_at_end=False)
+                    return _query()
+                else:
+                    raise
+
+        if first_data:
+            data = first_data
+        else:
+            data = _query()
+        yield from (edge['node'] for edge in data['edges'])
+        while data['page_info']['has_next_page']:
+            query_variables['after'] = data['page_info']['end_cursor']
+            data = _query()
+            yield from (edge['node'] for edge in data['edges'])
+
+    def get_iphone_json(self, path: str, params: Dict[str, Any]) -> Dict[str, Any]:
+        """JSON request to ``i.instagram.com``.
+
+        :param path: URL, relative to ``i.instagram.com/``
+        :param params: GET parameters
+        :return: Decoded response dictionary
+        :raises QueryReturnedBadRequestException: When the server responds with a 400.
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises ConnectionException: When query repeatedly failed.
+
+        .. versionadded:: 4.2.1"""
+        with copy_session(self._session, self.request_timeout) as tempsession:
+            # Set headers to simulate an API request from iPad
+            tempsession.headers['ig-intended-user-id'] = str(self.user_id)
+            tempsession.headers['x-pigeon-rawclienttime'] = '{:.6f}'.format(time.time())
+
+            # Add headers obtained from previous iPad request
+            tempsession.headers.update(self.iphone_headers)
+
+            # Extract key information from cookies if we haven't got it already from a previous request
+            header_cookies_mapping = {'x-mid': 'mid',
+                                     'ig-u-ds-user-id': 'ds_user_id',
+                                     'x-ig-device-id': 'ig_did',
+                                     'x-ig-family-device-id': 'ig_did',
+                                     'family_device_id': 'ig_did'}
+
+            # Map the cookie value to the matching HTTP request header
+            cookies = tempsession.cookies.get_dict().copy()
+            for key, value in header_cookies_mapping.items():
+                if value in cookies:
+                    if key not in tempsession.headers:
+                        tempsession.headers[key] = cookies[value]
+                    else:
+                        # Remove the cookie value if it's already specified as a header
+                        tempsession.cookies.pop(value, None)
+
+            # Edge case for ig-u-rur header due to special string encoding in cookie
+            if 'rur' in cookies:
+                if 'ig-u-rur' not in tempsession.headers:
+                    tempsession.headers['ig-u-rur'] = cookies['rur'].strip('\"').encode('utf-8') \
+                                                                    .decode('unicode_escape')
+                else:
+                    tempsession.cookies.pop('rur', None)
+
+            # Remove headers specific to Desktop version
+            for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With', 'Referer']:
+                tempsession.headers.pop(header, None)
+
+            # No need for cookies if we have a bearer token
+            if 'authorization' in tempsession.headers:
+                tempsession.cookies.clear()
+
+            response_headers = dict()    # type: Dict[str, Any]
+            response = self.get_json(path, params, 'i.instagram.com', tempsession, response_headers=response_headers)
+
+            # Extract the ig-set-* headers and use them in the next request
+            for key, value in response_headers.items():
+                if key.startswith('ig-set-'):
+                    self.iphone_headers[key.replace('ig-set-', '')] = value
+                elif key.startswith('x-ig-set-'):
+                    self.iphone_headers[key.replace('x-ig-set-', 'x-ig-')] = value
+
+            return response
+
+    def write_raw(self, resp: Union[bytes, requests.Response], filename: str) -> None:
+        """Write raw response data into a file.
+
+        .. versionadded:: 4.2.1"""
+        self.log(filename, end=' ', flush=True)
+        with open(filename + '.temp', 'wb') as file:
+            if isinstance(resp, requests.Response):
+                shutil.copyfileobj(resp.raw, file)
+            else:
+                file.write(resp)
+        os.replace(filename + '.temp', filename)
+
+    def get_raw(self, url: str, _attempt=1) -> requests.Response:
+        """Downloads a file anonymously.
+
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises QueryReturnedForbiddenException: When the server responds with a 403.
+        :raises ConnectionException: When download failed.
+
+        .. versionadded:: 4.2.1"""
+        with self.get_anonymous_session() as anonymous_session:
+            resp = anonymous_session.get(url, stream=True)
+        if resp.status_code == 200:
+            resp.raw.decode_content = True
+            return resp
+        else:
+            if resp.status_code == 403:
+                # suspected invalid URL signature
+                raise QueryReturnedForbiddenException("403 when accessing {}.".format(url))
+            if resp.status_code == 404:
+                # 404 not worth retrying.
+                raise QueryReturnedNotFoundException("404 when accessing {}.".format(url))
+            raise ConnectionException("HTTP error code {}.".format(resp.status_code))
+
+    def get_and_write_raw(self, url: str, filename: str) -> None:
+        """Downloads and writes anonymously-requested raw data into a file.
+
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises QueryReturnedForbiddenException: When the server responds with a 403.
+        :raises ConnectionException: When download repeatedly failed."""
+        self.write_raw(self.get_raw(url), filename)
+
+    def head(self, url: str, allow_redirects: bool = False) -> requests.Response:
+        """HEAD a URL anonymously.
+
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises QueryReturnedForbiddenException: When the server responds with a 403.
+        :raises ConnectionException: When request failed.
+
+        .. versionadded:: 4.7.6
+        """
+        with self.get_anonymous_session() as anonymous_session:
+            resp = anonymous_session.head(url, allow_redirects=allow_redirects)
+        if resp.status_code == 200:
+            return resp
+        else:
+            if resp.status_code == 403:
+                # suspected invalid URL signature
+                raise QueryReturnedForbiddenException("403 when accessing {}.".format(url))
+            if resp.status_code == 404:
+                # 404 not worth retrying.
+                raise QueryReturnedNotFoundException("404 when accessing {}.".format(url))
+            raise ConnectionException("HTTP error code {}.".format(resp.status_code))
+
+    @property
+    def root_rhx_gis(self) -> Optional[str]:
+        """rhx_gis string returned in the / query."""
+        if self.is_logged_in:
+            # At the moment, rhx_gis seems to be required for anonymous requests only. By returning None when logged
+            # in, we can save the root_rhx_gis lookup query.
+            return None
+        if self._root_rhx_gis is None:
+            self._root_rhx_gis = self.get_json('', {}).get('rhx_gis', '')
+        return self._root_rhx_gis or None
+
+
+class RateController:
+    """
+    Class providing request tracking and rate controlling to stay within rate limits.
+
+    It can be overridden to change Instaloader's behavior regarding rate limits, for example to raise a custom
+    exception when the rate limit is hit::
+
+       import instaloader
+
+       class MyRateController(instaloader.RateController):
+           def sleep(self, secs):
+               raise MyCustomException()
+
+       L = instaloader.Instaloader(rate_controller=lambda ctx: MyRateController(ctx))
+    """
+
+    def __init__(self, context: InstaloaderContext):
+        self._context = context
+        self._query_timestamps: Dict[str, List[float]] = dict()
+        self._earliest_next_request_time = 0.0
+        self._iphone_earliest_next_request_time = 0.0
+
+    def sleep(self, secs: float):
+        """Wait given number of seconds."""
+        # Not static, to allow for the behavior of this method to depend on context-inherent properties, such as
+        # whether we are logged in.
+        time.sleep(secs)
+
+    def _dump_query_timestamps(self, current_time: float, failed_query_type: str):
+        windows = [10, 11, 20, 22, 30, 60]
+        self._context.error("Number of requests within last {} minutes grouped by type:"
+                            .format('/'.join(str(w) for w in windows)),
+                            repeat_at_end=False)
+        for query_type, times in self._query_timestamps.items():
+            reqs_in_sliding_window = [sum(t > current_time - w * 60 for t in times) for w in windows]
+            self._context.error(" {} {:>32}: {}".format(
+                "*" if query_type == failed_query_type else " ",
+                query_type,
+                " ".join("{:4}".format(reqs) for reqs in reqs_in_sliding_window)
+            ), repeat_at_end=False)
+
+    def count_per_sliding_window(self, query_type: str) -> int:
+        """Return how many requests of the given type can be done within a sliding window of 11 minutes.
+
+        This is called by :meth:`RateController.query_waittime` and allows to simply customize wait times before queries
+        at query_type granularity. Consider overriding :meth:`RateController.query_waittime` directly if you need more
+        control."""
+        # Not static, to allow for the count_per_sliding_window to depend on context-inherent properties, such as
+        # whether we are logged in.
+        return 75 if query_type == 'other' else 200
+
+    def _reqs_in_sliding_window(self, query_type: Optional[str], current_time: float, window: float) -> List[float]:
+        if query_type is not None:
+            # timestamps of type query_type
+            relevant_timestamps = self._query_timestamps[query_type]
+        else:
+            # all GraphQL queries, i.e. not 'iphone' or 'other'
+            graphql_query_timestamps = filter(lambda tp: tp[0] not in ['iphone', 'other'],
+                                              self._query_timestamps.items())
+            relevant_timestamps = [t for times in (tp[1] for tp in graphql_query_timestamps) for t in times]
+        return list(filter(lambda t: t > current_time - window, relevant_timestamps))
+
+    def query_waittime(self, query_type: str, current_time: float, untracked_queries: bool = False) -> float:
+        """Calculate time needed to wait before query can be executed."""
+        per_type_sliding_window = 660
+        iphone_sliding_window = 1800
+        if query_type not in self._query_timestamps:
+            self._query_timestamps[query_type] = []
+        self._query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60,
+                                                         self._query_timestamps[query_type]))
+
+        def per_type_next_request_time():
+            reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, per_type_sliding_window)
+            if len(reqs_in_sliding_window) < self.count_per_sliding_window(query_type):
+                return 0.0
+            else:
+                return min(reqs_in_sliding_window) + per_type_sliding_window + 6
+
+        def gql_accumulated_next_request_time():
+            if query_type in ['iphone', 'other']:
+                return 0.0
+            gql_accumulated_sliding_window = 600
+            gql_accumulated_max_count = 275
+            reqs_in_sliding_window = self._reqs_in_sliding_window(None, current_time, gql_accumulated_sliding_window)
+            if len(reqs_in_sliding_window) < gql_accumulated_max_count:
+                return 0.0
+            else:
+                return min(reqs_in_sliding_window) + gql_accumulated_sliding_window
+
+        def untracked_next_request_time():
+            if untracked_queries:
+                if query_type == "iphone":
+                    reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
+                                                                          iphone_sliding_window)
+                    self._iphone_earliest_next_request_time = min(reqs_in_sliding_window) + iphone_sliding_window + 18
+                else:
+                    reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
+                                                                          per_type_sliding_window)
+                    self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6
+            return max(self._iphone_earliest_next_request_time, self._earliest_next_request_time)
+
+        def iphone_next_request():
+            if query_type == "iphone":
+                reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, iphone_sliding_window)
+                if len(reqs_in_sliding_window) >= 199:
+                    return min(reqs_in_sliding_window) + iphone_sliding_window + 18
+            return 0.0
+
+        return max(0.0,
+                   max(
+                       per_type_next_request_time(),
+                       gql_accumulated_next_request_time(),
+                       untracked_next_request_time(),
+                       iphone_next_request(),
+                   ) - current_time)
+
+    def wait_before_query(self, query_type: str) -> None:
+        """This method is called before a query to Instagram.
+
+        It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
+        :meth:`RateController.sleep` to wait until the request can be made."""
+        waittime = self.query_waittime(query_type, time.monotonic(), False)
+        assert waittime >= 0
+        if waittime > 15:
+            formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
+                                  "{} minutes".format(round(waittime / 60)))
+            self._context.log("\nToo many queries in the last time. Need to wait {}, until {:%H:%M}."
+                              .format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)))
+        if waittime > 0:
+            self.sleep(waittime)
+        if query_type not in self._query_timestamps:
+            self._query_timestamps[query_type] = [time.monotonic()]
+        else:
+            self._query_timestamps[query_type].append(time.monotonic())
+
+    def handle_429(self, query_type: str) -> None:
+        """This method is called to handle a 429 Too Many Requests response.
+
+        It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
+        :meth:`RateController.sleep` to wait until we can repeat the same request."""
+        current_time = time.monotonic()
+        waittime = self.query_waittime(query_type, current_time, True)
+        assert waittime >= 0
+        self._dump_query_timestamps(current_time, query_type)
+        text_for_429 = ("Instagram responded with HTTP error \"429 - Too Many Requests\". Please do not run multiple "
+                        "instances of Instaloader in parallel or within short sequence. Also, do not use any Instagram "
+                        "App while Instaloader is running.")
+        self._context.error(textwrap.fill(text_for_429), repeat_at_end=False)
+        if waittime > 1.5:
+            formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
+                                  "{} minutes".format(round(waittime / 60)))
+            self._context.error("The request will be retried in {}, at {:%H:%M}."
+                                .format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)),
+                                repeat_at_end=False)
+        if waittime > 0:
+            self.sleep(waittime)

instaloader/lateststamps.py

@@ -0,0 +1,117 @@
+import configparser
+from datetime import datetime, timezone
+from typing import Optional
+from os.path import dirname
+from os import makedirs
+
+
+class LatestStamps:
+    """LatestStamps class.
+
+    Convenience class for retrieving and storing data from the :option:`--latest-stamps` file.
+
+    :param latest_stamps_file: path to file.
+
+    .. versionadded:: 4.8"""
+    PROFILE_ID = 'profile-id'
+    PROFILE_PIC = 'profile-pic'
+    POST_TIMESTAMP = 'post-timestamp'
+    TAGGED_TIMESTAMP = 'tagged-timestamp'
+    IGTV_TIMESTAMP = 'igtv-timestamp'
+    STORY_TIMESTAMP = 'story-timestamp'
+    ISO_FORMAT = '%Y-%m-%dT%H:%M:%S.%f%z'
+
+    def __init__(self, latest_stamps_file):
+        self.file = latest_stamps_file
+        self.data = configparser.ConfigParser()
+        self.data.read(latest_stamps_file)
+
+    def _save(self):
+        if dn := dirname(self.file):
+            makedirs(dn, exist_ok=True)
+        with open(self.file, 'w') as f:
+            self.data.write(f)
+
+    def _ensure_section(self, section: str):
+        if not self.data.has_section(section):
+            self.data.add_section(section)
+
+    def get_profile_id(self, profile_name: str) -> Optional[int]:
+        """Returns stored ID of profile."""
+        try:
+            return self.data.getint(profile_name, self.PROFILE_ID)
+        except (configparser.Error, ValueError):
+            return None
+
+    def save_profile_id(self, profile_name: str, profile_id: int):
+        """Stores ID of profile."""
+        self._ensure_section(profile_name)
+        self.data.set(profile_name, self.PROFILE_ID, str(profile_id))
+        self._save()
+
+    def rename_profile(self, old_profile: str, new_profile: str):
+        """Renames a profile."""
+        self._ensure_section(new_profile)
+        for option in [self.PROFILE_ID, self.PROFILE_PIC, self.POST_TIMESTAMP,
+                       self.TAGGED_TIMESTAMP, self.IGTV_TIMESTAMP, self.STORY_TIMESTAMP]:
+            if self.data.has_option(old_profile, option):
+                value = self.data.get(old_profile, option)
+                self.data.set(new_profile, option, value)
+        self.data.remove_section(old_profile)
+        self._save()
+
+    def _get_timestamp(self, section: str, key: str) -> datetime:
+        try:
+            return datetime.strptime(self.data.get(section, key), self.ISO_FORMAT)
+        except (configparser.Error, ValueError):
+            return datetime.fromtimestamp(0, timezone.utc)
+
+    def _set_timestamp(self, section: str, key: str, timestamp: datetime):
+        self._ensure_section(section)
+        self.data.set(section, key, timestamp.strftime(self.ISO_FORMAT))
+        self._save()
+
+    def get_last_post_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's posts."""
+        return self._get_timestamp(profile_name, self.POST_TIMESTAMP)
+
+    def set_last_post_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's posts."""
+        self._set_timestamp(profile_name, self.POST_TIMESTAMP, timestamp)
+
+    def get_last_tagged_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's tagged posts."""
+        return self._get_timestamp(profile_name, self.TAGGED_TIMESTAMP)
+
+    def set_last_tagged_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's tagged posts."""
+        self._set_timestamp(profile_name, self.TAGGED_TIMESTAMP, timestamp)
+
+    def get_last_igtv_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's igtv posts."""
+        return self._get_timestamp(profile_name, self.IGTV_TIMESTAMP)
+
+    def set_last_igtv_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's igtv posts."""
+        self._set_timestamp(profile_name, self.IGTV_TIMESTAMP, timestamp)
+
+    def get_last_story_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's stories."""
+        return self._get_timestamp(profile_name, self.STORY_TIMESTAMP)
+
+    def set_last_story_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's stories."""
+        self._set_timestamp(profile_name, self.STORY_TIMESTAMP, timestamp)
+
+    def get_profile_pic(self, profile_name: str) -> str:
+        """Returns filename of profile's last downloaded profile pic."""
+        try:
+            return self.data.get(profile_name, self.PROFILE_PIC)
+        except configparser.Error:
+            return ""
+
+    def set_profile_pic(self, profile_name: str, profile_pic: str):
+        """Sets filename of profile's last downloaded profile pic."""
+        self._ensure_section(profile_name)
+        self.data.set(profile_name, self.PROFILE_PIC, profile_pic)
+        self._save()

instaloader/nodeiterator.py

@@ -0,0 +1,304 @@
+import base64
+import hashlib
+import json
+import os
+from contextlib import contextmanager
+from datetime import datetime, timedelta
+from lzma import LZMAError
+from typing import Any, Callable, Dict, Iterable, Iterator, NamedTuple, Optional, Tuple, TypeVar
+
+from .exceptions import AbortDownloadException, InvalidArgumentException, QueryReturnedBadRequestException
+from .instaloadercontext import InstaloaderContext
+
+class FrozenNodeIterator(NamedTuple):
+    query_hash: str
+    query_variables: Dict
+    query_referer: Optional[str]
+    context_username: Optional[str]
+    total_index: int
+    best_before: Optional[float]
+    remaining_data: Optional[Dict]
+    first_node: Optional[Dict]
+FrozenNodeIterator.query_hash.__doc__ = """The GraphQL ``query_hash`` parameter."""
+FrozenNodeIterator.query_variables.__doc__ = """The GraphQL ``query_variables`` parameter."""
+FrozenNodeIterator.query_referer.__doc__ = """The HTTP referer used for the GraphQL query."""
+FrozenNodeIterator.context_username.__doc__ = """The username who created the iterator, or ``None``."""
+FrozenNodeIterator.total_index.__doc__ = """Number of items that have already been returned."""
+FrozenNodeIterator.best_before.__doc__ = """Date when parts of the stored nodes might have expired."""
+FrozenNodeIterator.remaining_data.__doc__ = \
+    """The already-retrieved, yet-unprocessed ``edges`` and the ``page_info`` at time of freezing."""
+FrozenNodeIterator.first_node.__doc__ = """Node data of the first item, if an item has been produced."""
+
+T = TypeVar('T')
+
+
+class NodeIterator(Iterator[T]):
+    """
+    Iterate the nodes within edges in a GraphQL pagination. Instances of this class are returned by many (but not all)
+    of Instaloader's :class:`Post`-returning functions (such as :meth:`Profile.get_posts` etc.).
+
+    What makes this iterator special is its ability to freeze/store its current state, e.g. to interrupt an iteration,
+    and later thaw/resume from where it left off.
+
+    You can freeze a NodeIterator with :meth:`NodeIterator.freeze`::
+
+       post_iterator = profile.get_posts()
+       try:
+           for post in post_iterator:
+               do_something_with(post)
+       except KeyboardInterrupt:
+           save("resume_information.json", post_iterator.freeze())
+
+    and later reuse it with :meth:`NodeIterator.thaw` on an equally-constructed NodeIterator::
+
+       post_iterator = profile.get_posts()
+       post_iterator.thaw(load("resume_information.json"))
+
+    (an appropriate method to load and save the :class:`FrozenNodeIterator` is e.g.
+    :func:`load_structure_from_file` and :func:`save_structure_to_file`.)
+
+    A :class:`FrozenNodeIterator` can only be thawn with a matching NodeIterator, i.e. a NodeIterator instance that has
+    been constructed with the same parameters as the instance that is represented by the :class:`FrozenNodeIterator` in
+    question. This is to ensure that an iteration cannot be resumed in a wrong, unmatching loop. As a quick way to
+    distinguish iterators that are saved e.g. in files, there is the :attr:`NodeIterator.magic` string: Two
+    NodeIterators are matching if and only if they have the same magic.
+
+    See also :func:`resumable_iteration` for a high-level context manager that handles a resumable iteration.
+    """
+
+    _graphql_page_length = 50
+    _shelf_life = timedelta(days=29)
+
+    def __init__(self,
+                 context: InstaloaderContext,
+                 query_hash: str,
+                 edge_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
+                 node_wrapper: Callable[[Dict], T],
+                 query_variables: Optional[Dict[str, Any]] = None,
+                 query_referer: Optional[str] = None,
+                 first_data: Optional[Dict[str, Any]] = None,
+                 is_first: Optional[Callable[[T, Optional[T]], bool]] = None):
+        self._context = context
+        self._query_hash = query_hash
+        self._edge_extractor = edge_extractor
+        self._node_wrapper = node_wrapper
+        self._query_variables = query_variables if query_variables is not None else {}
+        self._query_referer = query_referer
+        self._page_index = 0
+        self._total_index = 0
+        if first_data is not None:
+            self._data = first_data
+            self._best_before = datetime.now() + NodeIterator._shelf_life
+        else:
+            self._data = self._query()
+        self._first_node: Optional[Dict] = None
+        self._is_first = is_first
+
+    def _query(self, after: Optional[str] = None) -> Dict:
+        pagination_variables: Dict[str, Any] = {'first': NodeIterator._graphql_page_length}
+        if after is not None:
+            pagination_variables['after'] = after
+        try:
+            data = self._edge_extractor(
+                self._context.graphql_query(
+                    self._query_hash, {**self._query_variables, **pagination_variables}, self._query_referer
+                )
+            )
+            self._best_before = datetime.now() + NodeIterator._shelf_life
+            return data
+        except QueryReturnedBadRequestException:
+            new_page_length = int(NodeIterator._graphql_page_length / 2)
+            if new_page_length >= 12:
+                NodeIterator._graphql_page_length = new_page_length
+                self._context.error("HTTP Error 400 (Bad Request) on GraphQL Query. Retrying with shorter page length.",
+                                    repeat_at_end=False)
+                return self._query(after)
+            else:
+                raise
+
+    def __iter__(self):
+        return self
+
+    def __next__(self) -> T:
+        if self._page_index < len(self._data['edges']):
+            node = self._data['edges'][self._page_index]['node']
+            page_index, total_index = self._page_index, self._total_index
+            try:
+                self._page_index += 1
+                self._total_index += 1
+            except KeyboardInterrupt:
+                self._page_index, self._total_index = page_index, total_index
+                raise
+            item = self._node_wrapper(node)
+            if self._is_first is not None:
+                if self._is_first(item, self.first_item):
+                    self._first_node = node
+            else:
+                if self._first_node is None:
+                    self._first_node = node
+            return item
+        if self._data.get('page_info', {}).get('has_next_page'):
+            query_response = self._query(self._data['page_info']['end_cursor'])
+            if self._data['edges'] != query_response['edges'] and len(query_response['edges']) > 0:
+                page_index, data = self._page_index, self._data
+                try:
+                    self._page_index = 0
+                    self._data = query_response
+                except KeyboardInterrupt:
+                    self._page_index, self._data = page_index, data
+                    raise
+                return self.__next__()
+        raise StopIteration()
+
+    @property
+    def count(self) -> Optional[int]:
+        """The ``count`` as returned by Instagram. This is not always the total count this iterator will yield."""
+        return self._data.get('count') if self._data is not None else None
+
+    @property
+    def total_index(self) -> int:
+        """Number of items that have already been returned."""
+        return self._total_index
+
+    @property
+    def magic(self) -> str:
+        """Magic string for easily identifying a matching iterator file for resuming (hash of some parameters)."""
+        magic_hash = hashlib.blake2b(digest_size=6)
+        magic_hash.update(json.dumps(
+            [self._query_hash, self._query_variables, self._query_referer, self._context.username]
+        ).encode())
+        return base64.urlsafe_b64encode(magic_hash.digest()).decode()
+
+    @property
+    def first_item(self) -> Optional[T]:
+        """
+        If this iterator has produced any items, returns the first item produced.
+
+        It is possible to override what is considered the first item (for example, to consider the
+        newest item in case items are not in strict chronological order) by passing a callback
+        function as the `is_first` parameter when creating the class.
+
+        .. versionadded:: 4.8
+        .. versionchanged:: 4.9.2
+           What is considered the first item can be overridden.
+        """
+        return self._node_wrapper(self._first_node) if self._first_node is not None else None
+
+    def freeze(self) -> FrozenNodeIterator:
+        """Freeze the iterator for later resuming."""
+        remaining_data = None
+        if self._data is not None:
+            remaining_data = {**self._data,
+                              'edges': (self._data['edges'][(max(self._page_index - 1, 0)):])}
+        return FrozenNodeIterator(
+            query_hash=self._query_hash,
+            query_variables=self._query_variables,
+            query_referer=self._query_referer,
+            context_username=self._context.username,
+            total_index=max(self.total_index - 1, 0),
+            best_before=self._best_before.timestamp() if self._best_before else None,
+            remaining_data=remaining_data,
+            first_node=self._first_node,
+        )
+
+    def thaw(self, frozen: FrozenNodeIterator) -> None:
+        """
+        Use this iterator for resuming from earlier iteration.
+
+        :raises InvalidArgumentException:
+           If
+
+           - the iterator on which this method is called has already been used, or
+           - the given :class:`FrozenNodeIterator` does not match, i.e. belongs to a different iteration.
+        """
+        if self._total_index or self._page_index:
+            raise InvalidArgumentException("thaw() called on already-used iterator.")
+        if (self._query_hash != frozen.query_hash or
+                self._query_variables != frozen.query_variables or
+                self._query_referer != frozen.query_referer or
+                self._context.username != frozen.context_username):
+            raise InvalidArgumentException("Mismatching resume information.")
+        if not frozen.best_before:
+            raise InvalidArgumentException("\"best before\" date missing.")
+        if frozen.remaining_data is None:
+            raise InvalidArgumentException("\"remaining_data\" missing.")
+        self._total_index = frozen.total_index
+        self._best_before = datetime.fromtimestamp(frozen.best_before)
+        self._data = frozen.remaining_data
+        if frozen.first_node is not None:
+            self._first_node = frozen.first_node
+
+
+@contextmanager
+def resumable_iteration(context: InstaloaderContext,
+                        iterator: Iterable,
+                        load: Callable[[InstaloaderContext, str], Any],
+                        save: Callable[[FrozenNodeIterator, str], None],
+                        format_path: Callable[[str], str],
+                        check_bbd: bool = True,
+                        enabled: bool = True) -> Iterator[Tuple[bool, int]]:
+    """
+    High-level context manager to handle a resumable iteration that can be interrupted
+    with a :class:`KeyboardInterrupt` or an :class:`AbortDownloadException`.
+
+    It can be used as follows to automatically load a previously-saved state into the iterator, save the iterator's
+    state when interrupted, and delete the resume file upon completion::
+
+       post_iterator = profile.get_posts()
+       with resumable_iteration(
+               context=L.context,
+               iterator=post_iterator,
+               load=lambda _, path: FrozenNodeIterator(**json.load(open(path))),
+               save=lambda fni, path: json.dump(fni._asdict(), open(path, 'w')),
+               format_path=lambda magic: "resume_info_{}.json".format(magic)
+       ) as (is_resuming, start_index):
+           for post in post_iterator:
+               do_something_with(post)
+
+    It yields a tuple (is_resuming, start_index).
+
+    When the passed iterator is not a :class:`NodeIterator`, it behaves as if ``resumable_iteration`` was not used,
+    just executing the inner body.
+
+    :param context: The :class:`InstaloaderContext`.
+    :param iterator: The fresh :class:`NodeIterator`.
+    :param load: Loads a FrozenNodeIterator from given path. The object is ignored if it has a different type.
+    :param save: Saves the given FrozenNodeIterator to the given path.
+    :param format_path: Returns the path to the resume file for the given magic.
+    :param check_bbd: Whether to check the best before date and reject an expired FrozenNodeIterator.
+    :param enabled: Set to False to disable all functionality and simply execute the inner body.
+
+    .. versionchanged:: 4.7
+       Also interrupt on :class:`AbortDownloadException`.
+    """
+    if not enabled or not isinstance(iterator, NodeIterator):
+        yield False, 0
+        return
+    is_resuming = False
+    start_index = 0
+    resume_file_path = format_path(iterator.magic)
+    resume_file_exists = os.path.isfile(resume_file_path)
+    if resume_file_exists:
+        try:
+            fni = load(context, resume_file_path)
+            if not isinstance(fni, FrozenNodeIterator):
+                raise InvalidArgumentException("Invalid type.")
+            if check_bbd and fni.best_before and datetime.fromtimestamp(fni.best_before) < datetime.now():
+                raise InvalidArgumentException("\"Best before\" date exceeded.")
+            iterator.thaw(fni)
+            is_resuming = True
+            start_index = iterator.total_index
+            context.log("Resuming from {}.".format(resume_file_path))
+        except (InvalidArgumentException, LZMAError, json.decoder.JSONDecodeError, EOFError) as exc:
+            context.error("Warning: Not resuming from {}: {}".format(resume_file_path, exc))
+    try:
+        yield is_resuming, start_index
+    except (KeyboardInterrupt, AbortDownloadException):
+        if os.path.dirname(resume_file_path):
+            os.makedirs(os.path.dirname(resume_file_path), exist_ok=True)
+        save(iterator.freeze(), resume_file_path)
+        context.log("\nSaved resume information to {}.".format(resume_file_path))
+        raise
+    if resume_file_exists:
+        os.unlink(resume_file_path)
+        context.log("Iteration complete, deleted resume information file {}.".format(resume_file_path))

instaloader/sectioniterator.py

@@ -0,0 +1,46 @@
+from typing import Any, Callable, Dict, Iterator, Optional, TypeVar
+
+from .instaloadercontext import InstaloaderContext
+
+T = TypeVar('T')
+
+
+class SectionIterator(Iterator[T]):
+    """Iterator for the new 'sections'-style responses.
+
+    .. versionadded:: 4.9"""
+    def __init__(self,
+                 context: InstaloaderContext,
+                 sections_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
+                 media_wrapper: Callable[[Dict], T],
+                 query_path: str,
+                 first_data: Optional[Dict[str, Any]] = None):
+        self._context = context
+        self._sections_extractor = sections_extractor
+        self._media_wrapper = media_wrapper
+        self._query_path = query_path
+        self._data = first_data or self._query()
+        self._page_index = 0
+        self._section_index = 0
+
+    def __iter__(self):
+        return self
+
+    def _query(self, max_id: Optional[str] = None) -> Dict[str, Any]:
+        pagination_variables = {"max_id": max_id} if max_id is not None else {}
+        return self._sections_extractor(
+            self._context.get_json(self._query_path, params={"__a": 1, "__d": "dis", **pagination_variables})
+        )
+
+    def __next__(self) -> T:
+        if self._page_index < len(self._data['sections']):
+            media = self._data['sections'][self._page_index]['layout_content']['medias'][self._section_index]['media']
+            self._section_index += 1
+            if self._section_index >= len(self._data['sections'][self._page_index]['layout_content']['medias']):
+                self._section_index = 0
+                self._page_index += 1
+            return self._media_wrapper(media)
+        if self._data['more_available']:
+            self._page_index, self._section_index, self._data = 0, 0, self._query(self._data["next_max_id"])
+            return self.__next__()
+        raise StopIteration()

setup.py

@@ -1,35 +1,63 @@
 #!/usr/bin/env python3
 
+import re
 import sys
+import os
 from setuptools import setup
 
-if sys.version_info < (3, 5):
-    sys.exit('Instaloader requires Python >= 3.5.')
+
+SRC = os.path.abspath(os.path.dirname(__file__))
+
+
+def get_version():
+    with open(os.path.join(SRC, 'instaloader/__init__.py')) as f:
+        for line in f:
+            m = re.match("__version__ = '(.*)'", line)
+            if m:
+                return m.group(1)
+    raise SystemExit("Could not find version string.")
+
+
+if sys.version_info < (3, 8):
+    sys.exit('Instaloader requires Python >= 3.8.')
+
+requirements = ['requests>=2.4']
+
+keywords = (['instagram', 'instagram-scraper', 'instagram-client', 'instagram-feed', 'downloader', 'videos', 'photos',
+             'pictures', 'instagram-user-photos', 'instagram-photos', 'instagram-metadata', 'instagram-downloader',
+             'instagram-stories'])
+
+# NOTE that many of the values defined in this file are duplicated on other places, such as the
+# documentation.
 
 setup(
     name='instaloader',
-    version='2.2.2',
-    py_modules=['instaloader'],
-    url='https://github.com/Thammus/instaloader',
+    version=get_version(),
+    packages=['instaloader'],
+    package_data={'instaloader': ['py.typed']},
+    url='https://instaloader.github.io/',
     license='MIT',
     author='Alexander Graf, André Koch-Kramer',
     author_email='mail@agraf.me, koch-kramer@web.de',
     description='Download pictures (or videos) along with their captions and other metadata '
                 'from Instagram.',
-    long_description=open('README.rst').read(),
-    install_requires=['requests>=2.4'],
-    python_requires='>=3.5',
-    entry_points={'console_scripts': ['instaloader=instaloader:main']},
-    zip_safe=True,
-    keywords='instagram downloader',
+    long_description=open(os.path.join(SRC, 'README.rst')).read(),
+    install_requires=requirements,
+    python_requires='>=3.8',
+    entry_points={'console_scripts': ['instaloader=instaloader.__main__:main']},
+    zip_safe=False,
+    keywords=keywords,
     classifiers=[
         'Development Status :: 5 - Production/Stable',
         'Environment :: Console',
         'Intended Audience :: End Users/Desktop',
+        'Intended Audience :: Developers',
         'License :: OSI Approved :: MIT License',
         'Operating System :: OS Independent',
-        'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: 3.6',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: 3.9',
+        'Programming Language :: Python :: 3.10',
+        'Programming Language :: Python :: 3.11',
         'Programming Language :: Python :: 3 :: Only',
         'Topic :: Internet',
         'Topic :: Multimedia :: Graphics'

test/instaloader_unittests.py

@@ -0,0 +1,214 @@
+"""Unit Tests for Instaloader"""
+
+import os
+import shutil
+import tempfile
+import unittest
+from itertools import islice
+from typing import Optional
+
+import instaloader
+
+PROFILE_WITH_HIGHLIGHTS = 325732271
+PUBLIC_PROFILE = "selenagomez"
+PUBLIC_PROFILE_ID = 460563723
+PUBLIC_PROFILE_WITH_IGTV = "natgeo"
+HASHTAG = "kitten"
+LOCATION = "362629379"
+OWN_USERNAME = "aandergr"
+NORMAL_MAX_COUNT = 2
+PAGING_MAX_COUNT = 15
+PRIVATE_PROFILE = "aandergr"
+PRIVATE_PROFILE_ID = 1706625676
+EMPTY_PROFILE = "not_public"
+EMPTY_PROFILE_ID = 1928659031
+
+ratecontroller: Optional[instaloader.RateController] = None
+
+
+class TestInstaloaderAnonymously(unittest.TestCase):
+
+    def setUp(self):
+        self.dir = tempfile.mkdtemp()
+        print("Testing in {}".format(self.dir))
+        os.chdir(self.dir)
+        self.L = instaloader.Instaloader(download_geotags=True,
+                                         download_comments=True,
+                                         save_metadata=True)
+        self.L.context.raise_all_errors = True
+        if ratecontroller is not None:
+            # pylint:disable=protected-access
+            ratecontroller._context = self.L.context
+            self.L.context._rate_controller = ratecontroller
+
+    def tearDown(self):
+        # pylint:disable=global-statement,protected-access
+        global ratecontroller
+        ratecontroller = self.L.context._rate_controller
+        self.L.close()
+        os.chdir('/')
+        print("Removing {}".format(self.dir))
+        shutil.rmtree(self.dir)
+
+    def post_paging_test(self, iterator):
+        previous_post = None
+        for post in islice(iterator, PAGING_MAX_COUNT):
+            print(post)
+            if previous_post:
+                self.assertTrue(post.date_utc < previous_post.date_utc)
+            previous_post = post
+
+    @unittest.skip('')
+    def test_public_profile_download(self):
+        profiles = {self.L.check_profile_id(PUBLIC_PROFILE)}
+        self.L.download_profiles(profiles, profile_pic=False, fast_update=True, raise_errors=True)
+        self.L.download_profiles(profiles, profile_pic=False, fast_update=True, raise_errors=True)
+
+    def test_public_profile_paging(self):
+        self.post_paging_test(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts())
+
+    def test_profile_pic_download(self):
+        self.L.download_profiles({self.L.check_profile_id(PUBLIC_PROFILE)}, posts=False, raise_errors=True)
+
+    def test_hashtag_download(self):
+        self.L.download_hashtag(HASHTAG, NORMAL_MAX_COUNT)
+
+    def test_hashtag_paging(self):
+        for count, post in enumerate(instaloader.Hashtag.from_name(self.L.context, HASHTAG).get_all_posts()):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
+    def test_get_id_by_username(self):
+        self.assertEqual(PUBLIC_PROFILE_ID,
+                         instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).userid)
+
+    def test_get_username_by_id_private(self):
+        self.assertEqual(PRIVATE_PROFILE.lower(),
+                         instaloader.Profile.from_id(self.L.context, PRIVATE_PROFILE_ID).username)
+
+    def test_get_username_by_id_public(self):
+        self.assertEqual(PUBLIC_PROFILE.lower(),
+                         instaloader.Profile.from_id(self.L.context, PUBLIC_PROFILE_ID).username)
+
+    def test_get_username_by_id_empty(self):
+        self.assertEqual(EMPTY_PROFILE.lower(),
+                         instaloader.Profile.from_id(self.L.context, EMPTY_PROFILE_ID).username)
+
+    def test_get_username_by_name_empty(self):
+        self.assertEqual(EMPTY_PROFILE_ID,
+                         instaloader.Profile.from_username(self.L.context, EMPTY_PROFILE).userid)
+
+    def test_post_from_mediaid(self):
+        for post in instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts():
+            post2 = instaloader.Post.from_mediaid(self.L.context, post.mediaid)
+            self.assertEqual(post, post2)
+            break
+
+    def test_public_profile_tagged_paging(self):
+        for post in islice(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_tagged_posts(),
+                           PAGING_MAX_COUNT):
+            print(post)
+
+    def test_public_profile_igtv(self):
+        for post in islice(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE_WITH_IGTV).get_igtv_posts(),
+                           PAGING_MAX_COUNT):
+            print(post)
+
+
+class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
+
+    def setUp(self):
+        super().setUp()
+        self.L.load_session_from_file(OWN_USERNAME)
+
+    @unittest.skip('')
+    def test_stories_download(self):
+        self.L.download_stories()
+
+    @unittest.skip('')
+    def test_private_profile_download(self):
+        self.L.download_profiles({self.L.check_profile_id(PRIVATE_PROFILE)}, stories=True, raise_errors=True)
+
+    def test_stories_paging(self):
+        for user_story in self.L.get_stories():
+            print("profile {}.".format(user_story.owner_username))
+            for item in user_story.get_items():
+                print(item)
+
+    def test_highlights_paging(self):
+        for user_highlight in self.L.get_highlights(PROFILE_WITH_HIGHLIGHTS):
+            print("Retrieving {} highlights \"{}\" from profile {}".format(user_highlight.itemcount,
+                                                                           user_highlight.title,
+                                                                           user_highlight.owner_username))
+            for item in user_highlight.get_items():
+                print(item)
+
+    def test_private_profile_paging(self):
+        self.post_paging_test(instaloader.Profile.from_username(self.L.context, PRIVATE_PROFILE).get_posts())
+
+    def test_feed_download(self):
+        self.L.download_feed_posts(NORMAL_MAX_COUNT)
+
+    def test_feed_paging(self):
+        for count, post in enumerate(self.L.get_feed_posts()):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
+    def test_saved_download(self):
+        self.L.download_saved_posts(NORMAL_MAX_COUNT)
+
+    def test_saved_paging(self):
+        for count, post in enumerate(instaloader.Profile.from_username(self.L.context, OWN_USERNAME).get_saved_posts()):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
+    def test_test_login(self):
+        self.assertEqual(OWN_USERNAME, self.L.test_login())
+
+    def test_followees_and_stories(self):
+        profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
+        followees = set(islice(profile.get_followees(), PAGING_MAX_COUNT))
+        self.L.download_profiles(followees, profile_pic=False, stories=True, posts=False, raise_errors=True)
+
+    def test_get_followees(self):
+        profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
+        for f in profile.get_followees():
+            print(f.username)
+
+    def test_get_followers(self):
+        profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
+        for f in profile.get_followers():
+            print(f.username)
+
+    def test_get_similar_accounts(self):
+        profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
+        for f in profile.get_similar_accounts():
+            print(f.username)
+
+    def test_get_likes(self):
+        for post in instaloader.Profile.from_username(self.L.context, OWN_USERNAME).get_posts():
+            for like in post.get_likes():
+                print(like.username)
+            break
+
+    def test_explore_paging(self):
+        for count, post in enumerate(self.L.get_explore_posts()):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
+    def test_location_download(self):
+        self.L.download_location(LOCATION, NORMAL_MAX_COUNT)
+
+    def test_location_paging(self):
+        for count, post in enumerate(self.L.get_location_posts(LOCATION)):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
+
+if __name__ == '__main__':
+    unittest.main()