Release of Version 4.12

Fix GitHub Actions Node.js deprecation warnings.

Use actions/setup-python cache where appropriate.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,6 +1,7 @@
 ---
-name: Bug report
-about: Create a report to help us improve
+name: 🐛 Bug report
+about: Create a report to help us improve 🤕
+labels: bug
 
 ---
 
@@ -12,7 +13,7 @@ Steps to reproduce the behavior:
 (e.g. Instaloader command line)
 
 **Expected behavior**
-A clear and concise description of what you expected to happen (if not obvious).
+A clear and concise description of what you expected to happen (even if it seems obvious).
 
 **Error messages and tracebacks**
 If applicable, add error messages and tracebacks to help explain your problem.
@@ -22,3 +23,10 @@ If applicable, add error messages and tracebacks to help explain your problem.
 
 **Additional context**
 Add any other context about the problem here.
+
+<!--
+Also see https://instaloader.github.io/contributing.html for how to report a bug.
+
+Please understand that a bug issue that lacks the required information cannot be
+processed.
+-->

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Contributing to Instaloader
+    url: https://instaloader.github.io/contributing.html
+    about: Guidelines for how to report an issue.
+  - name: Instaloader Troubleshooting
+    url: https://instaloader.github.io/troubleshooting.html
+    about: Many common problems and their workarounds are described here.
+  - name: Instaloader Documentation
+    url: https://instaloader.github.io/
+    about: Many questions might be already answered here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,17 +1,31 @@
 ---
-name: Feature request
-about: Suggest an idea for this project
+name: 🚀 Feature Suggestion
+about: Suggest an idea for this project 💡
+labels: "feature suggestion"
 
 ---
 
 **Provide us a use case of the feature**
-How could the user invoke the new function? Which problem would it solve? If new information is obtained, how would it be further processed?
+How could the user invoke the new function? Which problem would it solve?
+If new information is obtained, how would it be further processed?
 
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.
 
 **Describe alternatives you've considered**
 A clear and concise description of any alternative solutions or features you've considered.
+How do you achieve the here-described goal without the feature you are requesting?
+
+**If the feature request is accepted, would you be willing to submit a pull request?**
+Yes / No
+<!-- Help can be provided if you need assistance submitting a PR -->
 
 **Additional context**
 Add any other context about the feature request here.
+
+<!-- 
+Also see https://instaloader.github.io/contributing.html for how to suggest a feature.
+
+Please understand that a feature suggestion issue that lacks the required information cannot be
+processed.
+-->

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,18 @@
+---
+name: ❓ Question
+about: If you have a question about how to use Instaloader that's not well covered by our documentation
+labels: question
+
+---
+
+Your question here...
+
+<!--
+People will be better able and more motivated to provide help if your question
+
+ - has a concise and meaningful title,
+ - is not covered by the documentation: https://instaloader.github.io/,
+ - is actually related to Instaloader (this is not a Python help forum),
+ - has not already been answered here or on other sites such as Stack Overflow: https://stackoverflow.com/questions/tagged/instaloader,
+ - is written in a polite and welcoming tone.
+-->

.github/funding.yml

@@ -0,0 +1,2 @@
+github: [aandergr]
+custom: "https://instaloader.github.io/sponsors.html"

.github/pull_request_template.md

@@ -0,0 +1,18 @@
+
+
+<!--
+Thanks for your willingness to contribute to Instaloader! Please briefly
+describe
+
+- A motivation for this change, e.g.
+  - Fixes # .
+  - More general: What problem does the pull request solve?
+
+- The changes proposed in this pull request
+
+- The completeness of this change
+  - Is it just a proof of concept?
+  - Is the documentation updated (if appropriate)?
+  - Do you consider it ready to be merged or is it a draft?
+  - Can we help you at some point?
+-->

.github/workflows/documentation.yml

@@ -0,0 +1,32 @@
+name: "Documentation"
+
+on: [push, pull_request]
+
+jobs:
+  docs:
+    name: "Documentation"
+    runs-on: ubuntu-latest
+    steps:
+      - name: "Checkout Instaloader Repository"
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # needed for building docs
+      - name: "Setup Python"
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.12
+          cache: 'pipenv'
+      - name: "Install Dependencies"
+        run: |
+          python -m pip install pipenv==2023.12.1
+          pipenv --python `python --version | grep -Eo '3\.[0-9]+'` sync --dev
+      - name: "Build Documentation"
+        run: pipenv --python `python --version | grep -Eo '3\.[0-9]+'` run make -C docs html SPHINXOPTS="-W -n"
+      - name: "Deploy Documentation"
+        if: github.event_name == 'push' && github.ref == 'refs/heads/master'
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          branch: master
+          repository-name: instaloader/instaloader.github.io
+          folder: docs/_build/html
+          token: ${{ secrets.PAGES_DEPLOYMENT_TOKEN }}

.github/workflows/lint.yml

@@ -0,0 +1,28 @@
+name: PyLint and MyPy
+
+on: [push, pull_request]
+
+jobs:
+  lint:
+    name: PyLint and MyPy
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
+    steps:
+      - name: Checkout Instaloader Repository
+        uses: actions/checkout@v4
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: 'pipenv'
+      - name: Install Dependencies
+        run: |
+          python -m pip install pipenv==2023.12.1
+          pipenv --python `python --version | grep -Eo '3\.[0-9]+'` sync --dev
+      - name: PyLint
+        run: pipenv --python `python --version | grep -Eo '3\.[0-9]+'` run pylint instaloader
+      - name: MyPy
+        run: pipenv --python `python --version | grep -Eo '3\.[0-9]+'` run mypy -m instaloader

.github/workflows/packages.yml

@@ -0,0 +1,38 @@
+name: "Publish Packages"
+
+on:
+  push:
+    tags:
+      - v*
+
+jobs:
+  publish:
+    name: "Publish Package"
+    runs-on: ubuntu-latest
+    steps:
+      - name: "Checkout Repository"
+        uses: actions/checkout@v4
+      - name: "Setup Python"
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.12
+      - name: "Get the tagged version"
+        id: get_version
+        env:
+          GITHUB_REF: ${{ github.ref }}
+        run: echo "VERSION=${GITHUB_REF/refs\/tags\/v/}" >> $GITHUB_OUTPUT
+        shell: bash
+      - name: "Install Dependencies"
+        run: python -m pip install setuptools
+      - name: "Generate Python Package"
+        run: python setup.py sdist
+      - name: "Publish Distribution to PyPI"
+        uses: pypa/gh-action-pypi-publish@master
+        with:
+          password: ${{ secrets.PYPI_PASSWORD }}
+      - name: "Publish Distribution to AUR"
+        run: deploy/arch/deploy.sh $VERSION_TAG
+        env:
+          VERSION_TAG: v${{ steps.get_version.outputs.VERSION }}
+          AUR_KEY: ${{ secrets.AUR_KEY }}
+          AUR_IV: ${{ secrets.AUR_IV }}

.github/workflows/stale.yml

@@ -0,0 +1,34 @@
+name: Mark stale issues and pull requests
+
+on:
+  schedule:
+  - cron: "55 7 * * *"
+
+jobs:
+  stale:
+
+    name: Mark stale issues and pull requests
+    runs-on: ubuntu-latest
+    
+    steps:
+    - uses: actions/stale@v9
+      with:
+        repo-token: ${{ secrets.GITHUB_TOKEN }}
+        only-labels: 'question'
+        stale-issue-message: 'There has been no activity on this question for an extended period of time. This issue will be closed after further 14 days of inactivity.'
+        stale-issue-label: 'stale'
+        exempt-issue-labels: 'leave open'
+        days-before-stale: 21
+        days-before-close: -1
+        remove-stale-when-updated: false
+    - uses: actions/stale@v9
+      with:
+        repo-token: ${{ secrets.GITHUB_TOKEN }}
+        stale-issue-message: 'There has been no activity on this issue for an extended period of time. This issue will be closed after further 14 days of inactivity.'
+        stale-pr-message: 'There has been no activity on this Pull Request for an extended period of time. This Pull Request will be closed after further 14 days of inactivity.'
+        stale-issue-label: 'stale'
+        stale-pr-label: 'stale'
+        exempt-issue-labels: 'leave open'
+        exempt-pr-labels: 'leave open'
+        days-before-stale: 189
+        days-before-close: 14

.github/workflows/windows_exe.yml

@@ -0,0 +1,49 @@
+name: Windows EXE build
+
+on: [push, pull_request]
+
+jobs:
+  build:
+    name: Windows EXE build
+    runs-on: windows-latest
+    steps:
+      - name: Checkout Instaloader repository
+        uses: actions/checkout@v4
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: 'pipenv'
+          architecture: x64
+      - name: Get the tagged version
+        id: get_version
+        env:
+          GITHUB_REF: ${{ github.ref }}
+        run: echo "VERSION=${GITHUB_REF/refs\/tags\/v/}" >> $GITHUB_OUTPUT
+        shell: bash
+      - name: Run custom python script for EXE creation
+        env:
+          VERSION_TAG: v${{ steps.get_version.outputs.VERSION }}
+        run: python deploy/windows/create_exe.py
+      - name: Create draft release
+        if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+        id: create_release
+        uses: actions/create-release@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          tag_name: v${{ steps.get_version.outputs.VERSION }}
+          release_name: Version ${{ steps.get_version.outputs.VERSION }}
+          draft: true
+          prerelease: contains(steps.get_version.outputs.VERSION, 'a') || contains(steps.get_version.outputs.VERSION, 'b') || contains(steps.get_version.outputs.VERSION, 'c')
+      - name: Upload EXE as release asset
+        if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+        id: upload-release-asset
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: ./instaloader-v${{ steps.get_version.outputs.VERSION }}-windows-standalone.zip
+          asset_name: instaloader-v${{ steps.get_version.outputs.VERSION }}-windows-standalone.zip
+          asset_content_type: application/zip

.gitignore

@@ -1 +1,2 @@
 __pycache__/
+.mypy_cache/

.pylintrc

@@ -0,0 +1,646 @@
+[MAIN]
+
+# Analyse import fallback blocks. This can be used to support both Python 2 and
+# 3 compatible code, which means that the block might have code that exists
+# only in one or another interpreter, leading to false positives when analysed.
+analyse-fallback-blocks=no
+
+# Clear in-memory caches upon conclusion of linting. Useful if running pylint
+# in a server-like mode.
+clear-cache-post-run=no
+
+# Load and enable all available extensions. Use --list-extensions to see a list
+# all available extensions.
+#enable-all-extensions=
+
+# In error mode, messages with a category besides ERROR or FATAL are
+# suppressed, and no reports are done by default. Error mode is compatible with
+# disabling specific errors.
+#errors-only=
+
+# Always return a 0 (non-error) status code, even if lint errors are found.
+# This is primarily useful in continuous integration scripts.
+#exit-zero=
+
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code.
+extension-pkg-allow-list=
+
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
+# for backward compatibility.)
+extension-pkg-whitelist=
+
+# Return non-zero exit code if any of these messages/categories are detected,
+# even if score is above --fail-under value. Syntax same as enable. Messages
+# specified are enabled, while categories only check already-enabled messages.
+fail-on=
+
+# Specify a score threshold under which the program will exit with error.
+fail-under=10
+
+# Interpret the stdin as a python script, whose filename needs to be passed as
+# the module_or_package argument.
+#from-stdin=
+
+# Files or directories to be skipped. They should be base names, not paths.
+ignore=CVS
+
+# Add files or directories matching the regular expressions patterns to the
+# ignore-list. The regex matches against paths and can be in Posix or Windows
+# format. Because '\\' represents the directory delimiter on Windows systems,
+# it can't be used as an escape character.
+ignore-paths=
+
+# Files or directories matching the regular expression patterns are skipped.
+# The regex matches against base names, not paths. The default value ignores
+# Emacs file locks
+ignore-patterns=^\.#
+
+# List of module names for which member attributes should not be checked
+# (useful for modules/projects where namespaces are manipulated during runtime
+# and thus existing member attributes cannot be deduced by static analysis). It
+# supports qualified module names, as well as Unix pattern matching.
+ignored-modules=
+
+# Python code to execute, usually for sys.path manipulation such as
+# pygtk.require().
+#init-hook=
+
+# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
+# number of processors available to use, and will cap the count on Windows to
+# avoid hangs.
+jobs=1
+
+# Control the amount of potential inferred values when inferring a single
+# object. This can help the performance when dealing with large functions or
+# complex, nested conditions.
+limit-inference-results=100
+
+# List of plugins (as comma separated values of python module names) to load,
+# usually to register additional checkers.
+load-plugins=
+
+# Pickle collected data for later comparisons.
+persistent=yes
+
+# Minimum Python version to use for version dependent checks. Will default to
+# the version used to run pylint.
+py-version=3.8
+
+# Discover python modules and packages in the file system subtree.
+recursive=no
+
+# When enabled, pylint would attempt to guess common misconfiguration and emit
+# user-friendly hints instead of false-positive error messages.
+suggestion-mode=yes
+
+# Allow loading of arbitrary C extensions. Extensions are imported into the
+# active Python interpreter and may run arbitrary code.
+unsafe-load-any-extension=no
+
+# In verbose mode, extra non-checker-related info will be displayed.
+#verbose=
+
+
+[BASIC]
+
+# Naming style matching correct argument names.
+argument-naming-style=snake_case
+
+# Regular expression matching correct argument names. Overrides argument-
+# naming-style. If left empty, argument names will be checked with the set
+# naming style.
+#argument-rgx=
+
+# Naming style matching correct attribute names.
+attr-naming-style=snake_case
+
+# Regular expression matching correct attribute names. Overrides attr-naming-
+# style. If left empty, attribute names will be checked with the set naming
+# style.
+#attr-rgx=
+
+# Bad variable names which should always be refused, separated by a comma.
+bad-names=foo,
+          bar,
+          baz,
+          toto,
+          tutu,
+          tata
+
+# Bad variable names regexes, separated by a comma. If names match any regex,
+# they will always be refused
+bad-names-rgxs=
+
+# Naming style matching correct class attribute names.
+class-attribute-naming-style=any
+
+# Regular expression matching correct class attribute names. Overrides class-
+# attribute-naming-style. If left empty, class attribute names will be checked
+# with the set naming style.
+#class-attribute-rgx=
+
+# Naming style matching correct class constant names.
+class-const-naming-style=UPPER_CASE
+
+# Regular expression matching correct class constant names. Overrides class-
+# const-naming-style. If left empty, class constant names will be checked with
+# the set naming style.
+#class-const-rgx=
+
+# Naming style matching correct class names.
+class-naming-style=PascalCase
+
+# Regular expression matching correct class names. Overrides class-naming-
+# style. If left empty, class names will be checked with the set naming style.
+#class-rgx=
+
+# Naming style matching correct constant names.
+const-naming-style=UPPER_CASE
+
+# Regular expression matching correct constant names. Overrides const-naming-
+# style. If left empty, constant names will be checked with the set naming
+# style.
+#const-rgx=
+
+# Minimum line length for functions/classes that require docstrings, shorter
+# ones are exempt.
+docstring-min-length=-1
+
+# Naming style matching correct function names.
+function-naming-style=snake_case
+
+# Regular expression matching correct function names. Overrides function-
+# naming-style. If left empty, function names will be checked with the set
+# naming style.
+#function-rgx=
+
+# Good variable names which should always be accepted, separated by a comma.
+good-names=i,
+           j,
+           k,
+           ex,
+           Run,
+           _
+
+# Good variable names regexes, separated by a comma. If names match any regex,
+# they will always be accepted
+good-names-rgxs=
+
+# Include a hint for the correct naming format with invalid-name.
+include-naming-hint=no
+
+# Naming style matching correct inline iteration names.
+inlinevar-naming-style=any
+
+# Regular expression matching correct inline iteration names. Overrides
+# inlinevar-naming-style. If left empty, inline iteration names will be checked
+# with the set naming style.
+#inlinevar-rgx=
+
+# Naming style matching correct method names.
+method-naming-style=snake_case
+
+# Regular expression matching correct method names. Overrides method-naming-
+# style. If left empty, method names will be checked with the set naming style.
+#method-rgx=
+
+# Naming style matching correct module names.
+module-naming-style=snake_case
+
+# Regular expression matching correct module names. Overrides module-naming-
+# style. If left empty, module names will be checked with the set naming style.
+#module-rgx=
+
+# Colon-delimited sets of names that determine each other's naming style when
+# the name regexes allow several styles.
+name-group=
+
+# Regular expression which should only match function or class names that do
+# not require a docstring.
+no-docstring-rgx=^_
+
+# List of decorators that produce properties, such as abc.abstractproperty. Add
+# to this list to register other decorators that produce valid properties.
+# These decorators are taken in consideration only for invalid-name.
+property-classes=abc.abstractproperty
+
+# Regular expression matching correct type variable names. If left empty, type
+# variable names will be checked with the set naming style.
+#typevar-rgx=
+
+# Naming style matching correct variable names.
+variable-naming-style=snake_case
+
+# Regular expression matching correct variable names. Overrides variable-
+# naming-style. If left empty, variable names will be checked with the set
+# naming style.
+#variable-rgx=
+
+
+[CLASSES]
+
+# Warn about protected attribute access inside special methods
+check-protected-access-in-special-methods=no
+
+# List of method names used to declare (i.e. assign) instance attributes.
+defining-attr-methods=__init__,
+                      __new__,
+                      setUp,
+                      __post_init__
+
+# List of member names, which should be excluded from the protected access
+# warning.
+exclude-protected=_asdict,
+                  _fields,
+                  _replace,
+                  _source,
+                  _make
+
+# List of valid names for the first argument in a class method.
+valid-classmethod-first-arg=cls
+
+# List of valid names for the first argument in a metaclass class method.
+valid-metaclass-classmethod-first-arg=mcs
+
+
+[DESIGN]
+
+# List of regular expressions of class ancestor names to ignore when counting
+# public methods (see R0903)
+exclude-too-few-public-methods=
+
+# List of qualified class names to ignore when counting class parents (see
+# R0901)
+ignored-parents=
+
+# Maximum number of arguments for function / method.
+max-args=5
+
+# Maximum number of attributes for a class (see R0902).
+max-attributes=7
+
+# Maximum number of boolean expressions in an if statement (see R0916).
+max-bool-expr=5
+
+# Maximum number of branch for function / method body.
+max-branches=12
+
+# Maximum number of locals for function / method body.
+max-locals=15
+
+# Maximum number of parents for a class (see R0901).
+max-parents=7
+
+# Maximum number of public methods for a class (see R0904).
+max-public-methods=20
+
+# Maximum number of return / yield for function / method body.
+max-returns=6
+
+# Maximum number of statements in function / method body.
+max-statements=50
+
+# Minimum number of public methods for a class (see R0903).
+min-public-methods=2
+
+
+[EXCEPTIONS]
+
+# Exceptions that will emit a warning when caught.
+overgeneral-exceptions=builtins.BaseException,builtins.Exception
+
+
+[FORMAT]
+
+# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
+expected-line-ending-format=
+
+# Regexp for a line that is allowed to be longer than the limit.
+ignore-long-lines=^\s*(# )?<?https?://\S+>?$
+
+# Number of spaces of indent required inside a hanging or continued line.
+indent-after-paren=4
+
+# String used as indentation unit. This is usually "    " (4 spaces) or "\t" (1
+# tab).
+indent-string='    '
+
+# Maximum number of characters on a single line.
+max-line-length=120
+
+# Maximum number of lines in a module.
+max-module-lines=1000
+
+# Allow the body of a class to be on the same line as the declaration if body
+# contains single statement.
+single-line-class-stmt=no
+
+# Allow the body of an if to be on the same line as the test if there is no
+# else.
+single-line-if-stmt=no
+
+
+[IMPORTS]
+
+# List of modules that can be imported at any level, not just the top level
+# one.
+allow-any-import-level=
+
+# Allow explicit reexports by alias from a package __init__.
+allow-reexport-from-package=no
+
+# Allow wildcard imports from modules that define __all__.
+allow-wildcard-with-all=no
+
+# Deprecated modules which should not be used, separated by a comma.
+deprecated-modules=
+
+# Output a graph (.gv or any supported image format) of external dependencies
+# to the given file (report RP0402 must not be disabled).
+ext-import-graph=
+
+# Output a graph (.gv or any supported image format) of all (i.e. internal and
+# external) dependencies to the given file (report RP0402 must not be
+# disabled).
+import-graph=
+
+# Output a graph (.gv or any supported image format) of internal dependencies
+# to the given file (report RP0402 must not be disabled).
+int-import-graph=
+
+# Force import order to recognize a module as part of the standard
+# compatibility libraries.
+known-standard-library=
+
+# Force import order to recognize a module as part of a third party library.
+known-third-party=enchant
+
+# Couples of modules and preferred modules, separated by a comma.
+preferred-modules=
+
+
+[LOGGING]
+
+# The type of string formatting that logging methods do. `old` means using %
+# formatting, `new` is for `{}` formatting.
+logging-format-style=old
+
+# Logging modules to check that the string format arguments are in logging
+# function parameter format.
+logging-modules=logging
+
+
+[MESSAGES CONTROL]
+
+# Only show warnings with the listed confidence levels. Leave empty to show
+# all. Valid levels: HIGH, CONTROL_FLOW, INFERENCE, INFERENCE_FAILURE,
+# UNDEFINED.
+confidence=HIGH,
+           CONTROL_FLOW,
+           INFERENCE,
+           INFERENCE_FAILURE,
+           UNDEFINED
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once). You can also use "--disable=all" to
+# disable everything first and then re-enable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use "--disable=all --enable=classes
+# --disable=W".
+disable=bad-inline-option,
+        consider-using-f-string,
+        cyclic-import,
+        deprecated-pragma,
+        file-ignored,
+        invalid-name,
+        locally-disabled,
+        missing-class-docstring,
+        missing-function-docstring,
+        missing-module-docstring,
+        no-else-raise,
+        no-else-return,
+        raw-checker-failed,
+        suppressed-message,
+        too-many-arguments,
+        too-many-branches,
+        too-many-instance-attributes,
+        too-many-lines,
+        too-many-locals,
+        too-many-public-methods,
+        too-many-statements,
+        unnecessary-lambda-assignment,
+        unnecessary-pass,
+        unspecified-encoding,
+        unused-wildcard-import,
+        use-dict-literal,
+        use-list-literal,
+        use-symbolic-message-instead,
+        useless-suppression,
+        wildcard-import
+
+# Enable the message, report, category or checker with the given id(s). You can
+# either give multiple identifier separated by comma (,) or put this option
+# multiple time (only on the command line, not in the configuration file where
+# it should appear only once). See also the "--disable" option for examples.
+enable=c-extension-no-member
+
+
+[METHOD_ARGS]
+
+# List of qualified names (i.e., library.method) which require a timeout
+# parameter e.g. 'requests.api.get,requests.api.post'
+timeout-methods=requests.api.delete,requests.api.get,requests.api.head,requests.api.options,requests.api.patch,requests.api.post,requests.api.put,requests.api.request
+
+
+[MISCELLANEOUS]
+
+# List of note tags to take in consideration, separated by a comma.
+notes=FIXME,
+      XXX,
+      TODO
+
+# Regular expression of note tags to take in consideration.
+notes-rgx=
+
+
+[REFACTORING]
+
+# Maximum number of nested blocks for function / method body
+max-nested-blocks=5
+
+# Complete name of functions that never returns. When checking for
+# inconsistent-return-statements if a never returning function is called then
+# it will be considered as an explicit return statement and no message will be
+# printed.
+never-returning-functions=sys.exit,argparse.parse_error
+
+
+[REPORTS]
+
+# Python expression which should return a score less than or equal to 10. You
+# have access to the variables 'fatal', 'error', 'warning', 'refactor',
+# 'convention', and 'info' which contain the number of messages in each
+# category, as well as 'statement' which is the total number of statements
+# analyzed. This score is used by the global evaluation report (RP0004).
+evaluation=max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))
+
+# Template used to display messages. This is a python new-style format string
+# used to format the message information. See doc for all details.
+msg-template=
+
+# Set the output format. Available formats are text, parseable, colorized, json
+# and msvs (visual studio). You can also give a reporter class, e.g.
+# mypackage.mymodule.MyReporterClass.
+#output-format=
+
+# Tells whether to display a full report or only the messages.
+reports=no
+
+# Activate the evaluation score.
+score=yes
+
+
+[SIMILARITIES]
+
+# Comments are removed from the similarity computation
+ignore-comments=yes
+
+# Docstrings are removed from the similarity computation
+ignore-docstrings=yes
+
+# Imports are removed from the similarity computation
+ignore-imports=yes
+
+# Signatures are removed from the similarity computation
+ignore-signatures=yes
+
+# Minimum lines number of a similarity.
+min-similarity-lines=4
+
+
+[SPELLING]
+
+# Limits count of emitted suggestions for spelling mistakes.
+max-spelling-suggestions=4
+
+# Spelling dictionary name. Available dictionaries: none. To make it work,
+# install the 'python-enchant' package.
+spelling-dict=
+
+# List of comma separated words that should be considered directives if they
+# appear at the beginning of a comment and should not be checked.
+spelling-ignore-comment-directives=fmt: on,fmt: off,noqa:,noqa,nosec,isort:skip,mypy:
+
+# List of comma separated words that should not be checked.
+spelling-ignore-words=
+
+# A path to a file that contains the private dictionary; one word per line.
+spelling-private-dict-file=
+
+# Tells whether to store unknown words to the private dictionary (see the
+# --spelling-private-dict-file option) instead of raising a message.
+spelling-store-unknown-words=no
+
+
+[STRING]
+
+# This flag controls whether inconsistent-quotes generates a warning when the
+# character used as a quote delimiter is used inconsistently within a module.
+check-quote-consistency=no
+
+# This flag controls whether the implicit-str-concat should generate a warning
+# on implicit string concatenation in sequences defined over several lines.
+check-str-concat-over-line-jumps=no
+
+
+[TYPECHECK]
+
+# List of decorators that produce context managers, such as
+# contextlib.contextmanager. Add to this list to register other decorators that
+# produce valid context managers.
+contextmanager-decorators=contextlib.contextmanager
+
+# List of members which are set dynamically and missed by pylint inference
+# system, and so shouldn't trigger E1101 when accessed. Python regular
+# expressions are accepted.
+generated-members=
+
+# Tells whether to warn about missing members when the owner of the attribute
+# is inferred to be None.
+ignore-none=yes
+
+# This flag controls whether pylint should warn about no-member and similar
+# checks whenever an opaque object is returned when inferring. The inference
+# can return multiple potential results while evaluating a Python object, but
+# some branches might not be evaluated, which results in partial inference. In
+# that case, it might be useful to still emit no-member and other checks for
+# the rest of the inferred objects.
+ignore-on-opaque-inference=yes
+
+# List of symbolic message names to ignore for Mixin members.
+ignored-checks-for-mixins=no-member,
+                          not-async-context-manager,
+                          not-context-manager,
+                          attribute-defined-outside-init
+
+# List of class names for which member attributes should not be checked (useful
+# for classes with dynamically set attributes). This supports the use of
+# qualified names.
+ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
+
+# Show a hint with possible names when a member name was not found. The aspect
+# of finding the hint is based on edit distance.
+missing-member-hint=yes
+
+# The minimum edit distance a name should have in order to be considered a
+# similar match for a missing member name.
+missing-member-hint-distance=1
+
+# The total number of similar names that should be taken in consideration when
+# showing a hint for a missing member.
+missing-member-max-choices=1
+
+# Regex pattern to define which classes are considered mixins.
+mixin-class-rgx=.*[Mm]ixin
+
+# List of decorators that change the signature of a decorated function.
+signature-mutators=
+
+
+[VARIABLES]
+
+# List of additional names supposed to be defined in builtins. Remember that
+# you should avoid defining new builtins when possible.
+additional-builtins=
+
+# Tells whether unused global variables should be treated as a violation.
+allow-global-unused-variables=yes
+
+# List of names allowed to shadow builtins
+allowed-redefined-builtins=
+
+# List of strings which can identify a callback function by name. A callback
+# name must start or end with one of those strings.
+callbacks=cb_,
+          _cb
+
+# A regular expression matching the name of dummy variables (i.e. expected to
+# not be used).
+dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
+
+# Argument names that match this expression will be ignored.
+ignored-argument-names=_.*|^ignored_|^unused_
+
+# Tells whether we should check for unused import in __init__ files.
+init-import=no
+
+# List of qualified module names which can have objects that can redefine
+# builtins.
+redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io

.travis.yml

@@ -1,29 +0,0 @@
-language: python
-python:
- - "3.5"
- - "3.5-dev"
- - "3.6"
- - "3.6-dev"
-install:
- - pip install pylint requests
- - pip install -r docs/requirements.txt
-script:
- - python3 -m pylint -r n -d bad-whitespace,missing-docstring,too-many-arguments,locally-disabled,line-too-long,too-many-public-methods,too-many-lines,too-many-instance-attributes,too-many-locals,too-many-branches,too-many-statements,inconsistent-return-statements,invalid-name,wildcard-import,unused-wildcard-import,no-else-return instaloader
- - make -C docs html SPHINXOPTS="-W -n"
-deploy:
-  - provider: pypi
-    user: aandergr
-    on:
-      tags: true
-      python: 3.6
-    password:
-      secure: KBsATtXNnEzqr0X2ALdyx2zaYj2EUfK8YjP4jEOilBELpJ4evHYXVgRkn9CKXGsD7LKbWmR3gz2rr8NZ6U42pHw0fpQHizz91MpJSKbNY7RZZh4UVWMDh81izeglchWAF+m3Wdpxpo/GBDbrZIbRQFhPgB+kjSzJo3uiCS+yVXtGG+6csJHzd0WkJjsVHz+XSdxyOiLy/qFF3nmY6ZrVrgw6WbX2YLbx9je6DxK3YRrnBqgMgH7Ui2xWWbgJCk3KKpQTMiMKnTa9mdeMJAd26YMCz1gG/X7ck6VNly/6bs5+Zq7/bU0dSolhPBhNthyccYdS0IeGPdPF3YqatblJ4fLGTfHn8oJs3/T11yIW9pvOT3UftEynPOUX+en2Qnb/3WfgwUfVts8i8DAYgEFLGEzbF5XMGy4gwAD/bZ0JW0pmgAG2PZsdOokAt+ydlNj/uxq1GFW2doZbul6k4t03p1mik6itYscvuhKoEm8Yw6wr1P9aDvMR6FFpSu4gDt2XrKmRmWkSXxpNeCfZujTjyAnISxLVQGSY7BG5K0Z2HbDtqqNz9nj4WgNGzZmngiRofCevWR+vPIDUDW7NCX03csfcFS9RRllBO/BrL7NVUV+Qp9dfTQYp+8SqYLWgHtDoXdgyMREjJaAmbx2u/TYTY8bADZ5F3vGUpv2j5hP3NTU=
-  - provider: pages
-    skip-cleanup: true
-    github-token: $GITHUB_TOKEN
-    local-dir: docs/_build/html
-    repo: instaloader/instaloader.github.io
-    target-branch: master
-    on:
-      branch: master
-      python: 3.6

AUTHORS.md

@@ -6,3 +6,5 @@ Instaloader is written by
 - Alexander Graf (@aandergr)
 - André Koch-Kramer (@Thammus)
 - Lars Lindqvist (@e5150)
+- Eduardo M Kalinowksi (@ekalin)
+- ... and many more, see https://github.com/instaloader/instaloader/graphs/contributors

CODE_OF_CONDUCT.md

@@ -0,0 +1,135 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement by opening an
+issue or contacting one or more of the project maintainers.
+
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
+
+Community Impact Guidelines were inspired by 
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available 
+at [https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations
+

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2016-2018 Alexander Graf and André Koch-Kramer.
+Copyright (c) 2016-2020 Alexander Graf and André Koch-Kramer.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

Pipfile

@@ -0,0 +1,22 @@
+[[source]]
+name = "pypi"
+url = "https://pypi.org/simple"
+verify_ssl = true
+
+[dev-packages]
+pylint = "*"
+mypy = "*"
+sphinx = "*"
+pyinstaller = "*"
+psutil = "*"
+types-requests = "*"
+sphinx-autodoc-typehints = "*"
+sphinxcontrib-jquery = "*"
+tomli = "*"           # Needed for Pylint for "python_version < '3.11'"
+dill = "*"            # Needed for Pylint for "python_version < '3.11'"
+pywin32-ctypes = "*"  # Needed for pyinstaller on Windows
+pefile = "*"          # Needed for pyinstaller on Windows
+
+[packages]
+requests = "*"
+browser-cookie3 = "*"

Pipfile.lock

@@ -0,0 +1,787 @@
+{
+    "_meta": {
+        "hash": {
+            "sha256": "7a7000ad88c2f698c6e5f6b3eaaf24b66321e6d1743a8e088bae609626a02133"
+        },
+        "pipfile-spec": 6,
+        "requires": {},
+        "sources": [
+            {
+                "name": "pypi",
+                "url": "https://pypi.org/simple",
+                "verify_ssl": true
+            }
+        ]
+    },
+    "default": {
+        "browser-cookie3": {
+            "hashes": [
+                "sha256:114e40fe80853211d65f14bc690a7d90c4c287251593dc5767692a950a9006d4",
+                "sha256:3031ad14b96b47ef1e4c8545f2f463e10ad844ef834dcd0ebdae361e31c6119a"
+            ],
+            "index": "pypi",
+            "version": "==0.19.1"
+        },
+        "certifi": {
+            "hashes": [
+                "sha256:0569859f95fc761b18b45ef421b1290a0f65f147e92a1e5eb3e635f9a5e4e66f",
+                "sha256:dc383c07b76109f368f6106eee2b593b04a011ea4d55f652c6ca24a754d1cdd1"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==2024.2.2"
+        },
+        "charset-normalizer": {
+            "hashes": [
+                "sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027",
+                "sha256:06a81e93cd441c56a9b65d8e1d043daeb97a3d0856d177d5c90ba85acb3db087",
+                "sha256:0a55554a2fa0d408816b3b5cedf0045f4b8e1a6065aec45849de2d6f3f8e9786",
+                "sha256:0b2b64d2bb6d3fb9112bafa732def486049e63de9618b5843bcdd081d8144cd8",
+                "sha256:10955842570876604d404661fbccbc9c7e684caf432c09c715ec38fbae45ae09",
+                "sha256:122c7fa62b130ed55f8f285bfd56d5f4b4a5b503609d181f9ad85e55c89f4185",
+                "sha256:1ceae2f17a9c33cb48e3263960dc5fc8005351ee19db217e9b1bb15d28c02574",
+                "sha256:1d3193f4a680c64b4b6a9115943538edb896edc190f0b222e73761716519268e",
+                "sha256:1f79682fbe303db92bc2b1136016a38a42e835d932bab5b3b1bfcfbf0640e519",
+                "sha256:2127566c664442652f024c837091890cb1942c30937add288223dc895793f898",
+                "sha256:22afcb9f253dac0696b5a4be4a1c0f8762f8239e21b99680099abd9b2b1b2269",
+                "sha256:25baf083bf6f6b341f4121c2f3c548875ee6f5339300e08be3f2b2ba1721cdd3",
+                "sha256:2e81c7b9c8979ce92ed306c249d46894776a909505d8f5a4ba55b14206e3222f",
+                "sha256:3287761bc4ee9e33561a7e058c72ac0938c4f57fe49a09eae428fd88aafe7bb6",
+                "sha256:34d1c8da1e78d2e001f363791c98a272bb734000fcef47a491c1e3b0505657a8",
+                "sha256:37e55c8e51c236f95b033f6fb391d7d7970ba5fe7ff453dad675e88cf303377a",
+                "sha256:3d47fa203a7bd9c5b6cee4736ee84ca03b8ef23193c0d1ca99b5089f72645c73",
+                "sha256:3e4d1f6587322d2788836a99c69062fbb091331ec940e02d12d179c1d53e25fc",
+                "sha256:42cb296636fcc8b0644486d15c12376cb9fa75443e00fb25de0b8602e64c1714",
+                "sha256:45485e01ff4d3630ec0d9617310448a8702f70e9c01906b0d0118bdf9d124cf2",
+                "sha256:4a78b2b446bd7c934f5dcedc588903fb2f5eec172f3d29e52a9096a43722adfc",
+                "sha256:4ab2fe47fae9e0f9dee8c04187ce5d09f48eabe611be8259444906793ab7cbce",
+                "sha256:4d0d1650369165a14e14e1e47b372cfcb31d6ab44e6e33cb2d4e57265290044d",
+                "sha256:549a3a73da901d5bc3ce8d24e0600d1fa85524c10287f6004fbab87672bf3e1e",
+                "sha256:55086ee1064215781fff39a1af09518bc9255b50d6333f2e4c74ca09fac6a8f6",
+                "sha256:572c3763a264ba47b3cf708a44ce965d98555f618ca42c926a9c1616d8f34269",
+                "sha256:573f6eac48f4769d667c4442081b1794f52919e7edada77495aaed9236d13a96",
+                "sha256:5b4c145409bef602a690e7cfad0a15a55c13320ff7a3ad7ca59c13bb8ba4d45d",
+                "sha256:6463effa3186ea09411d50efc7d85360b38d5f09b870c48e4600f63af490e56a",
+                "sha256:65f6f63034100ead094b8744b3b97965785388f308a64cf8d7c34f2f2e5be0c4",
+                "sha256:663946639d296df6a2bb2aa51b60a2454ca1cb29835324c640dafb5ff2131a77",
+                "sha256:6897af51655e3691ff853668779c7bad41579facacf5fd7253b0133308cf000d",
+                "sha256:68d1f8a9e9e37c1223b656399be5d6b448dea850bed7d0f87a8311f1ff3dabb0",
+                "sha256:6ac7ffc7ad6d040517be39eb591cac5ff87416c2537df6ba3cba3bae290c0fed",
+                "sha256:6b3251890fff30ee142c44144871185dbe13b11bab478a88887a639655be1068",
+                "sha256:6c4caeef8fa63d06bd437cd4bdcf3ffefe6738fb1b25951440d80dc7df8c03ac",
+                "sha256:6ef1d82a3af9d3eecdba2321dc1b3c238245d890843e040e41e470ffa64c3e25",
+                "sha256:753f10e867343b4511128c6ed8c82f7bec3bd026875576dfd88483c5c73b2fd8",
+                "sha256:7cd13a2e3ddeed6913a65e66e94b51d80a041145a026c27e6bb76c31a853c6ab",
+                "sha256:7ed9e526742851e8d5cc9e6cf41427dfc6068d4f5a3bb03659444b4cabf6bc26",
+                "sha256:7f04c839ed0b6b98b1a7501a002144b76c18fb1c1850c8b98d458ac269e26ed2",
+                "sha256:802fe99cca7457642125a8a88a084cef28ff0cf9407060f7b93dca5aa25480db",
+                "sha256:80402cd6ee291dcb72644d6eac93785fe2c8b9cb30893c1af5b8fdd753b9d40f",
+                "sha256:8465322196c8b4d7ab6d1e049e4c5cb460d0394da4a27d23cc242fbf0034b6b5",
+                "sha256:86216b5cee4b06df986d214f664305142d9c76df9b6512be2738aa72a2048f99",
+                "sha256:87d1351268731db79e0f8e745d92493ee2841c974128ef629dc518b937d9194c",
+                "sha256:8bdb58ff7ba23002a4c5808d608e4e6c687175724f54a5dade5fa8c67b604e4d",
+                "sha256:8c622a5fe39a48f78944a87d4fb8a53ee07344641b0562c540d840748571b811",
+                "sha256:8d756e44e94489e49571086ef83b2bb8ce311e730092d2c34ca8f7d925cb20aa",
+                "sha256:8f4a014bc36d3c57402e2977dada34f9c12300af536839dc38c0beab8878f38a",
+                "sha256:9063e24fdb1e498ab71cb7419e24622516c4a04476b17a2dab57e8baa30d6e03",
+                "sha256:90d558489962fd4918143277a773316e56c72da56ec7aa3dc3dbbe20fdfed15b",
+                "sha256:923c0c831b7cfcb071580d3f46c4baf50f174be571576556269530f4bbd79d04",
+                "sha256:95f2a5796329323b8f0512e09dbb7a1860c46a39da62ecb2324f116fa8fdc85c",
+                "sha256:96b02a3dc4381e5494fad39be677abcb5e6634bf7b4fa83a6dd3112607547001",
+                "sha256:9f96df6923e21816da7e0ad3fd47dd8f94b2a5ce594e00677c0013018b813458",
+                "sha256:a10af20b82360ab00827f916a6058451b723b4e65030c5a18577c8b2de5b3389",
+                "sha256:a50aebfa173e157099939b17f18600f72f84eed3049e743b68ad15bd69b6bf99",
+                "sha256:a981a536974bbc7a512cf44ed14938cf01030a99e9b3a06dd59578882f06f985",
+                "sha256:a9a8e9031d613fd2009c182b69c7b2c1ef8239a0efb1df3f7c8da66d5dd3d537",
+                "sha256:ae5f4161f18c61806f411a13b0310bea87f987c7d2ecdbdaad0e94eb2e404238",
+                "sha256:aed38f6e4fb3f5d6bf81bfa990a07806be9d83cf7bacef998ab1a9bd660a581f",
+                "sha256:b01b88d45a6fcb69667cd6d2f7a9aeb4bf53760d7fc536bf679ec94fe9f3ff3d",
+                "sha256:b261ccdec7821281dade748d088bb6e9b69e6d15b30652b74cbbac25e280b796",
+                "sha256:b2b0a0c0517616b6869869f8c581d4eb2dd83a4d79e0ebcb7d373ef9956aeb0a",
+                "sha256:b4a23f61ce87adf89be746c8a8974fe1c823c891d8f86eb218bb957c924bb143",
+                "sha256:bd8f7df7d12c2db9fab40bdd87a7c09b1530128315d047a086fa3ae3435cb3a8",
+                "sha256:beb58fe5cdb101e3a055192ac291b7a21e3b7ef4f67fa1d74e331a7f2124341c",
+                "sha256:c002b4ffc0be611f0d9da932eb0f704fe2602a9a949d1f738e4c34c75b0863d5",
+                "sha256:c083af607d2515612056a31f0a8d9e0fcb5876b7bfc0abad3ecd275bc4ebc2d5",
+                "sha256:c180f51afb394e165eafe4ac2936a14bee3eb10debc9d9e4db8958fe36afe711",
+                "sha256:c235ebd9baae02f1b77bcea61bce332cb4331dc3617d254df3323aa01ab47bd4",
+                "sha256:cd70574b12bb8a4d2aaa0094515df2463cb429d8536cfb6c7ce983246983e5a6",
+                "sha256:d0eccceffcb53201b5bfebb52600a5fb483a20b61da9dbc885f8b103cbe7598c",
+                "sha256:d965bba47ddeec8cd560687584e88cf699fd28f192ceb452d1d7ee807c5597b7",
+                "sha256:db364eca23f876da6f9e16c9da0df51aa4f104a972735574842618b8c6d999d4",
+                "sha256:ddbb2551d7e0102e7252db79ba445cdab71b26640817ab1e3e3648dad515003b",
+                "sha256:deb6be0ac38ece9ba87dea880e438f25ca3eddfac8b002a2ec3d9183a454e8ae",
+                "sha256:e06ed3eb3218bc64786f7db41917d4e686cc4856944f53d5bdf83a6884432e12",
+                "sha256:e27ad930a842b4c5eb8ac0016b0a54f5aebbe679340c26101df33424142c143c",
+                "sha256:e537484df0d8f426ce2afb2d0f8e1c3d0b114b83f8850e5f2fbea0e797bd82ae",
+                "sha256:eb00ed941194665c332bf8e078baf037d6c35d7c4f3102ea2d4f16ca94a26dc8",
+                "sha256:eb6904c354526e758fda7167b33005998fb68c46fbc10e013ca97f21ca5c8887",
+                "sha256:eb8821e09e916165e160797a6c17edda0679379a4be5c716c260e836e122f54b",
+                "sha256:efcb3f6676480691518c177e3b465bcddf57cea040302f9f4e6e191af91174d4",
+                "sha256:f27273b60488abe721a075bcca6d7f3964f9f6f067c8c4c605743023d7d3944f",
+                "sha256:f30c3cb33b24454a82faecaf01b19c18562b1e89558fb6c56de4d9118a032fd5",
+                "sha256:fb69256e180cb6c8a894fee62b3afebae785babc1ee98b81cdf68bbca1987f33",
+                "sha256:fd1abc0d89e30cc4e02e4064dc67fcc51bd941eb395c502aac3ec19fab46b519",
+                "sha256:ff8fa367d09b717b2a17a052544193ad76cd49979c805768879cb63d9ca50561"
+            ],
+            "markers": "python_full_version >= '3.7.0'",
+            "version": "==3.3.2"
+        },
+        "idna": {
+            "hashes": [
+                "sha256:9ecdbbd083b06798ae1e86adcbfe8ab1479cf864e4ee30fe4e46a003d12491ca",
+                "sha256:c05567e9c24a6b9faaa835c4821bad0590fbb9d5779e7caa6e1cc4978e7eb24f"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==3.6"
+        },
+        "jeepney": {
+            "hashes": [
+                "sha256:5efe48d255973902f6badc3ce55e2aa6c5c3b3bc642059ef3a91247bcfcc5806",
+                "sha256:c0a454ad016ca575060802ee4d590dd912e35c122fa04e70306de3d076cce755"
+            ],
+            "markers": "python_version >= '3.7' and 'bsd' in sys_platform or sys_platform == 'linux'",
+            "version": "==0.8.0"
+        },
+        "lz4": {
+            "hashes": [
+                "sha256:01fe674ef2889dbb9899d8a67361e0c4a2c833af5aeb37dd505727cf5d2a131e",
+                "sha256:054b4631a355606e99a42396f5db4d22046a3397ffc3269a348ec41eaebd69d2",
+                "sha256:0a136e44a16fc98b1abc404fbabf7f1fada2bdab6a7e970974fb81cf55b636d0",
+                "sha256:0e9c410b11a31dbdc94c05ac3c480cb4b222460faf9231f12538d0074e56c563",
+                "sha256:222a7e35137d7539c9c33bb53fcbb26510c5748779364014235afc62b0ec797f",
+                "sha256:24b3206de56b7a537eda3a8123c644a2b7bf111f0af53bc14bed90ce5562d1aa",
+                "sha256:2b901c7784caac9a1ded4555258207d9e9697e746cc8532129f150ffe1f6ba0d",
+                "sha256:2f7b1839f795315e480fb87d9bc60b186a98e3e5d17203c6e757611ef7dcef61",
+                "sha256:30e8c20b8857adef7be045c65f47ab1e2c4fabba86a9fa9a997d7674a31ea6b6",
+                "sha256:31ea4be9d0059c00b2572d700bf2c1bc82f241f2c3282034a759c9a4d6ca4dc2",
+                "sha256:337cb94488a1b060ef1685187d6ad4ba8bc61d26d631d7ba909ee984ea736be1",
+                "sha256:33c9a6fd20767ccaf70649982f8f3eeb0884035c150c0b818ea660152cf3c809",
+                "sha256:363ab65bf31338eb364062a15f302fc0fab0a49426051429866d71c793c23394",
+                "sha256:43cf03059c0f941b772c8aeb42a0813d68d7081c009542301637e5782f8a33e2",
+                "sha256:56f4fe9c6327adb97406f27a66420b22ce02d71a5c365c48d6b656b4aaeb7775",
+                "sha256:5d35533bf2cee56f38ced91f766cd0038b6abf46f438a80d50c52750088be93f",
+                "sha256:6756212507405f270b66b3ff7f564618de0606395c0fe10a7ae2ffcbbe0b1fba",
+                "sha256:6cdc60e21ec70266947a48839b437d46025076eb4b12c76bd47f8e5eb8a75dcc",
+                "sha256:abc197e4aca8b63f5ae200af03eb95fb4b5055a8f990079b5bdf042f568469dd",
+                "sha256:b14d948e6dce389f9a7afc666d60dd1e35fa2138a8ec5306d30cd2e30d36b40c",
+                "sha256:b47839b53956e2737229d70714f1d75f33e8ac26e52c267f0197b3189ca6de24",
+                "sha256:b6d9ec061b9eca86e4dcc003d93334b95d53909afd5a32c6e4f222157b50c071",
+                "sha256:b891880c187e96339474af2a3b2bfb11a8e4732ff5034be919aa9029484cd201",
+                "sha256:bca8fccc15e3add173da91be8f34121578dc777711ffd98d399be35487c934bf",
+                "sha256:c81703b12475da73a5d66618856d04b1307e43428a7e59d98cfe5a5d608a74c6",
+                "sha256:d2507ee9c99dbddd191c86f0e0c8b724c76d26b0602db9ea23232304382e1f21",
+                "sha256:e36cd7b9d4d920d3bfc2369840da506fa68258f7bb176b8743189793c055e43d",
+                "sha256:e7d84b479ddf39fe3ea05387f10b779155fc0990125f4fb35d636114e1c63a2e",
+                "sha256:eac9af361e0d98335a02ff12fb56caeb7ea1196cf1a49dbf6f17828a131da807",
+                "sha256:edfd858985c23523f4e5a7526ca6ee65ff930207a7ec8a8f57a01eae506aaee7",
+                "sha256:ee9ff50557a942d187ec85462bb0960207e7ec5b19b3b48949263993771c6205",
+                "sha256:f0e822cd7644995d9ba248cb4b67859701748a93e2ab7fc9bc18c599a52e4604",
+                "sha256:f180904f33bdd1e92967923a43c22899e303906d19b2cf8bb547db6653ea6e7d",
+                "sha256:f1d18718f9d78182c6b60f568c9a9cec8a7204d7cb6fad4e511a2ef279e4cb05",
+                "sha256:f4c7bf687303ca47d69f9f0133274958fd672efaa33fb5bcde467862d6c621f0",
+                "sha256:f76176492ff082657ada0d0f10c794b6da5800249ef1692b35cf49b1e93e8ef7"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==4.3.3"
+        },
+        "pycryptodomex": {
+            "hashes": [
+                "sha256:0daad007b685db36d977f9de73f61f8da2a7104e20aca3effd30752fd56f73e1",
+                "sha256:108e5f1c1cd70ffce0b68739c75734437c919d2eaec8e85bffc2c8b4d2794305",
+                "sha256:19764605feea0df966445d46533729b645033f134baeb3ea26ad518c9fdf212c",
+                "sha256:1be97461c439a6af4fe1cf8bf6ca5936d3db252737d2f379cc6b2e394e12a458",
+                "sha256:25cd61e846aaab76d5791d006497134602a9e451e954833018161befc3b5b9ed",
+                "sha256:2a47bcc478741b71273b917232f521fd5704ab4b25d301669879e7273d3586cc",
+                "sha256:59af01efb011b0e8b686ba7758d59cf4a8263f9ad35911bfe3f416cee4f5c08c",
+                "sha256:5dcac11031a71348faaed1f403a0debd56bf5404232284cf8c761ff918886ebc",
+                "sha256:62a5ec91388984909bb5398ea49ee61b68ecb579123694bffa172c3b0a107079",
+                "sha256:645bd4ca6f543685d643dadf6a856cc382b654cc923460e3a10a49c1b3832aeb",
+                "sha256:653b29b0819605fe0898829c8ad6400a6ccde096146730c2da54eede9b7b8baa",
+                "sha256:69138068268127cd605e03438312d8f271135a33140e2742b417d027a0539427",
+                "sha256:6e186342cfcc3aafaad565cbd496060e5a614b441cacc3995ef0091115c1f6c5",
+                "sha256:76bd15bb65c14900d98835fcd10f59e5e0435077431d3a394b60b15864fddd64",
+                "sha256:7805830e0c56d88f4d491fa5ac640dfc894c5ec570d1ece6ed1546e9df2e98d6",
+                "sha256:7a710b79baddd65b806402e14766c721aee8fb83381769c27920f26476276c1e",
+                "sha256:7a7a8f33a1f1fb762ede6cc9cbab8f2a9ba13b196bfaf7bc6f0b39d2ba315a43",
+                "sha256:82ee7696ed8eb9a82c7037f32ba9b7c59e51dda6f105b39f043b6ef293989cb3",
+                "sha256:88afd7a3af7ddddd42c2deda43d53d3dfc016c11327d0915f90ca34ebda91499",
+                "sha256:8af1a451ff9e123d0d8bd5d5e60f8e3315c3a64f3cdd6bc853e26090e195cdc8",
+                "sha256:8ee606964553c1a0bc74057dd8782a37d1c2bc0f01b83193b6f8bb14523b877b",
+                "sha256:91852d4480a4537d169c29a9d104dda44094c78f1f5b67bca76c29a91042b623",
+                "sha256:9c682436c359b5ada67e882fec34689726a09c461efd75b6ea77b2403d5665b7",
+                "sha256:bc3ee1b4d97081260d92ae813a83de4d2653206967c4a0a017580f8b9548ddbc",
+                "sha256:bca649483d5ed251d06daf25957f802e44e6bb6df2e8f218ae71968ff8f8edc4",
+                "sha256:c39778fd0548d78917b61f03c1fa8bfda6cfcf98c767decf360945fe6f97461e",
+                "sha256:cbe71b6712429650e3883dc81286edb94c328ffcd24849accac0a4dbcc76958a",
+                "sha256:d00fe8596e1cc46b44bf3907354e9377aa030ec4cd04afbbf6e899fc1e2a7781",
+                "sha256:d3584623e68a5064a04748fb6d76117a21a7cb5eaba20608a41c7d0c61721794",
+                "sha256:e48217c7901edd95f9f097feaa0388da215ed14ce2ece803d3f300b4e694abea",
+                "sha256:f2e497413560e03421484189a6b65e33fe800d3bd75590e6d78d4dfdb7accf3b",
+                "sha256:ff5c9a67f8a4fba4aed887216e32cbc48f2a6fb2673bb10a99e43be463e15913"
+            ],
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
+            "version": "==3.20.0"
+        },
+        "requests": {
+            "hashes": [
+                "sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f",
+                "sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.7'",
+            "version": "==2.31.0"
+        },
+        "urllib3": {
+            "hashes": [
+                "sha256:450b20ec296a467077128bff42b73080516e71b56ff59a60a02bef2232c4fa9d",
+                "sha256:d0570876c61ab9e520d776c38acbbb5b05a776d3f9ff98a5c8fd5162a444cf19"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==2.2.1"
+        }
+    },
+    "develop": {
+        "alabaster": {
+            "hashes": [
+                "sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65",
+                "sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92"
+            ],
+            "markers": "python_version >= '3.9'",
+            "version": "==0.7.16"
+        },
+        "altgraph": {
+            "hashes": [
+                "sha256:1b5afbb98f6c4dcadb2e2ae6ab9fa994bbb8c1d75f4fa96d340f9437ae454406",
+                "sha256:642743b4750de17e655e6711601b077bc6598dbfa3ba5fa2b2a35ce12b508dff"
+            ],
+            "version": "==0.17.4"
+        },
+        "astroid": {
+            "hashes": [
+                "sha256:951798f922990137ac090c53af473db7ab4e70c770e6d7fae0cec59f74411819",
+                "sha256:ac248253bfa4bd924a0de213707e7ebeeb3138abeb48d798784ead1e56d419d4"
+            ],
+            "markers": "python_full_version >= '3.8.0'",
+            "version": "==3.1.0"
+        },
+        "babel": {
+            "hashes": [
+                "sha256:6919867db036398ba21eb5c7a0f6b28ab8cbc3ae7a73a44ebe34ae74a4e7d363",
+                "sha256:efb1a25b7118e67ce3a259bed20545c29cb68be8ad2c784c83689981b7a57287"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==2.14.0"
+        },
+        "certifi": {
+            "hashes": [
+                "sha256:0569859f95fc761b18b45ef421b1290a0f65f147e92a1e5eb3e635f9a5e4e66f",
+                "sha256:dc383c07b76109f368f6106eee2b593b04a011ea4d55f652c6ca24a754d1cdd1"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==2024.2.2"
+        },
+        "charset-normalizer": {
+            "hashes": [
+                "sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027",
+                "sha256:06a81e93cd441c56a9b65d8e1d043daeb97a3d0856d177d5c90ba85acb3db087",
+                "sha256:0a55554a2fa0d408816b3b5cedf0045f4b8e1a6065aec45849de2d6f3f8e9786",
+                "sha256:0b2b64d2bb6d3fb9112bafa732def486049e63de9618b5843bcdd081d8144cd8",
+                "sha256:10955842570876604d404661fbccbc9c7e684caf432c09c715ec38fbae45ae09",
+                "sha256:122c7fa62b130ed55f8f285bfd56d5f4b4a5b503609d181f9ad85e55c89f4185",
+                "sha256:1ceae2f17a9c33cb48e3263960dc5fc8005351ee19db217e9b1bb15d28c02574",
+                "sha256:1d3193f4a680c64b4b6a9115943538edb896edc190f0b222e73761716519268e",
+                "sha256:1f79682fbe303db92bc2b1136016a38a42e835d932bab5b3b1bfcfbf0640e519",
+                "sha256:2127566c664442652f024c837091890cb1942c30937add288223dc895793f898",
+                "sha256:22afcb9f253dac0696b5a4be4a1c0f8762f8239e21b99680099abd9b2b1b2269",
+                "sha256:25baf083bf6f6b341f4121c2f3c548875ee6f5339300e08be3f2b2ba1721cdd3",
+                "sha256:2e81c7b9c8979ce92ed306c249d46894776a909505d8f5a4ba55b14206e3222f",
+                "sha256:3287761bc4ee9e33561a7e058c72ac0938c4f57fe49a09eae428fd88aafe7bb6",
+                "sha256:34d1c8da1e78d2e001f363791c98a272bb734000fcef47a491c1e3b0505657a8",
+                "sha256:37e55c8e51c236f95b033f6fb391d7d7970ba5fe7ff453dad675e88cf303377a",
+                "sha256:3d47fa203a7bd9c5b6cee4736ee84ca03b8ef23193c0d1ca99b5089f72645c73",
+                "sha256:3e4d1f6587322d2788836a99c69062fbb091331ec940e02d12d179c1d53e25fc",
+                "sha256:42cb296636fcc8b0644486d15c12376cb9fa75443e00fb25de0b8602e64c1714",
+                "sha256:45485e01ff4d3630ec0d9617310448a8702f70e9c01906b0d0118bdf9d124cf2",
+                "sha256:4a78b2b446bd7c934f5dcedc588903fb2f5eec172f3d29e52a9096a43722adfc",
+                "sha256:4ab2fe47fae9e0f9dee8c04187ce5d09f48eabe611be8259444906793ab7cbce",
+                "sha256:4d0d1650369165a14e14e1e47b372cfcb31d6ab44e6e33cb2d4e57265290044d",
+                "sha256:549a3a73da901d5bc3ce8d24e0600d1fa85524c10287f6004fbab87672bf3e1e",
+                "sha256:55086ee1064215781fff39a1af09518bc9255b50d6333f2e4c74ca09fac6a8f6",
+                "sha256:572c3763a264ba47b3cf708a44ce965d98555f618ca42c926a9c1616d8f34269",
+                "sha256:573f6eac48f4769d667c4442081b1794f52919e7edada77495aaed9236d13a96",
+                "sha256:5b4c145409bef602a690e7cfad0a15a55c13320ff7a3ad7ca59c13bb8ba4d45d",
+                "sha256:6463effa3186ea09411d50efc7d85360b38d5f09b870c48e4600f63af490e56a",
+                "sha256:65f6f63034100ead094b8744b3b97965785388f308a64cf8d7c34f2f2e5be0c4",
+                "sha256:663946639d296df6a2bb2aa51b60a2454ca1cb29835324c640dafb5ff2131a77",
+                "sha256:6897af51655e3691ff853668779c7bad41579facacf5fd7253b0133308cf000d",
+                "sha256:68d1f8a9e9e37c1223b656399be5d6b448dea850bed7d0f87a8311f1ff3dabb0",
+                "sha256:6ac7ffc7ad6d040517be39eb591cac5ff87416c2537df6ba3cba3bae290c0fed",
+                "sha256:6b3251890fff30ee142c44144871185dbe13b11bab478a88887a639655be1068",
+                "sha256:6c4caeef8fa63d06bd437cd4bdcf3ffefe6738fb1b25951440d80dc7df8c03ac",
+                "sha256:6ef1d82a3af9d3eecdba2321dc1b3c238245d890843e040e41e470ffa64c3e25",
+                "sha256:753f10e867343b4511128c6ed8c82f7bec3bd026875576dfd88483c5c73b2fd8",
+                "sha256:7cd13a2e3ddeed6913a65e66e94b51d80a041145a026c27e6bb76c31a853c6ab",
+                "sha256:7ed9e526742851e8d5cc9e6cf41427dfc6068d4f5a3bb03659444b4cabf6bc26",
+                "sha256:7f04c839ed0b6b98b1a7501a002144b76c18fb1c1850c8b98d458ac269e26ed2",
+                "sha256:802fe99cca7457642125a8a88a084cef28ff0cf9407060f7b93dca5aa25480db",
+                "sha256:80402cd6ee291dcb72644d6eac93785fe2c8b9cb30893c1af5b8fdd753b9d40f",
+                "sha256:8465322196c8b4d7ab6d1e049e4c5cb460d0394da4a27d23cc242fbf0034b6b5",
+                "sha256:86216b5cee4b06df986d214f664305142d9c76df9b6512be2738aa72a2048f99",
+                "sha256:87d1351268731db79e0f8e745d92493ee2841c974128ef629dc518b937d9194c",
+                "sha256:8bdb58ff7ba23002a4c5808d608e4e6c687175724f54a5dade5fa8c67b604e4d",
+                "sha256:8c622a5fe39a48f78944a87d4fb8a53ee07344641b0562c540d840748571b811",
+                "sha256:8d756e44e94489e49571086ef83b2bb8ce311e730092d2c34ca8f7d925cb20aa",
+                "sha256:8f4a014bc36d3c57402e2977dada34f9c12300af536839dc38c0beab8878f38a",
+                "sha256:9063e24fdb1e498ab71cb7419e24622516c4a04476b17a2dab57e8baa30d6e03",
+                "sha256:90d558489962fd4918143277a773316e56c72da56ec7aa3dc3dbbe20fdfed15b",
+                "sha256:923c0c831b7cfcb071580d3f46c4baf50f174be571576556269530f4bbd79d04",
+                "sha256:95f2a5796329323b8f0512e09dbb7a1860c46a39da62ecb2324f116fa8fdc85c",
+                "sha256:96b02a3dc4381e5494fad39be677abcb5e6634bf7b4fa83a6dd3112607547001",
+                "sha256:9f96df6923e21816da7e0ad3fd47dd8f94b2a5ce594e00677c0013018b813458",
+                "sha256:a10af20b82360ab00827f916a6058451b723b4e65030c5a18577c8b2de5b3389",
+                "sha256:a50aebfa173e157099939b17f18600f72f84eed3049e743b68ad15bd69b6bf99",
+                "sha256:a981a536974bbc7a512cf44ed14938cf01030a99e9b3a06dd59578882f06f985",
+                "sha256:a9a8e9031d613fd2009c182b69c7b2c1ef8239a0efb1df3f7c8da66d5dd3d537",
+                "sha256:ae5f4161f18c61806f411a13b0310bea87f987c7d2ecdbdaad0e94eb2e404238",
+                "sha256:aed38f6e4fb3f5d6bf81bfa990a07806be9d83cf7bacef998ab1a9bd660a581f",
+                "sha256:b01b88d45a6fcb69667cd6d2f7a9aeb4bf53760d7fc536bf679ec94fe9f3ff3d",
+                "sha256:b261ccdec7821281dade748d088bb6e9b69e6d15b30652b74cbbac25e280b796",
+                "sha256:b2b0a0c0517616b6869869f8c581d4eb2dd83a4d79e0ebcb7d373ef9956aeb0a",
+                "sha256:b4a23f61ce87adf89be746c8a8974fe1c823c891d8f86eb218bb957c924bb143",
+                "sha256:bd8f7df7d12c2db9fab40bdd87a7c09b1530128315d047a086fa3ae3435cb3a8",
+                "sha256:beb58fe5cdb101e3a055192ac291b7a21e3b7ef4f67fa1d74e331a7f2124341c",
+                "sha256:c002b4ffc0be611f0d9da932eb0f704fe2602a9a949d1f738e4c34c75b0863d5",
+                "sha256:c083af607d2515612056a31f0a8d9e0fcb5876b7bfc0abad3ecd275bc4ebc2d5",
+                "sha256:c180f51afb394e165eafe4ac2936a14bee3eb10debc9d9e4db8958fe36afe711",
+                "sha256:c235ebd9baae02f1b77bcea61bce332cb4331dc3617d254df3323aa01ab47bd4",
+                "sha256:cd70574b12bb8a4d2aaa0094515df2463cb429d8536cfb6c7ce983246983e5a6",
+                "sha256:d0eccceffcb53201b5bfebb52600a5fb483a20b61da9dbc885f8b103cbe7598c",
+                "sha256:d965bba47ddeec8cd560687584e88cf699fd28f192ceb452d1d7ee807c5597b7",
+                "sha256:db364eca23f876da6f9e16c9da0df51aa4f104a972735574842618b8c6d999d4",
+                "sha256:ddbb2551d7e0102e7252db79ba445cdab71b26640817ab1e3e3648dad515003b",
+                "sha256:deb6be0ac38ece9ba87dea880e438f25ca3eddfac8b002a2ec3d9183a454e8ae",
+                "sha256:e06ed3eb3218bc64786f7db41917d4e686cc4856944f53d5bdf83a6884432e12",
+                "sha256:e27ad930a842b4c5eb8ac0016b0a54f5aebbe679340c26101df33424142c143c",
+                "sha256:e537484df0d8f426ce2afb2d0f8e1c3d0b114b83f8850e5f2fbea0e797bd82ae",
+                "sha256:eb00ed941194665c332bf8e078baf037d6c35d7c4f3102ea2d4f16ca94a26dc8",
+                "sha256:eb6904c354526e758fda7167b33005998fb68c46fbc10e013ca97f21ca5c8887",
+                "sha256:eb8821e09e916165e160797a6c17edda0679379a4be5c716c260e836e122f54b",
+                "sha256:efcb3f6676480691518c177e3b465bcddf57cea040302f9f4e6e191af91174d4",
+                "sha256:f27273b60488abe721a075bcca6d7f3964f9f6f067c8c4c605743023d7d3944f",
+                "sha256:f30c3cb33b24454a82faecaf01b19c18562b1e89558fb6c56de4d9118a032fd5",
+                "sha256:fb69256e180cb6c8a894fee62b3afebae785babc1ee98b81cdf68bbca1987f33",
+                "sha256:fd1abc0d89e30cc4e02e4064dc67fcc51bd941eb395c502aac3ec19fab46b519",
+                "sha256:ff8fa367d09b717b2a17a052544193ad76cd49979c805768879cb63d9ca50561"
+            ],
+            "markers": "python_full_version >= '3.7.0'",
+            "version": "==3.3.2"
+        },
+        "dill": {
+            "hashes": [
+                "sha256:3ebe3c479ad625c4553aca177444d89b486b1d84982eeacded644afc0cf797ca",
+                "sha256:c36ca9ffb54365bdd2f8eb3eff7d2a21237f8452b57ace88b1ac615b7e815bd7"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.8'",
+            "version": "==0.3.8"
+        },
+        "docutils": {
+            "hashes": [
+                "sha256:96f387a2c5562db4476f09f13bbab2192e764cac08ebbf3a34a95d9b1e4a59d6",
+                "sha256:f08a4e276c3a1583a86dce3e34aba3fe04d02bba2dd51ed16106244e8a923e3b"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==0.20.1"
+        },
+        "idna": {
+            "hashes": [
+                "sha256:9ecdbbd083b06798ae1e86adcbfe8ab1479cf864e4ee30fe4e46a003d12491ca",
+                "sha256:c05567e9c24a6b9faaa835c4821bad0590fbb9d5779e7caa6e1cc4978e7eb24f"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==3.6"
+        },
+        "imagesize": {
+            "hashes": [
+                "sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b",
+                "sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a"
+            ],
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+            "version": "==1.4.1"
+        },
+        "isort": {
+            "hashes": [
+                "sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109",
+                "sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6"
+            ],
+            "markers": "python_full_version >= '3.8.0'",
+            "version": "==5.13.2"
+        },
+        "jinja2": {
+            "hashes": [
+                "sha256:7d6d50dd97d52cbc355597bd845fabfbac3f551e1f99619e39a35ce8c370b5fa",
+                "sha256:ac8bd6544d4bb2c9792bf3a159e80bba8fda7f07e81bc3aed565432d5925ba90"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==3.1.3"
+        },
+        "markupsafe": {
+            "hashes": [
+                "sha256:00e046b6dd71aa03a41079792f8473dc494d564611a8f89bbbd7cb93295ebdcf",
+                "sha256:075202fa5b72c86ad32dc7d0b56024ebdbcf2048c0ba09f1cde31bfdd57bcfff",
+                "sha256:0e397ac966fdf721b2c528cf028494e86172b4feba51d65f81ffd65c63798f3f",
+                "sha256:17b950fccb810b3293638215058e432159d2b71005c74371d784862b7e4683f3",
+                "sha256:1f3fbcb7ef1f16e48246f704ab79d79da8a46891e2da03f8783a5b6fa41a9532",
+                "sha256:2174c595a0d73a3080ca3257b40096db99799265e1c27cc5a610743acd86d62f",
+                "sha256:2b7c57a4dfc4f16f7142221afe5ba4e093e09e728ca65c51f5620c9aaeb9a617",
+                "sha256:2d2d793e36e230fd32babe143b04cec8a8b3eb8a3122d2aceb4a371e6b09b8df",
+                "sha256:30b600cf0a7ac9234b2638fbc0fb6158ba5bdcdf46aeb631ead21248b9affbc4",
+                "sha256:397081c1a0bfb5124355710fe79478cdbeb39626492b15d399526ae53422b906",
+                "sha256:3a57fdd7ce31c7ff06cdfbf31dafa96cc533c21e443d57f5b1ecc6cdc668ec7f",
+                "sha256:3c6b973f22eb18a789b1460b4b91bf04ae3f0c4234a0a6aa6b0a92f6f7b951d4",
+                "sha256:3e53af139f8579a6d5f7b76549125f0d94d7e630761a2111bc431fd820e163b8",
+                "sha256:4096e9de5c6fdf43fb4f04c26fb114f61ef0bf2e5604b6ee3019d51b69e8c371",
+                "sha256:4275d846e41ecefa46e2015117a9f491e57a71ddd59bbead77e904dc02b1bed2",
+                "sha256:4c31f53cdae6ecfa91a77820e8b151dba54ab528ba65dfd235c80b086d68a465",
+                "sha256:4f11aa001c540f62c6166c7726f71f7573b52c68c31f014c25cc7901deea0b52",
+                "sha256:5049256f536511ee3f7e1b3f87d1d1209d327e818e6ae1365e8653d7e3abb6a6",
+                "sha256:58c98fee265677f63a4385256a6d7683ab1832f3ddd1e66fe948d5880c21a169",
+                "sha256:598e3276b64aff0e7b3451b72e94fa3c238d452e7ddcd893c3ab324717456bad",
+                "sha256:5b7b716f97b52c5a14bffdf688f971b2d5ef4029127f1ad7a513973cfd818df2",
+                "sha256:5dedb4db619ba5a2787a94d877bc8ffc0566f92a01c0ef214865e54ecc9ee5e0",
+                "sha256:619bc166c4f2de5caa5a633b8b7326fbe98e0ccbfacabd87268a2b15ff73a029",
+                "sha256:629ddd2ca402ae6dbedfceeba9c46d5f7b2a61d9749597d4307f943ef198fc1f",
+                "sha256:656f7526c69fac7f600bd1f400991cc282b417d17539a1b228617081106feb4a",
+                "sha256:6ec585f69cec0aa07d945b20805be741395e28ac1627333b1c5b0105962ffced",
+                "sha256:72b6be590cc35924b02c78ef34b467da4ba07e4e0f0454a2c5907f473fc50ce5",
+                "sha256:7502934a33b54030eaf1194c21c692a534196063db72176b0c4028e140f8f32c",
+                "sha256:7a68b554d356a91cce1236aa7682dc01df0edba8d043fd1ce607c49dd3c1edcf",
+                "sha256:7b2e5a267c855eea6b4283940daa6e88a285f5f2a67f2220203786dfa59b37e9",
+                "sha256:823b65d8706e32ad2df51ed89496147a42a2a6e01c13cfb6ffb8b1e92bc910bb",
+                "sha256:8590b4ae07a35970728874632fed7bd57b26b0102df2d2b233b6d9d82f6c62ad",
+                "sha256:8dd717634f5a044f860435c1d8c16a270ddf0ef8588d4887037c5028b859b0c3",
+                "sha256:8dec4936e9c3100156f8a2dc89c4b88d5c435175ff03413b443469c7c8c5f4d1",
+                "sha256:97cafb1f3cbcd3fd2b6fbfb99ae11cdb14deea0736fc2b0952ee177f2b813a46",
+                "sha256:a17a92de5231666cfbe003f0e4b9b3a7ae3afb1ec2845aadc2bacc93ff85febc",
+                "sha256:a549b9c31bec33820e885335b451286e2969a2d9e24879f83fe904a5ce59d70a",
+                "sha256:ac07bad82163452a6884fe8fa0963fb98c2346ba78d779ec06bd7a6262132aee",
+                "sha256:ae2ad8ae6ebee9d2d94b17fb62763125f3f374c25618198f40cbb8b525411900",
+                "sha256:b91c037585eba9095565a3556f611e3cbfaa42ca1e865f7b8015fe5c7336d5a5",
+                "sha256:bc1667f8b83f48511b94671e0e441401371dfd0f0a795c7daa4a3cd1dde55bea",
+                "sha256:bec0a414d016ac1a18862a519e54b2fd0fc8bbfd6890376898a6c0891dd82e9f",
+                "sha256:bf50cd79a75d181c9181df03572cdce0fbb75cc353bc350712073108cba98de5",
+                "sha256:bff1b4290a66b490a2f4719358c0cdcd9bafb6b8f061e45c7a2460866bf50c2e",
+                "sha256:c061bb86a71b42465156a3ee7bd58c8c2ceacdbeb95d05a99893e08b8467359a",
+                "sha256:c8b29db45f8fe46ad280a7294f5c3ec36dbac9491f2d1c17345be8e69cc5928f",
+                "sha256:ce409136744f6521e39fd8e2a24c53fa18ad67aa5bc7c2cf83645cce5b5c4e50",
+                "sha256:d050b3361367a06d752db6ead6e7edeb0009be66bc3bae0ee9d97fb326badc2a",
+                "sha256:d283d37a890ba4c1ae73ffadf8046435c76e7bc2247bbb63c00bd1a709c6544b",
+                "sha256:d9fad5155d72433c921b782e58892377c44bd6252b5af2f67f16b194987338a4",
+                "sha256:daa4ee5a243f0f20d528d939d06670a298dd39b1ad5f8a72a4275124a7819eff",
+                "sha256:db0b55e0f3cc0be60c1f19efdde9a637c32740486004f20d1cff53c3c0ece4d2",
+                "sha256:e61659ba32cf2cf1481e575d0462554625196a1f2fc06a1c777d3f48e8865d46",
+                "sha256:ea3d8a3d18833cf4304cd2fc9cbb1efe188ca9b5efef2bdac7adc20594a0e46b",
+                "sha256:ec6a563cff360b50eed26f13adc43e61bc0c04d94b8be985e6fb24b81f6dcfdf",
+                "sha256:f5dfb42c4604dddc8e4305050aa6deb084540643ed5804d7455b5df8fe16f5e5",
+                "sha256:fa173ec60341d6bb97a89f5ea19c85c5643c1e7dedebc22f5181eb73573142c5",
+                "sha256:fa9db3f79de01457b03d4f01b34cf91bc0048eb2c3846ff26f66687c2f6d16ab",
+                "sha256:fce659a462a1be54d2ffcacea5e3ba2d74daa74f30f5f143fe0c58636e355fdd",
+                "sha256:ffee1f21e5ef0d712f9033568f8344d5da8cc2869dbd08d87c84656e6a2d2f68"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==2.1.5"
+        },
+        "mccabe": {
+            "hashes": [
+                "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325",
+                "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e"
+            ],
+            "markers": "python_version >= '3.6'",
+            "version": "==0.7.0"
+        },
+        "mypy": {
+            "hashes": [
+                "sha256:0235391f1c6f6ce487b23b9dbd1327b4ec33bb93934aa986efe8a9563d9349e6",
+                "sha256:190da1ee69b427d7efa8aa0d5e5ccd67a4fb04038c380237a0d96829cb157913",
+                "sha256:2418488264eb41f69cc64a69a745fad4a8f86649af4b1041a4c64ee61fc61129",
+                "sha256:3a3c007ff3ee90f69cf0a15cbcdf0995749569b86b6d2f327af01fd1b8aee9dc",
+                "sha256:3cc5da0127e6a478cddd906068496a97a7618a21ce9b54bde5bf7e539c7af974",
+                "sha256:48533cdd345c3c2e5ef48ba3b0d3880b257b423e7995dada04248725c6f77374",
+                "sha256:49c87c15aed320de9b438ae7b00c1ac91cd393c1b854c2ce538e2a72d55df150",
+                "sha256:4d3dbd346cfec7cb98e6cbb6e0f3c23618af826316188d587d1c1bc34f0ede03",
+                "sha256:571741dc4194b4f82d344b15e8837e8c5fcc462d66d076748142327626a1b6e9",
+                "sha256:587ce887f75dd9700252a3abbc9c97bbe165a4a630597845c61279cf32dfbf02",
+                "sha256:5d741d3fc7c4da608764073089e5f58ef6352bedc223ff58f2f038c2c4698a89",
+                "sha256:5e6061f44f2313b94f920e91b204ec600982961e07a17e0f6cd83371cb23f5c2",
+                "sha256:61758fabd58ce4b0720ae1e2fea5cfd4431591d6d590b197775329264f86311d",
+                "sha256:653265f9a2784db65bfca694d1edd23093ce49740b2244cde583aeb134c008f3",
+                "sha256:68edad3dc7d70f2f17ae4c6c1b9471a56138ca22722487eebacfd1eb5321d612",
+                "sha256:81a10926e5473c5fc3da8abb04119a1f5811a236dc3a38d92015cb1e6ba4cb9e",
+                "sha256:85ca5fcc24f0b4aeedc1d02f93707bccc04733f21d41c88334c5482219b1ccb3",
+                "sha256:a260627a570559181a9ea5de61ac6297aa5af202f06fd7ab093ce74e7181e43e",
+                "sha256:aceb1db093b04db5cd390821464504111b8ec3e351eb85afd1433490163d60cd",
+                "sha256:b685154e22e4e9199fc95f298661deea28aaede5ae16ccc8cbb1045e716b3e04",
+                "sha256:d357423fa57a489e8c47b7c85dfb96698caba13d66e086b412298a1a0ea3b0ed",
+                "sha256:d4d5ddc13421ba3e2e082a6c2d74c2ddb3979c39b582dacd53dd5d9431237185",
+                "sha256:e49499be624dead83927e70c756970a0bc8240e9f769389cdf5714b0784ca6bf",
+                "sha256:e54396d70be04b34f31d2edf3362c1edd023246c82f1730bbf8768c28db5361b",
+                "sha256:f88566144752999351725ac623471661c9d1cd8caa0134ff98cceeea181789f4",
+                "sha256:f8a67616990062232ee4c3952f41c779afac41405806042a8126fe96e098419f",
+                "sha256:fe28657de3bfec596bbeef01cb219833ad9d38dd5393fc649f4b366840baefe6"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.8'",
+            "version": "==1.9.0"
+        },
+        "mypy-extensions": {
+            "hashes": [
+                "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d",
+                "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.0.0"
+        },
+        "packaging": {
+            "hashes": [
+                "sha256:2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5",
+                "sha256:eb82c5e3e56209074766e6885bb04b8c38a0c015d0a30036ebe7ece34c9989e9"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==24.0"
+        },
+        "pefile": {
+            "hashes": [
+                "sha256:82e6114004b3d6911c77c3953e3838654b04511b8b66e8583db70c65998017dc",
+                "sha256:da185cd2af68c08a6cd4481f7325ed600a88f6a813bad9dea07ab3ef73d8d8d6"
+            ],
+            "index": "pypi",
+            "markers": "python_full_version >= '3.6.0'",
+            "version": "==2023.2.7"
+        },
+        "platformdirs": {
+            "hashes": [
+                "sha256:0614df2a2f37e1a662acbd8e2b25b92ccf8632929bc6d43467e17fe89c75e068",
+                "sha256:ef0cc731df711022c174543cb70a9b5bd22e5a9337c8624ef2c2ceb8ddad8768"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==4.2.0"
+        },
+        "psutil": {
+            "hashes": [
+                "sha256:02615ed8c5ea222323408ceba16c60e99c3f91639b07da6373fb7e6539abc56d",
+                "sha256:05806de88103b25903dff19bb6692bd2e714ccf9e668d050d144012055cbca73",
+                "sha256:26bd09967ae00920df88e0352a91cff1a78f8d69b3ecabbfe733610c0af486c8",
+                "sha256:27cc40c3493bb10de1be4b3f07cae4c010ce715290a5be22b98493509c6299e2",
+                "sha256:36f435891adb138ed3c9e58c6af3e2e6ca9ac2f365efe1f9cfef2794e6c93b4e",
+                "sha256:50187900d73c1381ba1454cf40308c2bf6f34268518b3f36a9b663ca87e65e36",
+                "sha256:611052c4bc70432ec770d5d54f64206aa7203a101ec273a0cd82418c86503bb7",
+                "sha256:6be126e3225486dff286a8fb9a06246a5253f4c7c53b475ea5f5ac934e64194c",
+                "sha256:7d79560ad97af658a0f6adfef8b834b53f64746d45b403f225b85c5c2c140eee",
+                "sha256:8cb6403ce6d8e047495a701dc7c5bd788add903f8986d523e3e20b98b733e421",
+                "sha256:8db4c1b57507eef143a15a6884ca10f7c73876cdf5d51e713151c1236a0e68cf",
+                "sha256:aee678c8720623dc456fa20659af736241f575d79429a0e5e9cf88ae0605cc81",
+                "sha256:bc56c2a1b0d15aa3eaa5a60c9f3f8e3e565303b465dbf57a1b730e7a2b9844e0",
+                "sha256:bd1184ceb3f87651a67b2708d4c3338e9b10c5df903f2e3776b62303b26cb631",
+                "sha256:d06016f7f8625a1825ba3732081d77c94589dca78b7a3fc072194851e88461a4",
+                "sha256:d16bbddf0693323b8c6123dd804100241da461e41d6e332fb0ba6058f630f8c8"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
+            "version": "==5.9.8"
+        },
+        "pygments": {
+            "hashes": [
+                "sha256:b27c2826c47d0f3219f29554824c30c5e8945175d888647acd804ddd04af846c",
+                "sha256:da46cec9fd2de5be3a8a784f434e4c4ab670b4ff54d605c4c2717e9d49c4c367"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==2.17.2"
+        },
+        "pyinstaller": {
+            "hashes": [
+                "sha256:0dae0edbe6d667b6b0ccd8c97a148f86474a82da7ce582296f9025f4c7242ec6",
+                "sha256:1b3b7d6d3b18d76a833fd5a4d7f4544c5e2c2a4db4a728ea191e62f69d5cc33c",
+                "sha256:5f432f3fdef053989e0a44134e483131c533dab7637e6afd80c3f7c26e6dbcc9",
+                "sha256:61865eee5e0d8f8252722f6d001baec497b7cee79ebe62c33a6ba86ba0c7010d",
+                "sha256:6cfee8a74ea2d3a1dc8e99e732a87b314739dc14363778143caac31f8aee9039",
+                "sha256:6ffd76a0194dac4df5e66dcfccc7b597f3eaa40ef9a3f63548f260aa2c187512",
+                "sha256:7c76bfcb624803c311fa8fb137e4780d0ec86d11b7d90a8f43f185e2554afdcc",
+                "sha256:81ec15c0deb8c7a0f95bea85b49eecc2df1bdeaf5fe487a41d97de6b0ad29dff",
+                "sha256:9d828213aea5401bb33a36ca396f8dc76a59a25bce1d76a13c9ad94ba29fbe42",
+                "sha256:a54968df2228f0128607b1dced41bbff94149d459987fb5cd1a41893e9bb85df",
+                "sha256:b1e55113c5a40cb7041c908a57f212f3ebd3e444dbb245ca2f91d86a76dabec5",
+                "sha256:e1266498893ce1d6cc7337e8d2acbf7905a10ed2b7c8377270117d6b7b922fc4"
+            ],
+            "index": "pypi",
+            "markers": "python_version < '3.13' and python_version >= '3.8'",
+            "version": "==6.5.0"
+        },
+        "pyinstaller-hooks-contrib": {
+            "hashes": [
+                "sha256:6701752d525e1f4eda1eaec2c2affc206171e15c7a4e188a152fcf3ed3308024",
+                "sha256:d18657c29267c63563a96b8fc78db6ba9ae40af6702acb2f8c871df12c75b60b"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==2024.3"
+        },
+        "pylint": {
+            "hashes": [
+                "sha256:507a5b60953874766d8a366e8e8c7af63e058b26345cfcb5f91f89d987fd6b74",
+                "sha256:6a69beb4a6f63debebaab0a3477ecd0f559aa726af4954fc948c51f7a2549e23"
+            ],
+            "index": "pypi",
+            "markers": "python_full_version >= '3.8.0'",
+            "version": "==3.1.0"
+        },
+        "pywin32-ctypes": {
+            "hashes": [
+                "sha256:3426e063bdd5fd4df74a14fa3cf80a0b42845a87e1d1e81f6549f9daec593a60",
+                "sha256:bf490a1a709baf35d688fe0ecf980ed4de11d2b3e37b51e5442587a75d9957e7"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.6'",
+            "version": "==0.2.2"
+        },
+        "requests": {
+            "hashes": [
+                "sha256:58cd2187c01e70e6e26505bca751777aa9f2ee0b7f4300988b709f44e013003f",
+                "sha256:942c5a758f98d790eaed1a29cb6eefc7ffb0d1cf7af05c3d2791656dbd6ad1e1"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.7'",
+            "version": "==2.31.0"
+        },
+        "setuptools": {
+            "hashes": [
+                "sha256:0ff4183f8f42cd8fa3acea16c45205521a4ef28f73c6391d8a25e92893134f2e",
+                "sha256:c21c49fb1042386df081cb5d86759792ab89efca84cf114889191cd09aacc80c"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==69.2.0"
+        },
+        "snowballstemmer": {
+            "hashes": [
+                "sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1",
+                "sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a"
+            ],
+            "version": "==2.2.0"
+        },
+        "sphinx": {
+            "hashes": [
+                "sha256:1e09160a40b956dc623c910118fa636da93bd3ca0b9876a7b3df90f07d691560",
+                "sha256:9a5160e1ea90688d5963ba09a2dcd8bdd526620edbb65c328728f1b2228d5ab5"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.9'",
+            "version": "==7.2.6"
+        },
+        "sphinx-autodoc-typehints": {
+            "hashes": [
+                "sha256:12c0e161f6fe191c2cdfd8fa3caea271f5387d9fbc67ebcd6f4f1f24ce880993",
+                "sha256:7f2cdac2e70fd9787926b6e9e541cd4ded1e838d2b46fda2a1bb0a75ec5b7f3a"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.8'",
+            "version": "==2.0.0"
+        },
+        "sphinxcontrib-applehelp": {
+            "hashes": [
+                "sha256:c40a4f96f3776c4393d933412053962fac2b84f4c99a7982ba42e09576a70619",
+                "sha256:cb61eb0ec1b61f349e5cc36b2028e9e7ca765be05e49641c97241274753067b4"
+            ],
+            "markers": "python_version >= '3.9'",
+            "version": "==1.0.8"
+        },
+        "sphinxcontrib-devhelp": {
+            "hashes": [
+                "sha256:6485d09629944511c893fa11355bda18b742b83a2b181f9a009f7e500595c90f",
+                "sha256:9893fd3f90506bc4b97bdb977ceb8fbd823989f4316b28c3841ec128544372d3"
+            ],
+            "markers": "python_version >= '3.9'",
+            "version": "==1.0.6"
+        },
+        "sphinxcontrib-htmlhelp": {
+            "hashes": [
+                "sha256:0dc87637d5de53dd5eec3a6a01753b1ccf99494bd756aafecd74b4fa9e729015",
+                "sha256:393f04f112b4d2f53d93448d4bce35842f62b307ccdc549ec1585e950bc35e04"
+            ],
+            "markers": "python_version >= '3.9'",
+            "version": "==2.0.5"
+        },
+        "sphinxcontrib-jquery": {
+            "hashes": [
+                "sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a",
+                "sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '2.7'",
+            "version": "==4.1"
+        },
+        "sphinxcontrib-jsmath": {
+            "hashes": [
+                "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178",
+                "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"
+            ],
+            "markers": "python_version >= '3.5'",
+            "version": "==1.0.1"
+        },
+        "sphinxcontrib-qthelp": {
+            "hashes": [
+                "sha256:053dedc38823a80a7209a80860b16b722e9e0209e32fea98c90e4e6624588ed6",
+                "sha256:e2ae3b5c492d58fcbd73281fbd27e34b8393ec34a073c792642cd8e529288182"
+            ],
+            "markers": "python_version >= '3.9'",
+            "version": "==1.0.7"
+        },
+        "sphinxcontrib-serializinghtml": {
+            "hashes": [
+                "sha256:326369b8df80a7d2d8d7f99aa5ac577f51ea51556ed974e7716cfd4fca3f6cb7",
+                "sha256:93f3f5dc458b91b192fe10c397e324f262cf163d79f3282c158e8436a2c4511f"
+            ],
+            "markers": "python_version >= '3.9'",
+            "version": "==1.1.10"
+        },
+        "tomli": {
+            "hashes": [
+                "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc",
+                "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.7'",
+            "version": "==2.0.1"
+        },
+        "tomlkit": {
+            "hashes": [
+                "sha256:5cd82d48a3dd89dee1f9d64420aa20ae65cfbd00668d6f094d7578a78efbb77b",
+                "sha256:7ca1cfc12232806517a8515047ba66a19369e71edf2439d0f5824f91032b6cc3"
+            ],
+            "markers": "python_version >= '3.7'",
+            "version": "==0.12.4"
+        },
+        "types-requests": {
+            "hashes": [
+                "sha256:47872893d65a38e282ee9f277a4ee50d1b28bd592040df7d1fdaffdf3779937d",
+                "sha256:b1c1b66abfb7fa79aae09097a811c4aa97130eb8831c60e47aee4ca344731ca5"
+            ],
+            "index": "pypi",
+            "markers": "python_version >= '3.8'",
+            "version": "==2.31.0.20240311"
+        },
+        "typing-extensions": {
+            "hashes": [
+                "sha256:69b1a937c3a517342112fb4c6df7e72fc39a38e7891a5730ed4985b5214b5475",
+                "sha256:b0abd7c89e8fb96f98db18d86106ff1d90ab692004eb746cf6eda2682f91b3cb"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==4.10.0"
+        },
+        "urllib3": {
+            "hashes": [
+                "sha256:450b20ec296a467077128bff42b73080516e71b56ff59a60a02bef2232c4fa9d",
+                "sha256:d0570876c61ab9e520d776c38acbbb5b05a776d3f9ff98a5c8fd5162a444cf19"
+            ],
+            "markers": "python_version >= '3.8'",
+            "version": "==2.2.1"
+        }
+    }
+}

README.rst

@@ -1,5 +1,34 @@
 .. image:: https://raw.githubusercontent.com/instaloader/instaloader/master/docs/logo_heading.png
 
+.. badges-start
+
+|pypi| |pyversion| |license| |aur| |contributors| |downloads|
+
+.. |pypi| image:: https://img.shields.io/pypi/v/instaloader.svg
+   :alt: Instaloader PyPI Project Page
+   :target: https://pypi.org/project/instaloader/
+
+.. |license| image:: https://img.shields.io/github/license/instaloader/instaloader.svg
+   :alt: MIT License
+   :target: https://github.com/instaloader/instaloader/blob/master/LICENSE
+
+.. |pyversion| image:: https://img.shields.io/pypi/pyversions/instaloader.svg
+   :alt: Supported Python Versions
+
+.. |contributors| image:: https://img.shields.io/github/contributors/instaloader/instaloader.svg
+   :alt: Contributor Count
+   :target: https://github.com/instaloader/instaloader/graphs/contributors
+
+.. |aur| image:: https://img.shields.io/aur/version/instaloader.svg
+   :alt: Arch User Repository Package
+   :target: https://aur.archlinux.org/packages/instaloader/
+
+.. |downloads| image:: https://pepy.tech/badge/instaloader/month
+   :alt: PyPI Download Count
+   :target: https://pepy.tech/project/instaloader
+
+.. badges-end
+
 ::
 
     $ pip3 install instaloader
@@ -17,11 +46,14 @@
   directory accordingly,
 
 - allows **fine-grained customization** of filters and where to store
-  downloaded media.
+  downloaded media,
+
+- automatically **resumes previously-interrupted** download iterations.
 
 ::
 
-    instaloader [--comments] [--geotags] [--stories] [--highlights] [--tagged]
+    instaloader [--comments] [--geotags]
+                [--stories] [--highlights] [--tagged] [--igtv]
                 [--login YOUR-USERNAME] [--fast-update]
                 profile | "#hashtag" | :stories | :feed | :saved
 
@@ -48,7 +80,19 @@ To later **update your local copy** of that profiles, you may run
     instaloader --fast-update profile [profile ...]
 
 If ``--fast-update`` is given, Instaloader stops when arriving at the
-first already-downloaded picture. When updating profiles, Instaloader
+first already-downloaded picture.
+
+Alternatively, you can use ``--latest-stamps`` to have Instaloader store
+the time each profile was last downloaded and only download newer media:
+
+::
+
+    instaloader --latest-stamps -- profile [profile ...]
+
+With this option it's possible to move or delete downloaded media and still keep
+the archive updated.
+
+When updating profiles, Instaloader
 automatically **detects profile name changes** and renames the target directory
 accordingly.
 
@@ -66,6 +110,37 @@ already have a valid session cookie file.
 
 `Instaloader Documentation <https://instaloader.github.io/basic-usage.html>`__
 
+Contributing
+------------
+
+As an open source project, Instaloader heavily depends on the contributions from
+its community. See
+`contributing <https://instaloader.github.io/contributing.html>`__
+for how you may help Instaloader to become an even greater tool.
+
+Supporters
+----------
+
+.. current-sponsors-start
+
+| Instaloader is proudly sponsored by
+|  `@rocketapi-io <https://github.com/rocketapi-io>`__
+
+See `Alex' GitHub Sponsors <https://github.com/sponsors/aandergr>`__ page for
+how you can sponsor the development of Instaloader!
+
+.. current-sponsors-end
+
+It is a pleasure for us to share our Instaloader to the world, and we are proud
+to have attracted such an active and motivating community, with so many users
+who share their suggestions and ideas with us. Buying a community-sponsored beer
+or coffee from time to time is very likely to further raise our passion for the
+development of Instaloader.
+
+| For Donations, we provide GitHub Sponsors page, a PayPal.Me link and a Bitcoin address.
+|  GitHub Sponsors: `Sponsor @aandergr on GitHub Sponsors <https://github.com/sponsors/aandergr>`__
+|  PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
+|  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
 
 Disclaimer
 ----------
@@ -78,21 +153,3 @@ subsidiaries. This is an independent and unofficial project. Use at your own ris
 Instaloader is licensed under an MIT license. Refer to ``LICENSE`` file for more information.
 
 .. disclaimer-end
-
-Contributing
-------------
-
-As an open source project, Instaloader heavily depends on the contributions from
-its community. See
-`contributing <https://instaloader.github.io/contributing.html>`__
-for how you may help Instaloader to become an even greater tool.
-
-It is a pleasure for us to share our Instaloader to the world, and we are proud
-to have attracted such an active and motivating community, with so many users
-who share their suggestions and ideas with us. Buying a community-sponsored beer
-or coffee from time to time is very likely to further raise our passion for the
-development of Instaloader.
-
-| For Donations, we provide a PayPal.Me link and a Bitcoin address.
-|  PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
-|  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY

deploy/arch/.SRCINFO.template

@@ -0,0 +1,15 @@
+pkgbase = instaloader
+	pkgdesc = Command line tool to download pictures, videos and metadata from Instagram
+	pkgver = {{version}}
+	pkgrel = 1
+	url = https://instaloader.github.io/
+	arch = any
+	license = MIT
+	makedepends = python-setuptools
+	depends = python>=3.8
+	depends = python-requests>=2.4
+	options = !emptydirs
+	source = instaloader-{{version}}.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v{{version}}
+	sha512sums = {{hash}}
+
+pkgname = instaloader

deploy/arch/PKGBUILD.template

@@ -0,0 +1,27 @@
+# Maintainer: André Koch-Kramer <koch-kramer@web.de>
+
+pkgname=instaloader
+pkgver={{version}}
+pkgrel=1
+pkgdesc="Command line tool to download pictures, videos and metadata from Instagram"
+arch=('any')
+url="https://instaloader.github.io/"
+license=('MIT')
+groups=()
+depends=('python>=3.8' 'python-requests>=2.4')
+makedepends=('python-setuptools')
+options=('!emptydirs')
+source=($pkgname-$pkgver.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v$pkgver)
+sha512sums=('{{hash}}')
+
+build() {
+	cd "$srcdir/$pkgname-$pkgver"
+	python setup.py clean --all
+	python setup.py build
+}
+
+package() {
+	cd "$srcdir/$pkgname-$pkgver"
+	install -Dm644 LICENSE "$pkgdir"/usr/share/licenses/$pkgname/LICENSE
+	python setup.py install --root "$pkgdir/" --skip-build
+}

deploy/arch/deploy.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+
+VERSION=${1:1}
+
+# do not deploy pre-releases
+echo $VERSION | grep -qe "[abc]" && exit 0
+
+cd $(dirname $0)
+
+# decrypt and add ssh key
+openssl aes-256-cbc -K $AUR_KEY -iv $AUR_IV -in id_rsa_AUR.enc -out /tmp/AUR_openssh -d
+eval "$(ssh-agent -s)"
+chmod 600 /tmp/AUR_openssh
+ssh-add /tmp/AUR_openssh
+mkdir -p ~/.ssh
+ssh-keyscan -t rsa aur.archlinux.org >> ~/.ssh/known_hosts
+
+# clone and modify AUR repo
+git clone --depth 1 ssh://aur@aur.archlinux.org/instaloader.git
+
+curl -sSfOJ https://codeload.github.com/instaloader/instaloader/tar.gz/$1
+HASH=$(sha512sum instaloader-$VERSION.tar.gz | cut -f1 -d " ")
+sed -e "s/{{version}}/$VERSION/g" -e "s/{{hash}}/$HASH/g" PKGBUILD.template > instaloader/PKGBUILD
+sed -e "s/{{version}}/$VERSION/g" -e "s/{{hash}}/$HASH/g" .SRCINFO.template > instaloader/.SRCINFO
+
+# commit and push changes
+cd instaloader
+git config user.email "koch-kramer@web.de"
+git config user.name "André Koch-Kramer"
+git add .
+git commit -m "Release of version $VERSION"
+git push

deploy/windows/create_exe.py

@@ -0,0 +1,71 @@
+import hashlib
+import os
+import re
+import shutil
+import subprocess
+import sys
+
+# copy the required files into repo root
+shutil.copy('docs/favicon.ico', '.')
+shutil.copy('deploy/windows/instaloader.spec', '.')
+
+code = """
+import contextlib
+import psutil
+import subprocess
+
+def __main():
+    with contextlib.suppress(AttributeError, psutil.Error):
+        if psutil.Process().parent().parent().name() == "explorer.exe":
+            subprocess.Popen("powershell -NoExit -Command \\\"& '{0}'\\\"".format(sys.argv[0]))
+            return
+    main()
+
+if __name__ == "__main__":
+    __main()
+"""
+
+with open('instaloader/__main__.py', 'r') as f:
+    # adjust imports for changed file structure
+    regex = re.compile(r'from (?:(\.[^ ]+ )|\.( ))import')
+    lines = [regex.sub(r'from instaloader\1\2import', line) for line in f.readlines()]
+
+    # insert code for magic exe behavior
+    index = lines.index('if __name__ == "__main__":\n')
+    code_lines = [cl + '\n' for cl in code.splitlines()]
+    for i, code_line in enumerate(code_lines):
+        if i + index < len(lines):
+            lines[i + index] = code_line
+        else:
+            lines.extend(code_lines[i:])
+            break
+
+with open('__main__.py', 'w+') as f:
+    f.writelines(lines)
+
+# install dependencies and invoke PyInstaller
+commands = ["pip install pipenv==2023.12.1",
+            f"pipenv --python {sys.version_info.major}.{sys.version_info.minor} sync --dev",
+            "pipenv run pyinstaller --log-level=DEBUG instaloader.spec"]
+
+for command in commands:
+    print()
+    print('#' * (len(command) + 6))
+    print('## {} ##'.format(command))
+    print('#' * (len(command) + 6))
+    print(flush=True)
+    err = subprocess.Popen(command).wait()
+    if err != 0:
+        sys.exit(err)
+
+# calculate and store MD5 hash for created executable
+hash_md5 = hashlib.md5()
+with open('dist/instaloader.exe', 'rb') as f:
+    for chunk in iter(lambda: f.read(4096), b''):
+        hash_md5.update(chunk)
+
+with open('dist/instaloader.exe.md5', 'w+') as f:
+    f.write('{} *instaloader.exe\n'.format(hash_md5.hexdigest()))
+
+# Create ZIP file
+shutil.make_archive('instaloader-{}-windows-standalone'.format(os.getenv('VERSION_TAG')), 'zip', 'dist')

deploy/windows/instaloader.spec

@@ -0,0 +1,28 @@
+# -*- mode: python -*-
+
+block_cipher = None
+
+
+a = Analysis(['__main__.py'],
+             binaries=[],
+             datas=[],
+             hiddenimports=[],
+             hookspath=[],
+             runtime_hooks=[],
+             excludes=[],
+             win_no_prefer_redirects=False,
+             win_private_assemblies=False,
+             cipher=block_cipher)
+pyz = PYZ(a.pure, a.zipped_data,
+             cipher=block_cipher)
+exe = EXE(pyz,
+          a.scripts,
+          a.binaries,
+          a.zipfiles,
+          a.datas,
+          name='instaloader',
+          debug=False,
+          strip=False,
+          upx=False,
+          runtime_tmpdir=None,
+          console=True , icon='favicon.ico')

docs/_static/style.css

@@ -1,28 +0,0 @@
-code {
-    color: #222;
-    background-color: #f5f5f5;
-}
-
-dd {
-    padding-left: 8px;
-}
-
-dl.option dd {
-    padding-left: 12px;
-}
-
-a {
-    color: #008d06;
-}
-
-a:hover {
-    color: #f48400;
-}
-
-body {
-    font-size: 16px;
-}
-
-.versionmodified {
-    font-size: 14px;
-}

docs/_templates/globaltoc.html

@@ -1,14 +0,0 @@
-<li class="dropdown globaltoc-container">
-  <a role="button"
-     id="dLabelGlobalToc"
-     data-toggle="dropdown"
-     data-target="#"
-     href="{{ pathto(master_doc) }}">{{ theme_navbar_site_name }} <b class="caret"></b></a>
-  <ul class="dropdown-menu globaltoc"
-      role="menu"
-      aria-labelledby="dLabelGlobalToc">
-    {{ toctree(maxdepth=theme_globaltoc_depth|toint, collapse=False, includehidden=theme_globaltoc_includehidden|tobool) }}
-    <li role="separator" class="divider"></li>
-    <li class="toctree-l1"><a class="reference internal" href="genindex.html">Index</a></li>
-  </ul>
-</li>

docs/_templates/layout.html

@@ -1,18 +1,83 @@
 {% extends "!layout.html" %}
+{% set html5_doctype = True %}
+{% set script_files = script_files + ['_static/instaloaderdoc.js'] %}
 {% block htmltitle %}
   {% if pagename == "index" %}
-  <title>{{ title|striptags|e }}</title>
+    <title>{{ title|striptags|e }}</title>
   {% else %}
-  <title>{{ title|striptags|e }}{{ titlesuffix }}</title>
+    <title>{{ title|striptags|e }}{{ titlesuffix }}</title>
   {% endif %}
 {% endblock %}
 {% block extrahead %}
-{{ super() }}
-  <meta name="google-site-verification" content="FNUsDdkOJM9obm9QEdrNBVd9AtAzHzA-aaSMCUrS6C8" />
+  {{ super() }}
+  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
+  <meta name="google-site-verification" content="FNUsDdkOJM9obm9QEdrNBVd9AtAzHzA-aaSMCUrS6C8"/>
   {% if pagename == "index" %}
-  <link rel="canonical" href="https://instaloader.github.io/" />
+    <link rel="canonical" href="https://instaloader.github.io/"/>
   {% else %}
-  <link rel="canonical" href="https://instaloader.github.io/{{ pagename }}.html" />
+    <link rel="canonical" href="https://instaloader.github.io/{{ pagename }}.html"/>
   {% endif %}
-  <script async defer src="https://buttons.github.io/buttons.js"></script>
+{% endblock %}
+{% block header %}
+  <nav class="navbar navbar-light navbar-expand bg-light border-bottom">
+    <a class="navbar-brand" href="{{ pathto(master_doc) }}">
+      <img src="{{ pathto('_static/logo.png', 1) }}" width="30" height="30" class="d-inline-block align-top" alt="logo">
+      <span class="ml-3">Instaloader</span>
+    </a>
+    <ul class="navbar-nav mr-auto">
+      <li class="nav-item ml-0 ml-sm-2">
+        <a class="nav-link"
+           href="https://github.com/instaloader/instaloader/releases/tag/v{{ current_release }}"
+           title="Released on {{ current_release_date }}" data-toggle="tooltip">
+          v<b>{{ current_release }}</b>
+        </a>
+      </li>
+    </ul>
+    <ul class="navbar-nav ml-auto d-none d-sm-flex">
+      <li class="nav-item">
+        <a class="nav-link" href="https://github.com/instaloader/instaloader/issues">
+          Issues
+        </a>
+      </li>
+      <li class="nav-item">
+        <a class="nav-link" href="https://github.com/instaloader/instaloader/releases">
+          Releases
+        </a>
+      </li>
+      <li class="nav-item">
+        <a class="nav-link" href="https://github.com/instaloader/instaloader">
+          GitHub
+        </a>
+      </li>
+    </ul>
+    <button class="d-inline-block d-md-none btn btn-outline-primary btn-sm" type="button"
+            data-toggle="collapse" data-target="#instaloaderdocSidebar" aria-controls="instaloaderdocSidebar"
+            aria-expanded="false" aria-label="Toggle navigation">
+      <span class="navbar-toggler-icon"></span>
+    </button>
+  </nav>
+{% endblock %}
+{% block relbar1 %}{% endblock %}
+{% block relbar2 %}{% endblock %}
+{% block footer %}{% endblock %}
+{% block content %}
+  <div class="container-fluid">
+    <div class="row">
+      <div class="instaloaderdoc-sidebar collapse d-md-block col-md-3 col-xl-2 bg-light"
+           id="instaloaderdocSidebar">
+        <nav role="navigation" aria-label="Table of Contents">
+          <form action="{{ pathto('search') }}" method="get" role="search" class="border-bottom mr-2 pb-2 mb-2 px-1">
+            <label class="sr-only" for="searchQueryInput">Search</label>
+            <input type="search" name="q" id="searchQueryInput" class="form-control my-2"
+                   placeholder="Search..." aria-label="Search">
+          </form>
+          <h4>Contents</h4>
+          {{ toctree(maxdepth=-1) }}
+        </nav>
+      </div>
+      <main class="col-12 col-md-9 col-xl-10 pt-2" role="main">
+        {% block body %}{% endblock %}
+      </main>
+    </div>
+  </div>
 {% endblock %}

docs/_templates/navbar.html

@@ -1,60 +0,0 @@
-  <div id="navbar" class="{{ theme_navbar_class }} navbar-default {% if theme_navbar_fixed_top|tobool -%} navbar-fixed-top{%- endif -%}">
-    <div class="container">
-      <div class="navbar-header">
-        <!-- .btn-navbar is used as the toggle for collapsed navbar content -->
-        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">
-          <span class="icon-bar"></span>
-          <span class="icon-bar"></span>
-          <span class="icon-bar"></span>
-        </button>
-        <a class="navbar-brand" href="{{ pathto(master_doc) }}">
-          {%- block sidebarlogo %}
-            {%- if logo %}<span><img src="{{ pathto('_static/' + logo, 1) }}"></span>{%- endif %}
-          {%- endblock %}
-          {% if theme_navbar_title -%}{{ theme_navbar_title|e }}{%- else -%}{{ project|e }}{%- endif -%}
-        </a>
-        <span class="navbar-text navbar-version pull-left">
-          <a href="https://github.com/instaloader/instaloader/releases/tag/v{{current_release}}" title="Released on {{current_release_date}}">v<b>{{current_release}}</b></a>
-        </span>
-      </div>
-
-        <div class="collapse navbar-collapse nav-collapse">
-          <ul class="nav navbar-nav">
-            {% if theme_navbar_links %}
-              {%- for link in theme_navbar_links %}
-                <li><a href="{{ pathto(*link[1:]) }}">{{ link[0] }}</a></li>
-              {%- endfor %}
-            {% endif %}
-            {% block navbartoc %}
-              {% include "globaltoc.html" %}
-              {% if theme_navbar_pagenav and pagename not in ("index", "genindex", "installation") %}
-                {% include "navbartoc.html" %}
-              {% endif %}
-            {% endblock %}
-            {% if theme_navbar_sidebarrel %}
-              {% block sidebarrel %}
-                {% include "relations.html" %}
-              {% endblock %}
-            {% endif %}
-            </ul>
-            <ul class="nav navbar-nav navbar-right">
-              <li class="dropdown">
-                <a role="button" id="dLabelUsefulLinks" data-toggle="dropdown" data-target="#" href="#">
-                  Useful Links
-                  <b class="caret"></b>
-                </a>
-                <ul class="dropdown-menu" role="menu" aria-labelledby="dLabelUsefulLinks">
-                  <li><a href="https://github.com/instaloader/instaloader/issues">Issue Tracker</a></li>
-                  <li><a href="https://github.com/instaloader/instaloader/releases">Version History</a></li>
-                  <li><a href="https://pypi.org/project/instaloader/">Instaloader on PyPI</a></li>
-                  <li><a href="https://github.com/instaloader/instaloader">Instaloader on GitHub</a></li>
-                  <li role="separator" class="divider"></li>
-                  <li style="padding-left: 20px;"><a class="github-button" href="https://github.com/instaloader/instaloader" data-icon="octicon-star" data-size="large" data-show-count="true" aria-label="Star Instaloader on GitHub">Star</a></li>
-                  <li role="separator" class="divider"></li>
-                  <li><a href="https://www.paypal.me/aandergr"><b>Donate (via Paypal)</b></a></li>
-                </ul>
-              </li>
-            </ul>
-        </div>
-    </div>
-  </div>

docs/_templates/search.html

@@ -0,0 +1,28 @@
+{% extends "!search.html" %}
+
+{% block body %}
+  <div id="fallback" class="admonition warning">
+    <script type="text/javascript">$('#fallback').hide();</script>
+    <p>
+      {% trans %}Please activate JavaScript to enable the search
+        functionality.{% endtrans %}
+    </p>
+  </div>
+  {% if search_performed %}
+    <h1>{{ _('Search Results') }}</h1>
+    {% if not search_results %}
+      <p>{{ _('Your search did not match any documents. Please make sure that all words are spelled correctly and that you\'ve selected enough categories.') }}</p>
+    {% endif %}
+  {% endif %}
+  <div id="search-results">
+    {% if search_results %}
+      <ul>
+        {% for href, caption, context in search_results %}
+          <li><a href="{{ pathto(item.href) }}">{{ caption }}</a>
+            <div class="context">{{ context|e }}</div>
+          </li>
+        {% endfor %}
+      </ul>
+    {% endif %}
+  </div>
+{% endblock %}

docs/as-module.rst

@@ -12,11 +12,8 @@ Python Module :mod:`instaloader`
 
 .. highlight:: python
 
-.. contents::
-  :backlinks: none
-
 Instaloader exposes its internally used methods and structures as a Python
-module, making it a **powerful and intuitive Python API for Instagram**,
+module, making it a powerful and intuitive Python API for Instagram,
 allowing to further customize obtaining media and metadata.
 
 Start with getting an instance of :class:`Instaloader`::
@@ -37,7 +34,7 @@ picture, video or sidecar (set of multiple pictures/videos) posted in a user's
 profile. :class:`Instaloader` provides methods to iterate over Posts from a
 certain source::
 
-    for post in L.get_hashtag_posts('cat'):
+    for post in instaloader.Hashtag.from_name(L.context, 'cat').get_posts():
         # post is an instance of instaloader.Post
         L.download_post(post, target='#cat')
 
@@ -63,7 +60,7 @@ certain source::
 - :meth:`Instaloader.get_explore_posts`
    Media that is suggested by Instagram to explore.
 
-- :meth:`Instaloader.get_hashtag_posts`
+- :meth:`Hashtag.get_posts`
    Media associated with given hashtag.
 
 With the :class:`Profile` class, Instaloader also makes it easy to access
@@ -85,119 +82,26 @@ metadata of a Profile. :class:`Profile` instances can be created with:
 - :meth:`Profile.get_followers`
    Profiles that follow given user.
 
-- :attr:`Post.owner_profile`, :attr:`Story.owner_profile` and :attr:`StoryItem.owner_profile`
+- :attr:`Post.owner_profile`, :attr:`Story.owner_profile`, :attr:`StoryItem.owner_profile`
    Owner profile of particular object.
 
 - :meth:`Post.get_likes`
    Profiles that liked a given :class:`Post`
 
-- :attr:`PostComment.owner` attribute for comment in :meth:`Post.get_comments`
+- :attr:`PostComment.owner`
    Profile of a Post comment.
 
-A reference of the many methods provided by the :mod:`instaloader` module is
-provided in the remainder of this document.
+For a list of a few code examples that use the Instaloader module, see
+:ref:`codesnippets`.
 
-For a list of real code examples using the Instaloader module for advanced
-tasks, see :ref:`codesnippets`.
+The reference of the classes and functions provided by the :mod:`instaloader` module is
+divided into the following subsections:
 
-``Instaloader`` (Main Class)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. toctree::
+   :maxdepth: 2
 
-.. autoclass:: Instaloader
-   :no-show-inheritance:
-
-Instagram Structures
-^^^^^^^^^^^^^^^^^^^^
-
-Posts
-"""""
-
-.. autoclass:: Post
-   :no-show-inheritance:
-
-Additionally, the following trivial structures are defined:
-
-.. autoclass:: PostSidecarNode
-   :no-show-inheritance:
-
-.. autoclass:: PostComment
-   :no-show-inheritance:
-
-.. autoclass:: PostLocation
-   :no-show-inheritance:
-
-User Stories
-""""""""""""
-
-.. autoclass:: Story
-   :no-show-inheritance:
-
-.. autoclass:: StoryItem
-   :no-show-inheritance:
-
-Highlights
-""""""""""
-
-.. autoclass:: Highlight
-   :no-show-inheritance:
-   :inherited-members:
-
-   Bases: :class:`Story`
-
-   .. versionadded:: 4.1
-
-Profiles
-""""""""
-
-.. autoclass:: Profile
-   :no-show-inheritance:
-
-Loading and Saving
-""""""""""""""""""
-
-:class:`Post`, :class:`StoryItem` and :class:`Profile` can be saved and loaded
-to/from JSON files.
-
-.. autofunction:: load_structure_from_file
-
-.. autofunction:: save_structure_to_file
-
-Exceptions
-^^^^^^^^^^
-
-.. currentmodule:: instaloader.exceptions
-
-.. autoexception:: InstaloaderException
-   :no-show-inheritance:
-
-.. autoexception:: ConnectionException
-
-.. currentmodule:: instaloader
-
-.. autoexception:: QueryReturnedBadRequestException
-
-.. autoexception:: QueryReturnedNotFoundException
-
-.. autoexception:: QueryReturnedForbiddenException
-
-.. autoexception:: ProfileNotExistsException
-
-.. autoexception:: ProfileHasNoPicsException
-
-.. autoexception:: PrivateProfileNotFollowedException
-
-.. autoexception:: LoginRequiredException
-
-.. autoexception:: InvalidArgumentException
-
-.. autoexception:: BadResponseException
-
-.. autoexception:: BadCredentialsException
-
-.. autoexception:: TooManyRequestsException
-
-``InstaloaderContext`` (Low-level functions)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. autoclass:: InstaloaderContext
-   :no-show-inheritance:
+   module/instaloader
+   module/structures
+   module/nodeiterator
+   module/instaloadercontext
+   module/exceptions

docs/basic-usage.rst

@@ -39,6 +39,16 @@ To later **update your local copy** of that profiles, you may run
 If :option:`--fast-update` is given, Instaloader stops when arriving at the
 first already-downloaded picture.
 
+Alternatively, you can use :option:`--latest-stamps` to have Instaloader store
+the time each profile was last downloaded and only download newer media:
+
+::
+
+    instaloader --latest-stamps -- profile [profile ...]
+
+With this option it's possible to move or delete downloaded media and still keep
+the archive updated.
+
 When updating profiles, Instaloader
 automatically **detects profile name changes** and renames the target directory
 accordingly.
@@ -51,7 +61,7 @@ invoke it with
     instaloader --login=your_username profile [profile ...]
 
 When logging in, Instaloader **stores the session cookies** in a file in your
-temporary directory, which will be reused later the next time :option:`--login`
+home directory, which will be reused later the next time :option:`--login`
 is given.  So you can download private profiles **non-interactively** when you
 already have a valid session cookie file.
 
@@ -65,7 +75,7 @@ What to Download
 Instaloader supports the following targets:
 
 - ``profile``
-   Public profile, or private profile with :option:`--login`.
+   Public profile, or private profile with :ref:`login<login>`.
 
    If an already-downloaded profile has been renamed, Instaloader automatically
    finds it by its unique ID and renames the folder accordingly.
@@ -77,31 +87,46 @@ Instaloader supports the following targets:
       instructs Instaloader to also **download the user's stories**,
 
    - :option:`--highlights`
-      to **download highlights of each profile that is downloaded**, and
+      to **download the highlights of that profile**,
 
    - :option:`--tagged`
-      to **download posts where the user is tagged**.
+      to **download posts where the user is tagged**, and
+
+   - :option:`--igtv`
+      to **download IGTV videos**.
 
 - ``"#hashtag"``
-   Posts with a certain **hashtag** (the quotes are usually necessary),
+   Posts with a certain **hashtag** (the quotes are usually necessary).
+
+- ``%location id``
+   Posts tagged with a given location; the location ID is the numerical ID
+   Instagram labels a location with (e.g.
+   \https://www.instagram.com/explore/locations/**362629379**/plymouth-naval-memorial/).
+   Requires :ref:`login<login>`.
+
+   .. versionadded:: 4.2
 
 - ``:stories``
    The currently-visible **stories** of your followees (requires
-   :option:`--login`),
+   :ref:`login<login>`).
 
 - ``:feed``
-   Your **feed** (requires :option:`--login`),
+   Your **feed** (requires :ref:`login<login>`).
 
 - ``:saved``
-   Posts which are marked as **saved** (requires :option:`--login`),
+   Posts which are marked as **saved** (requires :ref:`login<login>`).
 
 - ``@profile``
    All profiles that are followed by ``profile``, i.e. the *followees* of
-   ``profile`` (requires :option:`--login`).
+   ``profile`` (requires :ref:`login<login>`).
 
 - ``-post``
-   The single **post** with the given shortcode. Must be preceeded by ``--`` in
-   the argument list to not be mistaken as an option flag.
+   Replace **post** with the post's shortcode to download single post. Must be preceded by ``--`` in
+   the argument list to not be mistaken as an option flag::
+
+    instaloader -- -B_K4CykAOtf
+
+
 
    .. versionadded:: 4.1
 
@@ -115,7 +140,7 @@ downloads the pictures and videos and their captions. You can specify
 
 - :option:`--geotags`
    **download geotags** of each post and save them as
-   Google Maps link,
+   Google Maps link (requires :ref:`login<login>`),
 
 For a reference of all supported command line options, see
 :ref:`command-line-options`.
@@ -126,7 +151,7 @@ Filename Specification
 ^^^^^^^^^^^^^^^^^^^^^^
 
 For each target, Instaloader creates a directory named after the target,
-i.e. ``profile``, ``#hashtag``, ``:feed``, etc. and therein saves the
+i.e. ``profile``, ``#hashtag``, ``%location id``, ``:feed``, etc. and therein saves the
 posts in files named after the post's timestamp.
 
 :option:`--dirname-pattern` allows to configure the directory name of each
@@ -134,27 +159,40 @@ target. The default is ``--dirname-pattern={target}``. In the dirname
 pattern, the token ``{target}`` is replaced by the target name, and
 ``{profile}`` is replaced by the owner of the post which is downloaded.
 
-:option:`--filename-pattern` configures the path of the post's files relative
+:option:`--filename-pattern` configures the path of the post and story's files relative
 to the target directory that is specified with :option:`--dirname-pattern`.
 The default is ``--filename-pattern={date_utc}_UTC``.
 The tokens ``{target}`` and ``{profile}`` are replaced like in the
-dirname pattern. The following tokens are defined for usage with
-:option:`--filename-pattern`:
+dirname pattern.
+
+:option:`--title-pattern` is similar to :option:`--filename-pattern`, but for profile
+pics, hashtag profile pics, and highlight covers. The default is
+``{date_utc}_UTC_{typename}`` if :option:`--dirname-pattern` contains ``{target}`` or
+``{profile}``, or ``{target}_{date_utc}_UTC_{typename}`` if it does not. Some tokens
+are not supported for this option, see below for details.
+
+The following tokens are defined for usage with
+:option:`--filename-pattern` and :option:`--title-pattern`:
 
 - ``{target}``
    Target name (as given in Instaloader command line)
 
 - ``{profile}`` (same as ``{owner_username}``)
-   Owner of the Post / StoryItem.
+   Owner of the Post / StoryItem / ProfilePic. For hashtag profile pics and
+   highlight covers, equivalent to ``{target}``.
 
 - ``{owner_id}``
-   Unique integer ID of owner profile.
+   Unique integer ID of owner profile. For hashtag profile pics, equivalent to
+   ``{target}``.
 
 - ``{shortcode}``
-   Shortcode (identifier string).
+   Shortcode (identifier string). Not available for :option:`--title-pattern`.
 
 - ``{mediaid}``
-   Integer representation of shortcode.
+   Integer representation of shortcode. Not available for :option:`--title-pattern`.
+
+- ``{filename}``
+   Instagram's internal filename.
 
 - ``{date_utc}`` (same as ``{date}``)
    Creation time in UTC timezone.
@@ -164,6 +202,10 @@ dirname pattern. The following tokens are defined for usage with
 
       {date_utc:%Y-%m-%d_%H-%M-%S}
 
+- ``{typename}``
+   Type of media being saved, such as GraphImage, GraphStoryVideo, profile_pic,
+   etc.
+
 For example, encode the poster's profile name in the filenames with::
 
     instaloader --filename-pattern={date_utc}_UTC_{profile} "#hashtag"
@@ -181,7 +223,7 @@ Filter Posts
 .. py:currentmodule:: instaloader
 
 The options :option:`--post-filter` and :option:`--storyitem-filter`
-allows to specify criteria that posts or story items have to
+allow to specify criteria that posts or story items have to
 meet to be downloaded. If not given, all posts are downloaded.
 
 The filter string must be a
@@ -192,13 +234,8 @@ where the attributes from :class:`Post` or
 Id est, the following attributes can be used with both
 :option:`--post-filter` and :option:`--storyitem-filter`:
 
-- :attr:`~Post.is_video` (bool)
-   Post/StoryItem is a video. For example, you may skip videos::
-
-      instaloader --post-filter="not is_video" target
-
 - :attr:`~Post.owner_username` (str), :attr:`~Post.owner_id` (int)
-   Owner profile username / userid.
+   Owner profile username / user ID.
 
 - :attr:`~Post.date_utc` (datetime), :attr:`~Post.date_local` (datetime)
    Creation timestamp. Since :class:`~datetime.datetime` objects can be created
@@ -206,17 +243,26 @@ Id est, the following attributes can be used with both
 
       instaloader --post-filter="date_utc <= datetime(2018, 5, 31)" target
 
+- :attr:`~Post.is_video` (bool)
+   Whether Post/StoryItem is a video. For example, you may skip videos::
+
+      instaloader --post-filter="not is_video" target
+
+   This is not the same as :option:`--no-videos` and
+   :option:`--no-video-thumbnails`, since sidecar posts (posts that contain
+   multiple pictures/videos in one post) have this attribute set to False.
+
 As :option:`--post-filter`, the following attributes can be used additionally:
 
 - :attr:`~Post.viewer_has_liked` (bool)
-   Whether user (with :option:`--login`) has liked given post. To download the
+   Whether user (with :ref:`login<login>`) has liked given post. To download the
    pictures from your feed that you have liked::
 
       instaloader --login=your_username --post-filter=viewer_has_liked :feed
 
 - :attr:`~Post.likes` (int), :attr:`~Post.comments` (int)
-   Likes count / Comments count. You might only want to download posts that
-   either you liked or were liked by many others**::
+   Likes count / comments count. You might only want to download posts that
+   were either liked by yourself or by many others::
 
       instaloader --login=your_username --post-filter="likes>100 or viewer_has_liked" profile
 
@@ -246,7 +292,7 @@ along with each post where the Post's caption is saved.
 You can customize what metadata to save for each Post or StoryItem with
 :option:`--post-metadata-txt` and :option:`--storyitem-metadata-txt`. The
 default is ``--post-metadata-txt={caption}`` and no storyitem metadata txt.
-These strings are formatted similar as in the :ref:`filename-specification` and
+These strings are formatted similar as the path patterns described in :ref:`filename-specification` and
 the result is saved in text files, unless it is empty.
 
 Specifying these options multiple times results in output having multiple lines,
@@ -261,13 +307,47 @@ the post's caption::
 
    instaloader --post-metadata-txt="{likes} likes." <target>
 
-Note that with this feature, it is possible to easily and fastly extract
+Note that with this feature, it is possible to easily and quickly extract
 additional metadata of already-downloaded posts, by reimporting their JSON
 files. Say, you now also want to export the number of comments the Posts had
 when they were downloaded::
 
    instaloader --post-metadata-txt="{likes} likes, {comments} comments." <target>/*.json.xz
 
+.. _exit_codes:
+
+Exit codes
+^^^^^^^^^^
+
+Different exit codes are used to indicate different kinds of error:
+
+0
+  No error, all downloads were successful.
+
+1
+  A non-fatal error happened. One or more posts, or even one or more
+  profiles could not be downloaded, but execution was not stopped. The
+  errors are repeated at the end of the log for easy access.
+
+2
+  Command-line error. An unrecognized option was passed, or an invalid
+  combination of options, for example. No interaction with Instagram
+  was made.
+
+3
+  Login error. It was not possible to login. Downloads were not
+  attempted.
+
+4
+  Fatal download error. Downloads were interrupted and no further
+  attempts were made. Happens when a response with one of the status
+  codes in the :option:`--abort-on` option were passed, or when
+  Instagram logs the user out during downloads.
+
+5
+  Interrupted by the user. Happens when the user presses Control-C or
+  sends SIGINT to the process.
+
 .. _instaloader-as-cronjob:
 
 Instaloader as Cronjob
@@ -276,17 +356,18 @@ Instaloader as Cronjob
 Instaloader is suitable for running as a cronjob to periodically update your
 personal Instagram archive. The :option:`--quiet` option disables user
 interactions and logging of non-error messages. To non-interactively use
-Instaloader logged-in, create a session file, e.g. in your home directory::
+Instaloader logged-in, create a session file::
 
-   instaloader --login=your_username --sessionfile=~/.instaloader-session
+   instaloader --login=your_username
 
-Then use the same parameters in your cronjob to load the session and download
+Then use the same username in your cronjob to load the session and download
 the given targets::
 
-   instaloader --login=your_username --sessionfile=~/.instaloader-session --quiet <target> [...]
+   instaloader --login=your_username --quiet target [...]
 
-Without :option:`--sessionfile` option, Instaloader saves the session file in
-a path within your temporary directory.
+Instaloader saves the session file to
+``~/.config/instaloader/session-YOUR-USERNAME``. See
+:option:`--sessionfile` option for how to override this path.
 
 Programming Instaloader
 ^^^^^^^^^^^^^^^^^^^^^^^
@@ -297,5 +378,6 @@ Instaloader exposes its internally used methods and structures, making it a
 powerful and intuitive Python API for Instagram, allowing to further customize
 obtaining media and metadata.
 
-Also see :ref:`codesnippets`, where we collect example scripts that use
-Instaloader to achieve more complex tasks.
+Also see :ref:`codesnippets`, where we collect a few example scripts that use
+Instaloader for simple tasks that cannot be done with the command line
+interface.

docs/cli-options.rst

@@ -8,7 +8,7 @@ Instaloader is invoked with::
    $ instaloader [options] target [target ...]
 
 where ``target`` is a ``profile``, a ``"#hashtag"``, ``@profile`` (all profiles
-that *profile* is following), or if logged in ``:feed`` (pictures from your
+that *profile* is following), ``%location ID``, or if logged in ``:feed`` (pictures from your
 feed), ``:stories`` (stories of your followees) or ``:saved`` (collection of
 posts marked as saved).
 
@@ -26,7 +26,7 @@ Targets
 ^^^^^^^
 
 Specify a list of targets. For each of these, Instaloader creates a folder and
-stores all posts along with the pictures's captions there.
+stores all posts along with the pictures' captions there.
 
 .. include:: basic-usage.rst
    :start-after: targets-start
@@ -36,7 +36,7 @@ stores all posts along with the pictures's captions there.
    Re-Download the given object
 
 - ``+args.txt``
-   Read targets (and options) from given textfile. See :option:`+args.txt`.
+   Read targets (and options) from given text file. See :option:`+args.txt`.
 
 What to Download of each Post
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -59,16 +59,15 @@ What to Download of each Post
 
 .. option:: --geotags, -G
 
-   **Download geotags** when available. Geotags are stored as a text file with
+   Download geotags when available. Geotags are stored as a text file with
    the location's name and a Google Maps link. This requires an additional
-   request to the Instagram server for each picture, which is why it is disabled
-   by default.
+   request to the Instagram server for each picture. Requires :ref:`login<login>`.
 
 .. option:: --comments, -C
 
    Download and update comments for each post. This requires an additional
    request to the Instagram server for each post, which is why it is disabled by
-   default.
+   default. Requires :ref:`login<login>`.
 
 .. option:: --no-captions
 
@@ -83,6 +82,16 @@ What to Download of each Post
    Template to write in txt file for each StoryItem. See
    :ref:`metadata-text-files`.
 
+.. option:: --slide
+
+   Download only selected images of a sidecar. You can select single images using their
+   index in the sidecar starting with the leftmost or you can specify a range of images
+   with the following syntax: ``start_index-end_index``. Example:
+   ``--slide 1`` will select only the first image, ``--slide last`` only the last one and ``--slide 1-3`` will select only
+   the first three images.
+
+   .. versionadded:: 4.6
+
 .. option:: --no-metadata-json
 
    Do not create a JSON file containing the metadata of each post.
@@ -91,17 +100,9 @@ What to Download of each Post
 
    Do not xz compress JSON files, rather create pretty formatted JSONs.
 
-
 What to Download of each Profile
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-.. option:: --profile-pic-only, -P
-
-   .. deprecated:: 4.1
-      Use :option:`--no-posts`.
-
-   Only download profile picture.
-
 .. option:: --no-posts
 
    Do not download regular posts.
@@ -114,13 +115,13 @@ What to Download of each Profile
 
 .. option:: --stories, -s
 
-   Also **download stories** of each profile that is downloaded. Requires
-   :option:`--login`.
+   Also download stories of each profile that is downloaded. Requires
+   :ref:`login<login>`.
 
 .. option:: --highlights
 
-   Also **download highlights** of each profile that is downloaded. Requires
-   :option:`--login`.
+   Also download highlights of each profile that is downloaded. Requires
+   :ref:`login<login>`.
 
    .. versionadded:: 4.1
 
@@ -130,19 +131,11 @@ What to Download of each Profile
 
    .. versionadded:: 4.1
 
-.. option:: --stories-only
+.. option:: --igtv
 
-   .. deprecated:: 4.1
-      Use :option:`--stories` :option:`--no-posts`.
+   Also download IGTV videos.
 
-   Rather than downloading regular posts of each specified profile, only
-   download stories.  Requires :option:`--login`. Does not imply
-   :option:`--no-profile-pic`.
-
-   .. note::
-
-      If possible, use ``:stories`` target rather than :option:`--stories-only`
-      with all your followees. ``:stories`` uses fewer API requests.
+   .. versionadded:: 4.3
 
 Which Posts to Download
 ^^^^^^^^^^^^^^^^^^^^^^^
@@ -153,6 +146,23 @@ Which Posts to Download
    This flag is recommended when you use Instaloader to update your personal
    Instagram archive.
 
+.. option:: --latest-stamps [STAMPSFILE]
+
+   Works similarly to :option:`--fast-update`, but instead of relying on already
+   downloaded media, the time each profile was downloaded is stored, and only
+   media newer than the last download is fetched. This allows updating your
+   personal Instagram archive while emptying the target directories.
+
+   Only works for media associated with a specific profile, and that is returned
+   in chronological order: profile posts, profile stories, profile IGTV posts
+   and profile tagged posts.
+
+   By default, the information is stored in
+   ``~/.config/instaloader/latest-stamps.ini``, but you can specify an
+   alternative location.
+
+   .. versionadded:: 4.8
+
 .. option:: --post-filter filter, --only-if filter
 
    Expression that, if given, must evaluate to True for each post to be
@@ -170,27 +180,49 @@ Which Posts to Download
 
 .. option:: --count COUNT, -c
 
-   Do not attempt to download more than COUNT posts.  Applies only to
-   ``#hashtag`` and ``:feed``.
+   Do not attempt to download more than COUNT posts.  Applies to
+   ``#hashtag``, ``%location_id``, ``:feed``, and ``:saved``.
 
+.. _login:
 
 Login (Download Private Profiles)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Instaloader can **login to Instagram**. This allows downloading private
+Instaloader can login to Instagram. This allows downloading private
 profiles. To login, pass the :option:`--login` option. Your session cookie (not your
 password!) will be saved to a local file to be reused next time you want
 Instaloader to login.
 
+Instead of :option:`--login`, it is possible to use
+:option:`--load-cookies` to import a session from a browser.
+
 .. option:: --login YOUR-USERNAME, -l YOUR-USERNAME
 
    Login name (profile name) for your Instagram account.
 
+.. option:: --load-cookies BROWSER-NAME, -b BROWSER-NAME
+
+   Use Instagram cookie in your browser to login.
+   This feature requires the browser_cookie3 library.
+   Compatible with :option:`--cookiefile` if you want to load cookies from browser profiles.
+   Incompatible with :option:`--login` due to potential username mismatch between user input and browser login.
+   Supported browsers: Brave, Chrome, Chromium, Edge, Firefox, LibreWolf, Opera, Opera_GX, Safari and Vivaldi.
+
+   In subsequent runs, you can just use :option:`--login` to reuse the
+   same session, which is saved by Instaloader.
+
+   .. versionadded:: 4.11
+
+.. option:: --cookiefile COOKIE-FILE, -B COOKIE-FILE
+
+   Cookie file path of a browser profile to load cookies from.
+
+  .. versionadded:: 4.11
+
 .. option:: --sessionfile SESSIONFILE, -f SESSIONFILE
 
-   Path for loading and storing session key file.  Defaults to a path within
-   your temporary directory, encoding your local username and your Instagram
-   profile name.
+   Path for loading and storing session key file.  Defaults to
+   ``~/.config/instaloader/session-YOUR-USERNAME``.
 
 .. option:: --password YOUR-PASSWORD, -p YOUR-PASSWORD
 
@@ -213,16 +245,66 @@ How to Download
 
 .. option:: --filename-pattern FILENAME_PATTERN
 
-   Prefix of filenames, relative to the directory given with
+   Prefix of filenames for posts and stories, relative to the directory given with
    :option:`--dirname-pattern`. ``{profile}`` is replaced by the profile name,
    ``{target}`` is replaced by the target you specified, i.e.  either ``:feed``,
    ``#hashtag`` or the profile name. Defaults to ``{date_utc}_UTC``.
    See :ref:`filename-specification` for a list of supported tokens.
 
+.. option:: --title-pattern TITLE_PATTERN
+
+   Prefix of filenames for profile pics, hashtag profile pics, and highlight
+   covers, relative to the directory given with :option:`--dirname-pattern`.
+   Defaults to ``{date_utc}_UTC_{typename}`` if :option:`--dirname-pattern`
+   contains ``{target}`` or ``{profile}``, otherwise defaults to
+   ``{target}_{date_utc}_UTC_{typename}``.
+   See :ref:`filename-specification` for a list of supported tokens.
+
+   .. versionadded:: 4.8
+
+.. option:: --sanitize-paths
+
+   Force sanitization of paths so that the resulting file and directory names
+   are valid on both Windows and Unix.
+
+   .. versionadded:: 4.9
+
+.. option:: --resume-prefix prefix
+
+   For many targets, Instaloader is capable of resuming a previously-aborted
+   download loop.  To do so, it creates a JSON file within the target directory
+   when interrupted.  This option controls the prefix for filenames that are
+   used to save the information to resume an interrupted download.  The default
+   prefix is ``iterator``.
+
+   Resuming an interrupted download is supported for the following targets:
+    - Profile posts,
+    - Profile IGTV posts (:option:`--igtv`),
+    - Profile tagged posts (:option:`--tagged`),
+    - Saved posts (``:saved``),
+    - Hashtags.
+
+   This feature is enabled by default for targets where it is supported;
+   :option:`--resume-prefix` only changes the name of the iterator files.
+
+   To turn this feature off, use :option:`--no-resume`.
+
+   JSON files with resume information are always compressed, regardless of
+   :option:`--no-compress-json`.
+
+   .. versionadded:: 4.5
+
+.. option:: --no-resume
+
+   Do not resume a previously-aborted download iteration, and do not save such
+   information when interrupted.
+
+   .. versionadded:: 4.5
+
 .. option:: --user-agent USER_AGENT
 
    User Agent to use for HTTP requests. Per default, Instaloader pretends being
-   Chrome/51.
+   Chrome/120 on Linux.
 
 .. option:: --max-connection-attempts N
 
@@ -230,6 +312,32 @@ How to Download
    to ``3``. If a connection fails, it can be manually skipped by hitting
    :kbd:`Control-c`. Set this to ``0`` to retry infinitely.
 
+.. option:: --request-timeout N
+
+   Seconds to wait before timing out a connection request. Defaults to 300.
+
+   .. versionadded:: 4.3
+
+   .. versionchanged:: 4.6
+      Enabled this option by default with a timeout of 300 seconds.
+
+.. option:: --abort-on STATUS_CODE_LIST
+
+   Comma-separated list of HTTP status codes that cause Instaloader to abort,
+   bypassing all retry logic.
+
+   For example, with ``--abort-on=302,400,429``, Instaloader will stop if a
+   request is responded with a 302 redirect, a Bad Request error, or a Too Many
+   Requests error.
+
+   .. versionadded:: 4.7
+
+.. option:: --no-iphone
+
+   Do not attempt to download iPhone version of images and videos.
+
+   .. versionadded:: 4.8
+
 Miscellaneous Options
 ^^^^^^^^^^^^^^^^^^^^^
 
@@ -241,18 +349,21 @@ Miscellaneous Options
 
 .. option:: +args.txt
 
-   Read arguments from file `args.txt`, a shortcut to provide argument from
-   file rather than command-line. This provide a convient way to hide login
-   info from CLI. and also can use for simplify managment of long arguments.
+   Read arguments from file `args.txt`, a shortcut to provide arguments from
+   file rather than command-line. This provides a convenient way to hide login
+   info from CLI, and can also be used to simplify management of long arguments.
+   You can provide more than one file at once, e.g.: ``+args1.txt +args2.txt``.
 
    .. note::
 
-      text file should separate arg with line break.
+      Text file should separate arguments with line breaks.
 
-      args.txt example::
+   `args.txt` example::
 
-         --login MYUSENAME
-         --password MYPASSWORD
-         --fast-update
+      --login=MYUSERNAME
+      --password=MYPASSWORD
+      --fast-update
+      profile1
+      profile2
 
    .. versionadded:: 4.1

docs/codesnippets.rst

@@ -14,6 +14,9 @@ Here we present code examples that use the :ref:`python-module-instaloader` for
 more advanced Instagram downloading or metadata mining than what is possible
 with the Instaloader command line interface.
 
+The scripts presented here can be downloaded from our source tree:
+`instaloader/docs/codesnippets/ <https://github.com/instaloader/instaloader/tree/master/docs/codesnippets>`__
+
 .. For each code snippet:
    - title
    - brief description of what it does / motivation / how it works
@@ -25,8 +28,9 @@ Download Posts in a Specific Period
 -----------------------------------
 
 To only download Instagram pictures (and metadata) that are within a specific
-period, you can play around with :func:`~itertools.dropwhile` and
-:func:`~itertools.takewhile` from :mod:`itertools` like in this snippet.
+period, you can simply use :func:`~itertools.dropwhile` and
+:func:`~itertools.takewhile` from :mod:`itertools` on a generator that returns
+Posts in **exact chronological order**, such as :meth:`Profile.get_posts`.
 
 .. literalinclude:: codesnippets/121_since_until.py
 
@@ -34,11 +38,19 @@ See also :class:`Post`, :meth:`Instaloader.download_post`.
 
 Discussed in :issue:`121`.
 
+The code example with :func:`~itertools.dropwhile` and
+:func:`~itertools.takewhile` makes the assumption that the post iterator returns
+posts in exact chronological order.  As discussed in :issue:`666`, the following
+approach fits for an **almost chronological order**, where up to *k* older posts
+are inserted into an otherwise chronological order, such as a Hashtag feed.
+
+.. literalinclude:: codesnippets/666_historical_hashtag_data.py
+
 Likes of a Profile / Ghost Followers
 ------------------------------------
 
 To obtain a list of your inactive followers, i.e. followers that did not like
-any of your pictures, into a file you can use this approach.
+any of your pictures, you can use this approach.
 
 .. literalinclude:: codesnippets/120_ghost_followers.py
 
@@ -66,11 +78,11 @@ Discussed in :issue:`56`.
 Only one Post per User
 ----------------------
 
-To download only the one most recent post from each user, this snippet creates a
-:class:`set` that contains the users of which a post has already been
-downloaded. While iterating the posts, it checks whether the post's owner
-already is in the set. If not, the post is downloaded from Instagram and the
-user is added to that set.
+To download only the single most recent post per user within a hashtag feed,
+this snippet uses a :class:`set` that contains the users of whom a post has
+already been downloaded. For each post, it checks whether the post's creator is
+already contained in that set. If not, the post is downloaded from Instagram and
+the user is added to that set.
 
 .. literalinclude:: codesnippets/113_only_one_per_user.py
 
@@ -79,30 +91,14 @@ See also :class:`Post`, :meth:`Instaloader.download_post`,
 
 Discussed in :issue:`113`.
 
-Upgrade Images by Local Copies
-------------------------------
+Top X Posts of User
+-------------------
 
-The following script finds local versions of images fetched by Instaloader, in
-order to upgrade the downloaded images by locally-found versions with better
-quality. It uses image hashing to identify similar images.
+With Instaloader, it is easy to download the few most-liked pictures of a user.
 
-`updgrade-instaloader-images.py <https://gist.github.com/pavelkryukov/15f93d19a99428a284a8bcec27e0187b>`__ (external link to GitHub Gist)
+.. literalinclude:: codesnippets/194_top_x_of_user.py
 
-Discussed in :issue:`46`.
-
-Add Captions to Images
-----------------------
-
-Instaloader does not modify the downloaded JPEG file. However, one could combine
-it with an imaging library such as Pillow or PIL to render the caption on
-Instagram pictures.  The following shows an approach.
-
-.. literalinclude:: codesnippets/110_pil_captions.py
-
-See also :attr:`Post.caption`, :attr:`Post.url`, :meth:`Post.from_shortcode`,
-:func:`load_structure_from_file`.
-
-Discussed in :issue:`110`.
+Discussed in :issue:`194`.
 
 Metadata JSON Files
 -------------------

docs/codesnippets/110_pil_captions.py

@@ -1,20 +0,0 @@
-from io import BytesIO
-
-from requests import get
-from PIL import Image, ImageDraw
-from instaloader import *
-
-L = Instaloader()
-
-# Load Post instance
-post = load_structure_from_file(L.context, '2017-10-01_18-53-03_UTC.json.xz')
-# or post = Post.from_shortcode(L.context, SHORTCODE)
-
-# Render caption
-image = Image.open(BytesIO(get(post.url).content))
-draw = ImageDraw.Draw(image)
-color = 'rgb(0, 0, 0)'  # black color
-draw.text((300,100), post.caption.encode('latin1', errors='ignore'), fill=color)
-
-# Save image
-image.save('test.jpg')

docs/codesnippets/113_only_one_per_user.py

@@ -2,13 +2,13 @@ import instaloader
 
 L = instaloader.Instaloader()
 
-posts = L.get_hashtag_posts('milfgarden')
+posts = instaloader.Hashtag.from_name(L.context, 'urbanphotography').get_posts()
 
 users = set()
 
 for post in posts:
     if not post.owner_profile in users:
-        L.download_post(post, '#milfgarden')
+        L.download_post(post, '#urbanphotography')
         users.add(post.owner_profile)
     else:
         print("{} from {} skipped.".format(post, post.owner_profile))

docs/codesnippets/120_ghost_followers.py

@@ -2,26 +2,26 @@ import instaloader
 
 L = instaloader.Instaloader()
 
-USER = 'your_account'
+USER = "your_account"
 PROFILE = USER
 
-# Your preferred way of logging in:
+# Load session previously saved with `instaloader -l USERNAME`:
 L.load_session_from_file(USER)
 
 profile = instaloader.Profile.from_username(L.context, PROFILE)
 
 likes = set()
-print('Fetching likes of all posts of profile {}.'.format(profile.username))
+print("Fetching likes of all posts of profile {}.".format(profile.username))
 for post in profile.get_posts():
     print(post)
     likes = likes | set(post.get_likes())
 
-print('Fetching followers of profile {}.'.format(profile.username))
+print("Fetching followers of profile {}.".format(profile.username))
 followers = set(profile.get_followers())
 
 ghosts = followers - likes
 
-print('Storing ghosts into file.')
-with open('/YOUR PATH/inactive-users.txt', 'w') as f:
+print("Storing ghosts into file.")
+with open("inactive-users.txt", 'w') as f:
     for ghost in ghosts:
         print(ghost.username, file=f)

docs/codesnippets/121_since_until.py

@@ -5,13 +5,11 @@ import instaloader
 
 L = instaloader.Instaloader()
 
-posts = L.get_hashtag_posts('milfgarden')
-# or
-# posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts()
+posts = instaloader.Profile.from_username(L.context, "instagram").get_posts()
 
 SINCE = datetime(2015, 5, 1)
 UNTIL = datetime(2015, 3, 1)
 
 for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
     print(post.date)
-    L.download_post(post, '#milfgarden')
+    L.download_post(post, "instagram")

docs/codesnippets/194_top_x_of_user.py

@@ -0,0 +1,17 @@
+from itertools import islice
+from math import ceil
+
+from instaloader import Instaloader, Profile
+
+PROFILE = ...        # profile to download from
+X_percentage = 10    # percentage of posts that should be downloaded
+
+L = Instaloader()
+
+profile = Profile.from_username(L.context, PROFILE)
+posts_sorted_by_likes = sorted(profile.get_posts(),
+                               key=lambda p: p.likes + p.comments,
+                               reverse=True)
+
+for post in islice(posts_sorted_by_likes, ceil(profile.mediacount * X_percentage / 100)):
+    L.download_post(post, PROFILE)

docs/codesnippets/615_import_firefox_session.py

@@ -0,0 +1,53 @@
+from argparse import ArgumentParser
+from glob import glob
+from os.path import expanduser
+from platform import system
+from sqlite3 import OperationalError, connect
+
+try:
+    from instaloader import ConnectionException, Instaloader
+except ModuleNotFoundError:
+    raise SystemExit("Instaloader not found.\n  pip install [--user] instaloader")
+
+
+def get_cookiefile():
+    default_cookiefile = {
+        "Windows": "~/AppData/Roaming/Mozilla/Firefox/Profiles/*/cookies.sqlite",
+        "Darwin": "~/Library/Application Support/Firefox/Profiles/*/cookies.sqlite",
+    }.get(system(), "~/.mozilla/firefox/*/cookies.sqlite")
+    cookiefiles = glob(expanduser(default_cookiefile))
+    if not cookiefiles:
+        raise SystemExit("No Firefox cookies.sqlite file found. Use -c COOKIEFILE.")
+    return cookiefiles[0]
+
+
+def import_session(cookiefile, sessionfile):
+    print("Using cookies from {}.".format(cookiefile))
+    conn = connect(f"file:{cookiefile}?immutable=1", uri=True)
+    try:
+        cookie_data = conn.execute(
+            "SELECT name, value FROM moz_cookies WHERE baseDomain='instagram.com'"
+        )
+    except OperationalError:
+        cookie_data = conn.execute(
+            "SELECT name, value FROM moz_cookies WHERE host LIKE '%instagram.com'"
+        )
+    instaloader = Instaloader(max_connection_attempts=1)
+    instaloader.context._session.cookies.update(cookie_data)
+    username = instaloader.test_login()
+    if not username:
+        raise SystemExit("Not logged in. Are you logged in successfully in Firefox?")
+    print("Imported session cookie for {}.".format(username))
+    instaloader.context.username = username
+    instaloader.save_session_to_file(sessionfile)
+
+
+if __name__ == "__main__":
+    p = ArgumentParser()
+    p.add_argument("-c", "--cookiefile")
+    p.add_argument("-f", "--sessionfile")
+    args = p.parse_args()
+    try:
+        import_session(args.cookiefile or get_cookiefile(), args.sessionfile)
+    except (ConnectionException, OperationalError) as e:
+        raise SystemExit("Cookie import failed: {}".format(e))

docs/codesnippets/666_historical_hashtag_data.py

@@ -0,0 +1,31 @@
+from datetime import datetime
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = instaloader.Hashtag.from_name(L.context, "urbanphotography").get_posts()
+
+SINCE = datetime(2020, 5, 10)  # further from today, inclusive
+UNTIL = datetime(2020, 5, 11)  # closer to today, not inclusive
+
+k = 0  # initiate k
+#k_list = []  # uncomment this to tune k
+
+for post in posts:
+    postdate = post.date
+
+    if postdate > UNTIL:
+        continue
+    elif postdate <= SINCE:
+        k += 1
+        if k == 50:
+            break
+        else:
+            continue
+    else:
+        L.download_post(post, "#urbanphotography")
+        # if you want to tune k, uncomment below to get your k max
+        #k_list.append(k)
+        k = 0  # set k to 0
+
+#max(k_list)

docs/codesnippets/README.md

@@ -0,0 +1,4 @@
+# Advanced Instaloader Examples
+
+This directory contains the scripts that are presented at:
+https://instaloader.github.io/codesnippets.html

docs/conf.py

@@ -1,379 +1,88 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
+# Configuration file for the Sphinx documentation builder.
 #
-# Instaloader documentation build configuration file, created by
-# sphinx-quickstart on Fri Aug 25 15:38:26 2017.
-#
-# This file is execfile()d with the current directory set to its
-# containing dir.
-#
-# Note that not all possible configuration values are present in this
-# autogenerated file.
-#
-# All configuration values have a default; values that are commented out
-# serve to show the default.
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
 
-# If extensions (or modules to document with autodoc) are in another directory,
-# add these directories to sys.path here. If the directory is relative to the
-# documentation root, use os.path.abspath to make it absolute, like shown here.
-#
 import os
+import platform
 import subprocess
 import sys
 
-import sphinx_bootstrap_theme
-
 sys.path.insert(0, os.path.abspath('..'))
 
-import docs.sphinx_autodoc_typehints as sphinx_autodoc_typehints
 
-# -- General configuration ------------------------------------------------
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
-# If your documentation needs a minimal Sphinx version, state it here.
-#
-# needs_sphinx = '1.0'
-
-# Add any Sphinx extension module names here, as strings. They can be
-# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
-# ones.
-extensions = [
-    'sphinx.ext.autodoc',
-    'sphinx.ext.githubpages',
-    'sphinx.ext.intersphinx',
-    'sphinx.ext.extlinks'
-]
-
-autodoc_default_flags = ['show-inheritance', 'members', 'undoc-members']
-
-autodoc_member_order = 'bysource'
-
-intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
-                       'requests': ('http://docs.python-requests.org/en/master', None)}
-
-extlinks = {'issue': ('https://github.com/instaloader/instaloader/issues/%s', 'issue #')}
-
-# Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
-
-# The suffix(es) of source filenames.
-# You can specify multiple suffix as a list of string:
-#
-# source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
-
-# The encoding of source files.
-#
-# source_encoding = 'utf-8-sig'
-
-# The master toctree document.
-master_doc = 'index'
-
-# General information about the project.
 project = 'Instaloader'
-copyright = '2017, Alexander Graf and Andre Koch-Kramer'
+copyright = '2023, Alexander Graf and Andre Koch-Kramer'
 author = 'Alexander Graf and Andre Koch-Kramer'
 
-# The version info for the project you're documenting, acts as replacement for
-# |version| and |release|, also used in various other places throughout the
-# built documents.
-#
-# The short X.Y version.
-#version = '3.0'
-# The full version, including alpha/beta/rc tags.
-#release = '3.0rc0'
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
 
-# The language for content autogenerated by Sphinx. Refer to documentation
-# for a list of supported languages.
-#
-# This is also used if you do content translation via gettext catalogs.
-# Usually you set "language" from the command line for these cases.
-language = None
+extensions = [
+    "sphinx.ext.autodoc",
+    "sphinx.ext.extlinks",
+    "sphinx.ext.githubpages",
+    "sphinx.ext.intersphinx",
+    "sphinxcontrib.jquery",
+]
 
-# There are two options for replacing |today|: either, you set today to some
-# non-false value, then it is used:
-#
-# today = ''
-#
-# Else, today_fmt is used as the format for a strftime call.
-#
-# today_fmt = '%B %d, %Y'
+# autodoc options
 
-# List of patterns, relative to source directory, that match files and
-# directories to ignore when looking for source files.
-# This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
-
-# The reST default role (used for this markup: `text`) to use for all
-# documents.
-#
-# default_role = None
-
-# If true, '()' will be appended to :func: etc. cross-reference text.
-#
-# add_function_parentheses = True
-
-# If true, the current module name will be prepended to all description
-# unit titles (such as .. function::).
-#
-add_module_names = False
-
-# If true, sectionauthor and moduleauthor directives will be shown in the
-# output. They are ignored by default.
-#
-# show_authors = False
-
-# The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
-
-# A list of ignored prefixes for module index sorting.
-# modindex_common_prefix = []
-
-# If true, keep warnings as "system message" paragraphs in the built documents.
-# keep_warnings = False
-
-# If true, `todo` and `todoList` produce output, else they produce nothing.
-todo_include_todos = False
-
-
-# -- Options for HTML output ----------------------------------------------
-
-# The theme to use for HTML and HTML Help pages.  See the documentation for
-# a list of builtin themes.
-#
-html_theme = 'bootstrap'
-html_theme_path = sphinx_bootstrap_theme.get_html_theme_path()
-
-# Theme options are theme-specific and customize the look and feel of a theme
-# further.  For a list of options available for each theme, see the
-# documentation.
-#
-html_theme_options = {
-    'navbar_site_name': 'Site Contents',
-    'navbar_pagenav_name': 'Page Contents',
-    'navbar_pagenav': True,
-    'navbar_sidebarrel': True,
-    'nosidebar': True,
-     }
-
-# Add any paths that contain custom themes here, relative to this directory.
-# html_theme_path = []
-
-# The name for this set of Sphinx documents.
-# "<project> v<release> documentation" by default.
-#
-# html_title = 'Instaloader v3.0rc0'
-
-# A shorter title for the navigation bar.  Default is the same as html_title.
-#
-# html_short_title = None
-
-# The name of an image file (relative to this directory) to place at the top
-# of the sidebar.
-#
-html_logo = "logo.png"
-
-# The name of an image file (relative to this directory) to use as a favicon of
-# the docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
-# pixels large.
-#
-html_favicon = "favicon.ico"
-
-# Add any paths that contain custom static files (such as style sheets) here,
-# relative to this directory. They are copied after the builtin static files,
-# so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
-
-# Add any extra paths that contain custom files (such as robots.txt or
-# .htaccess) here, relative to this directory. These files are copied
-# directly to the root of the documentation.
-#
-# html_extra_path = []
-
-# If not None, a 'Last updated on:' timestamp is inserted at every page
-# bottom, using the given strftime format.
-# The empty string is equivalent to '%b %d, %Y'.
-#
-# html_last_updated_fmt = None
-
-# If true, SmartyPants will be used to convert quotes and dashes to
-# typographically correct entities.
-#
-# html_use_smartypants = True
-
-# Custom sidebar templates, maps document names to template names.
-#
-#html_sidebars = {'**': ["relations.html", "links.html"] }
-html_sidebars = {}
-
-# Additional templates that should be rendered to pages, maps page names to
-# template names.
-#
-# html_additional_pages = {}
-
-# If false, no module index is generated.
-#
-html_domain_indices = False
-
-# If false, no index is generated.
-#
-html_use_index = True
-
-# If true, the index is split into individual pages for each letter.
-#
-# html_split_index = False
-
-# If true, links to the reST sources are added to the pages.
-#
-html_show_sourcelink = False
-
-# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
-#
-html_show_sphinx = False
-
-# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
-#
-html_show_copyright = False
-
-# If true, an OpenSearch description file will be output, and all pages will
-# contain a <link> tag referring to it.  The value of this option must be the
-# base URL from which the finished HTML is served.
-#
-# html_use_opensearch = ''
-
-# This is the file name suffix for HTML files (e.g. ".xhtml").
-# html_file_suffix = None
-
-# Language to be used for generating the HTML full-text search index.
-# Sphinx supports the following languages:
-#   'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
-#   'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr', 'zh'
-#
-# html_search_language = 'en'
-
-# A dictionary with options for the search language support, empty by default.
-# 'ja' uses this config value.
-# 'zh' user can custom change `jieba` dictionary path.
-#
-# html_search_options = {'type': 'default'}
-
-# The name of a javascript file (relative to the configuration directory) that
-# implements a search results scorer. If empty, the default will be used.
-#
-# html_search_scorer = 'scorer.js'
-
-# Output file base name for HTML help builder.
-htmlhelp_basename = 'Instaloaderdoc'
-
-# -- Options for LaTeX output ---------------------------------------------
-
-latex_elements = {
-     # The paper size ('letterpaper' or 'a4paper').
-     #
-     # 'papersize': 'letterpaper',
-
-     # The font size ('10pt', '11pt' or '12pt').
-     #
-     # 'pointsize': '10pt',
-
-     # Additional stuff for the LaTeX preamble.
-     #
-     # 'preamble': '',
-
-     # Latex figure (float) alignment
-     #
-     # 'figure_align': 'htbp',
+autodoc_default_options = {
+    'show-inheritance': True,
+    'members': True,
+    'undoc-members': True,
 }
 
-# Grouping the document tree into LaTeX files. List of tuples
-# (source start file, target name, title,
-#  author, documentclass [howto, manual, or own class]).
-latex_documents = [
-    (master_doc, 'Instaloader.tex', 'Instaloader Documentation',
-     'Alexander Graf, André Koch-Kramer', 'manual'),
-]
+# extlinks options
 
-# The name of an image file (relative to this directory) to place at the top of
-# the title page.
-#
-# latex_logo = None
+extlinks = {
+    'issue': (
+        'https://github.com/instaloader/instaloader/issues/%s',
+        'Issue #%s'
+    ),
+    'example': (
+        'https://raw.githubusercontent.com/instaloader/instaloader/master/docs/codesnippets/%s',
+        'Example %s'
+    ),
+}
 
-# For "manual" documents, if this is true, then toplevel headings are parts,
-# not chapters.
-#
-# latex_use_parts = False
+# intersphinx options
 
-# If true, show page references after internal links.
-#
-# latex_show_pagerefs = False
+intersphinx_mapping = {
+    'python': ('https://docs.python.org/3', None),
+    'requests': ('https://requests.readthedocs.io/en/latest/', None),
+}
 
-# If true, show URL addresses after external links.
-#
-# latex_show_urls = False
+# other options
 
-# Documents to append as an appendix to all manuals.
-#
-# latex_appendices = []
+templates_path = ['_templates']
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
 
-# It false, will not define \strong, \code, 	itleref, \crossref ... but only
-# \sphinxstrong, ..., \sphinxtitleref, ... To help avoid clash with user added
-# packages.
-#
-# latex_keep_old_macro_names = True
+# ignore TypeVar as it is currently unsupported by Sphinx
+nitpick_ignore = [('py:class', 'instaloader.nodeiterator.T')]
 
-# If false, no module index is generated.
-#
-# latex_domain_indices = True
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
 
+pygments_style = 'fruity'
 
-# -- Options for manual page output ---------------------------------------
+html_theme = 'basic'
+html_static_path = ['_static']
 
-# One entry per manual page. List of tuples
-# (source start file, name, description, authors, manual section).
-man_pages = [
-    (master_doc, 'instaloader', 'Instaloader Documentation',
-     [author], 1)
-]
+html_logo = "logo.png"
 
-# If true, show URL addresses after external links.
-#
-# man_show_urls = False
-
-
-# -- Options for Texinfo output -------------------------------------------
-
-# Grouping the document tree into Texinfo files. List of tuples
-# (source start file, target name, title, author,
-#  dir menu entry, description, category)
-texinfo_documents = [
-    (master_doc, 'Instaloader', 'Instaloader Documentation',
-     author, 'Instaloader', 'One line description of project.',
-     'Miscellaneous'),
-]
-
-# Documents to append as an appendix to all manuals.
-#
-# texinfo_appendices = []
-
-# If false, no module index is generated.
-#
-# texinfo_domain_indices = True
-
-# How to display URL addresses: 'footnote', 'no', or 'inline'.
-#
-# texinfo_show_urls = 'footnote'
-
-# If true, do not generate a @detailmenu in the "Top" node's menu.
-#
-# texinfo_no_detailmenu = False
+html_css_files = ['instaloaderdoc.css']
+html_js_files = ['instaloaderdoc.js']
 
+# get release number
 
 current_release = subprocess.check_output(["git", "describe", "--abbrev=0"]).decode("ascii")[1:-1]
-current_release_date = subprocess.check_output(["git", "log", "-1", "--tags", "--format=%ad", "--date=format:%e %b %Y"]).decode("ascii")[:-1]
+date_format = "%e %b %Y" if platform.system() != "Windows" else "%d %b %Y"
+current_release_date = subprocess.check_output(
+    ["git", "log", "-1", "--tags", "--format=%ad", "--date=format:"+date_format]).decode("ascii")[:-1]
 
 html_context = {'current_release': current_release, 'current_release_date': current_release_date}
-
-
-def setup(app):
-    app.add_stylesheet("style.css")
-    app.connect('autodoc-process-signature', sphinx_autodoc_typehints.process_signature)
-    app.connect('autodoc-process-docstring', sphinx_autodoc_typehints.process_docstring)

docs/contributing.rst

@@ -11,18 +11,35 @@ Instaloader's development is organized on
 `GitHub <https://github.com/instaloader/instaloader>`__, where Issues and Pull
 Requests are discussed.
 
+Contributing to an open source project is an unmatchable opportunity to improve
+and manifest programming skills. For issues that are good starting points to get
+involved in Instaloader development, as they do not require deep insight into
+Instaloader's code base, have a look at our
+`Issues labelled 'good first issue' <https://github.com/instaloader/instaloader/contribute>`__.
+
+This document covers several ways of how you can contribute back to Instaloader.
+
+Answering Questions
+-------------------
+
+The easiest way to help out is to answer questions. If you are interested in
+answering questions regarding Instaloader, good places to begin are
+
+- `Questions tagged 'instaloader' on Stack Overflow <https://stackoverflow.com/questions/tagged/instaloader>`__,
+- `Instaloader Issues labeled 'question' <https://github.com/instaloader/instaloader/issues?q=is%3Aissue+is%3Aopen+label%3Aquestion>`__.
+
 Reporting Bugs
 --------------
 
 If you encounter a bug, do not hesitate to report it in our
-`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. Since
-Instaloader is actively developed, the majority of bugs is fixed within only
-**3 days** after being reported. When reporting a problem, please keep the
-following in mind:
+`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. When
+reporting a problem, please keep the following in mind:
 
 - Ensure you **use the latest version** of Instaloader. The currently-installed
   version can be found out with ``instaloader --version``.
 
+- Check whether there is a valid solution in our :ref:`troubleshooting` section.
+
 - Briefly **check whether the bug has already been reported**. If you find an
   issue reporting the same bug you encountered, comment there rather than
   opening a new issue. However, if unsure, please create a new issue.
@@ -32,42 +49,65 @@ following in mind:
 
 - Include all **error messages and tracebacks** in the report.
 
-- If not obvious, describe **which behavior you expected**
+- Even if it seems obvious, describe **which behavior you expected**
   instead of what actually happened.
 
-- If you think an issue has been closed accidentally or inappropriately, feel
-  free to reopen it.
+- If we have closed an issue apparently inadvertently or inappropriately, please
+  let us know.
 
 Writing Code or Improving the Documentation
 -------------------------------------------
 
-Changes of the Instaloader source can be proposed as a
-`Pull Request <https://github.com/instaloader/instaloader/pulls>`__. There are only
-few things to consider:
+Improvements of the Instaloader source or its documentation can be proposed as a
+`Pull Request <https://github.com/instaloader/instaloader/pulls>`__.
 
-- Sometimes, the most current code is not in the ``master`` branch. Check that
-  you forked from the most recent branch.
+- Please base your Pull Request on
 
-- We use `Pylint <https://www.pylint.org/>`__ for error and syntax checking of
-  the source. The file ``.travis.yml`` in the project's root directory
-  shows how it is invoked. Note that sometimes it might be better to disable a
-  warning rather than adapting the code to Pylint's desires.
+  - ``master``, which will be released with the next minor release, if it is
 
-- The documentation source is located in the ``docs`` folder. The file
-  ``cli-options.rst`` is merely an RST-formatted copy of ``instaloader --help``
-  output, of which the source is in ``instaloader/__main__.py``.
+    - a bug fix that does not require extensive testing,
+    - an improvement to the documentation,
 
-- Feel free to contact us, even if you "only" have Proof-of-Concepts or
-  not-fully integrated changes. They already might be an advance for the
-  project.
+  - ``upcoming/v4.X``, if it is
 
-Suggesting Features
--------------------
+    - a new feature,
+    - a bug fix that does require thorough testing before being released to a
+      final version.
 
-If you have an idea of how Instaloader should be enhanced, but do not want to
-implement the feature yourself, feel free to open a ticket in our 
-`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__.
-Please consider the following:
+- All Pull Requests are analyzed by `Pylint <https://www.pylint.org/>`__ for
+  error and syntax checking of the source and
+  `Mypy <https://github.com/python/mypy>`__ for type checking. You can run them
+  locally with::
+
+     pylint instaloader
+     mypy -m instaloader
+
+- Improvements to the documentation are very welcome. The documentation is
+  created with `Sphinx <https://www.sphinx-doc.org/en/2.0/>`__, version 2,
+  and can be build locally using::
+
+     make -C docs html
+
+- Feel free to create an issue to make sure someone from the Instaloader team
+  agrees that the change might be an improvement, or if you want to discuss
+  your basic proposal, before working on a pull request.
+
+Proposing Features
+------------------
+
+.. goal-start
+
+Instaloader's goal is to mimic the browser's behavior to access the data that
+is available through the Instagram web interface, pack this data into complete
+and easily-(re)usable python objects, and provide a user interface for the most
+common downloading and metadata collection tasks, without changing any of the
+online data.
+
+.. goal-end
+
+Prior spending effort on implementing a new feature, it might be appropriate to
+clarify how it could fit into the project's scope or discuss implementation
+details. If you feel the need to do so, please create a "feature suggestion".
 
 - Instaloader already has plenty of features. **Check the documentation**
   beforehand to ensure your desired suggestion is not already implemented.
@@ -76,29 +116,25 @@ Please consider the following:
   an issue suggesting the same or a similar feature, share your thoughts in a
   comment there, instead of opening a new issue.
 
-- If possible, provide us a **use case of the feature**: How could the user
-  invoke the new function? Which problem would it solve? If new information is
-  obtained, how would it be further processed?
+- **Motivate the feature**, i.e.
 
-- **Be patient**. Naturally, bugs and pull requests have a higher priority than
-  feature suggestions. Keep in mind that this is a free software project, and
-  unfortunately we only have limited time to work on it.
+  - Provide us a **use case of the feature**: How could the user
+    invoke the new function? Which problem would it solve? If new information is
+    obtained, how would it be further processed?
 
-Donations
----------
+  - Describe already-working **alternatives of the feature** and how they
+    compare to your proposed feature.
 
-.. donations-start
+  - Briefly describe how your suggested feature **conforms with Instaloader's
+    project goal**.
 
-It is a pleasure for us to share our Instaloader to the world, and we are proud
-to have attracted such an active and motivating community, with so many users
-who share their suggestions and ideas with us. Buying a community-sponsored beer
-or coffee from time to time is very likely to further raise our passion for the
-development of Instaloader.
+- Explain your **solution ideas**. Describe your ideas on how the feature could
+  be implemented and the underlying problem could be solved. Also **describe
+  alternatives** that you have considered.
 
-| For donations, we provide a PayPal.Me link and a Bitcoin address.
-|  `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
-|  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
+Sponsoring
+----------
 
-.. donations-end
-
-.. (Discussion in :issue:`130`)
+.. include:: sponsors.rst
+   :start-after: donations-start
+   :end-before: donations-end

docs/index.rst

@@ -15,6 +15,10 @@ Instaloader
 **Instaloader** is a tool to download pictures (or videos) along with
 their captions and other metadata from Instagram.
 
+.. include:: ../README.rst
+   :start-after: badges-start
+   :end-before: badges-end
+
 With `Python <https://www.python.org/>`__ installed, do::
 
     $ pip3 install instaloader
@@ -36,14 +40,17 @@ See :ref:`install` for more options on how to install Instaloader.
 - allows **fine-grained customization** of filters and where to store
   downloaded media,
 
+- automatically **resumes previously-interrupted** download iterations,
+
 - is free `open source <https://github.com/instaloader/instaloader>`__
   software written in Python.
 
 ::
 
-    instaloader [--comments] [--geotags] [--stories] [--highlights] [--tagged]
+    instaloader [--comments] [--geotags]
+                [--stories] [--highlights] [--tagged] [--igtv]
                 [--login YOUR-USERNAME] [--fast-update]
-                profile | "#hashtag" |
+                profile | "#hashtag" | %location_id |
                 :stories | :feed | :saved
 
 See :ref:`download-pictures-from-instagram` for a detailed introduction on how
@@ -60,6 +67,7 @@ Instaloader Documentation
    cli-options
    as-module
    codesnippets
+   troubleshooting
    contributing
 
 Useful Links
@@ -70,13 +78,6 @@ Useful Links
 - `Issue Tracker / Bug Tracker <https://github.com/instaloader/instaloader/issues>`__
 - `Version History <https://github.com/instaloader/instaloader/releases>`__
 
-Disclaimer
-----------
-
-.. include:: ../README.rst
-   :start-after: disclaimer-start
-   :end-before: disclaimer-end
-
 Contributing
 ------------
 
@@ -84,10 +85,20 @@ As an open source project, Instaloader heavily depends on the contributions from
 its community. See :ref:`contributing` for how you may help Instaloader to
 become an even greater tool.
 
-.. include:: contributing.rst
+Supporters
+----------
+
+.. include:: sponsors.rst
    :start-after: donations-start
    :end-before: donations-end
 
+Disclaimer
+----------
+
+.. include:: ../README.rst
+   :start-after: disclaimer-start
+   :end-before: disclaimer-end
+
 ..
    * :ref:`genindex`
    * :ref:`modindex`

docs/installation.rst

@@ -5,27 +5,27 @@ Install Instaloader
 
 .. highlight:: none
 
-**To install Instaloader**,
+To **install Instaloader**,
 
 #. Ensure that you have `Python <https://www.python.org/>`__, at least
-   version 3.5 and `pip <https://pypi.python.org/pypi/pip>`__
-   installed
+   version 3.8, and `pip <https://pypi.python.org/pypi/pip>`__
+   installed.
 
 #. Then, install Instaloader using::
 
       pip3 install instaloader
 
-**To upgrade Instaloader**, do::
+To **upgrade Instaloader** to its current version, do::
 
    pip3 install --upgrade instaloader
 
 
-**Alternative methods for installing Instaloader:**
+**Alternative methods** for installing Instaloader:
 
 - If you do not want to use pip, even though it is highly recommended,
   and prefer to **install Instaloader manually**,
   `Download the Source <https://github.com/instaloader/instaloader/releases/latest>`__,
-  extract the Zip or Tarball and execute ``instaloader.py`` from there.
+  extract the Zip or Tarball and run ``setup.py`` from there.
 
 - On **Arch Linux**, you may install Instaloader using the
   `Instaloader AUR package <https://aur.archlinux.org/packages/instaloader/>`__.
@@ -33,6 +33,9 @@ Install Instaloader
 - On **Windows 10**, you may download the standalone executable from the
   `current release page <https://github.com/instaloader/instaloader/releases/latest>`__.
 
-- To test the most current pre-release or development version of Instaloader::
+- On **Android**, you can use Instaloader with `Termux <https://play.google.com/store/apps/details?id=com.termux>`__
+  after typing ``pkg install python`` and ``pip3 install instaloader``.
+
+- To test the most current **pre-release** version of Instaloader::
 
      pip3 install --pre instaloader

docs/module/exceptions.rst

@@ -0,0 +1,61 @@
+Exceptions
+^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. currentmodule:: instaloader.exceptions
+
+.. autoexception:: InstaloaderException
+   :no-show-inheritance:
+
+.. autoexception:: ConnectionException
+
+.. currentmodule:: instaloader
+
+.. autoexception:: QueryReturnedBadRequestException
+
+.. autoexception:: QueryReturnedForbiddenException
+
+.. autoexception:: ProfileNotExistsException
+
+.. autoexception:: ProfileHasNoPicsException
+
+.. autoexception:: PrivateProfileNotFollowedException
+
+.. autoexception:: LoginRequiredException
+
+.. autoexception:: LoginException
+
+   .. versionadded:: 4.12
+
+.. autoexception:: TwoFactorAuthRequiredException
+
+   .. versionadded:: 4.2
+
+   .. versionchanged:: 4.12
+      Inherits LoginException
+
+.. autoexception:: InvalidArgumentException
+
+.. autoexception:: BadResponseException
+
+.. autoexception:: BadCredentialsException
+
+   .. versionchanged:: 4.12
+      Inherits LoginException
+
+.. autoexception:: PostChangedException
+
+.. autoexception:: QueryReturnedNotFoundException
+
+   .. versionchanged:: 4.3
+      QueryReturnedNotFoundException now inherits ConnectionException
+      to retry on 404 errors.
+
+
+.. autoexception:: TooManyRequestsException
+
+.. autoexception:: AbortDownloadException

docs/module/instaloader.rst

@@ -0,0 +1,10 @@
+``Instaloader`` (Main Class)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. autoclass:: Instaloader
+   :no-show-inheritance:

docs/module/instaloadercontext.rst

@@ -0,0 +1,24 @@
+``InstaloaderContext`` (Low-level functions)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+``InstaloaderContext``
+""""""""""""""""""""""
+
+.. autoclass:: InstaloaderContext
+   :no-show-inheritance:
+
+``RateController``
+""""""""""""""""""
+
+.. autoclass:: RateController
+   :no-show-inheritance:
+
+   .. versionadded:: 4.5

docs/module/nodeiterator.rst

@@ -0,0 +1,38 @@
+Resumable Iterations
+^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+For many download targets, Instaloader is able to resume a
+previously-interrupted iteration. It provides an interruptible
+Iterator :class:`NodeIterator` and a context manager
+:func:`resumable_iteration`, which we both present here.
+
+.. versionadded:: 4.5
+
+``NodeIterator``
+""""""""""""""""
+
+.. autoclass:: NodeIterator
+   :no-show-inheritance:
+
+.. autoclass:: FrozenNodeIterator
+   :no-show-inheritance:
+
+   A serializable representation of a :class:`NodeIterator` instance, saving
+   its iteration state.
+
+   It can be serialized and deserialized with :func:`save_structure_to_file`
+   and :func:`load_structure_from_file`, as well as with :mod:`json` and
+   :mod:`pickle` thanks to being a :class:`~typing.NamedTuple`.
+
+``resumable_iteration``
+"""""""""""""""""""""""
+
+.. autofunction:: resumable_iteration

docs/module/structures.rst

@@ -0,0 +1,100 @@
+Instagram Structures
+^^^^^^^^^^^^^^^^^^^^
+
+.. module:: instaloader
+   :noindex:
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+Posts
+"""""
+
+.. autoclass:: Post
+   :no-show-inheritance:
+
+Additionally, the following trivial structures are defined:
+
+.. autoclass:: PostSidecarNode
+   :no-show-inheritance:
+
+.. autoclass:: PostComment
+   :no-show-inheritance:
+
+.. autoclass:: PostCommentAnswer
+   :no-show-inheritance:
+
+.. autoclass:: PostLocation
+   :no-show-inheritance:
+
+User Stories
+""""""""""""
+
+.. autoclass:: Story
+   :no-show-inheritance:
+
+.. autoclass:: StoryItem
+   :no-show-inheritance:
+
+Highlights
+""""""""""
+
+.. autoclass:: Highlight
+   :no-show-inheritance:
+   :inherited-members:
+
+   Bases: :class:`Story`
+
+   .. versionadded:: 4.1
+
+Profiles
+""""""""
+
+.. autoclass:: Profile
+   :no-show-inheritance:
+
+Hashtags
+""""""""
+
+.. autoclass:: Hashtag
+   :no-show-inheritance:
+
+   .. versionadded:: 4.4
+
+TopSearchResults
+""""""""""""""""
+
+.. autoclass:: TopSearchResults
+   :no-show-inheritance:
+
+   .. versionadded:: 4.3
+
+TitlePic
+""""""""
+
+.. autoclass:: TitlePic
+   :no-show-inheritance:
+
+   .. versionadded:: 4.8
+
+Loading and Saving
+""""""""""""""""""
+
+:class:`Post`, :class:`StoryItem`, :class:`Profile`, :class:`Hashtag` and
+:class:`FrozenNodeIterator` can be saved and loaded to/from JSON files.
+
+.. autofunction:: load_structure
+
+.. autofunction:: load_structure_from_file
+
+.. autofunction:: get_json_structure
+
+.. autofunction:: save_structure_to_file
+
+LatestStamps
+""""""""""""
+
+.. autoclass:: LatestStamps
+   :no-show-inheritance:

docs/requirements.txt

@@ -1,3 +1,2 @@
 requests
-sphinx!=1.7.7
-sphinx-bootstrap-theme
+sphinx~=2.3

docs/sphinx_autodoc_typehints.py

@@ -1,228 +0,0 @@
-# sphinx_autodoc_typehints from Mar 18 2018.
-#  https://github.com/agronholm/sphinx-autodoc-typehints
-# (slightly modified to fix class links, maybe related to agronholm/sphinx-autodoc-typehints#38)
-#
-# This is the MIT license: http://www.opensource.org/licenses/mit-license.php
-#
-# Copyright (c) Alex Grönholm
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy of this
-# software and associated documentation files (the "Software"), to deal in the Software
-# without restriction, including without limitation the rights to use, copy, modify, merge,
-# publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
-# to whom the Software is furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in all copies or
-# substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
-# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
-# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
-# FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
-# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-# DEALINGS IN THE SOFTWARE.
-#
-
-
-import inspect
-from typing import get_type_hints, TypeVar, Any, AnyStr, Generic, Union
-
-from sphinx.util.inspect import getargspec
-from sphinx.ext.autodoc import formatargspec
-
-try:
-    from inspect import unwrap
-except ImportError:
-    def unwrap(func, *, stop=None):
-        """This is the inspect.unwrap() method copied from Python 3.5's standard library."""
-        if stop is None:
-            def _is_wrapper(f):
-                return hasattr(f, '__wrapped__')
-        else:
-            def _is_wrapper(f):
-                return hasattr(f, '__wrapped__') and not stop(f)
-        f = func  # remember the original func for error reporting
-        memo = {id(f)}  # Memoise by id to tolerate non-hashable objects
-        while _is_wrapper(func):
-            func = func.__wrapped__
-            id_func = id(func)
-            if id_func in memo:
-                raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
-            memo.add(id_func)
-        return func
-
-
-def format_annotation(annotation):
-    if inspect.isclass(annotation) and annotation.__module__ == 'builtins':
-        if annotation.__qualname__ == 'NoneType':
-            return '``None``'
-        else:
-            return ':py:class:`{}`'.format(annotation.__qualname__)
-
-    annotation_cls = annotation if inspect.isclass(annotation) else type(annotation)
-    class_name = None
-    if annotation_cls.__module__ == 'typing':
-        params = None
-        prefix = ':py:class:'
-        module = 'typing'
-        extra = ''
-
-        if inspect.isclass(getattr(annotation, '__origin__', None)):
-            annotation_cls = annotation.__origin__
-            try:
-                if Generic in annotation_cls.mro():
-                    module = annotation_cls.__module__
-            except TypeError:
-                pass  # annotation_cls was either the "type" object or typing.Type
-
-        if annotation is Any:
-            return ':py:data:`~typing.Any`'
-        elif annotation is AnyStr:
-            return ':py:data:`~typing.AnyStr`'
-        elif isinstance(annotation, TypeVar):
-            return '\\%r' % annotation
-        elif (annotation is Union or getattr(annotation, '__origin__', None) is Union or
-              hasattr(annotation, '__union_params__')):
-            prefix = ':py:data:'
-            class_name = 'Union'
-            if hasattr(annotation, '__union_params__'):
-                params = annotation.__union_params__
-            elif hasattr(annotation, '__args__'):
-                params = annotation.__args__
-
-            if params and len(params) == 2 and (hasattr(params[1], '__qualname__') and
-                                                params[1].__qualname__ == 'NoneType'):
-                class_name = 'Optional'
-                params = (params[0],)
-        elif annotation_cls.__qualname__ == 'Tuple' and hasattr(annotation, '__tuple_params__'):
-            params = annotation.__tuple_params__
-            if annotation.__tuple_use_ellipsis__:
-                params += (Ellipsis,)
-        elif annotation_cls.__qualname__ == 'Callable':
-            prefix = ':py:data:'
-            arg_annotations = result_annotation = None
-            if hasattr(annotation, '__result__'):
-                arg_annotations = annotation.__args__
-                result_annotation = annotation.__result__
-            elif getattr(annotation, '__args__', None):
-                arg_annotations = annotation.__args__[:-1]
-                result_annotation = annotation.__args__[-1]
-
-            if arg_annotations in (Ellipsis, (Ellipsis,)):
-                params = [Ellipsis, result_annotation]
-            elif arg_annotations is not None:
-                params = [
-                    '\\[{}]'.format(
-                        ', '.join(format_annotation(param) for param in arg_annotations)),
-                    result_annotation
-                ]
-        elif hasattr(annotation, 'type_var'):
-            # Type alias
-            class_name = annotation.name
-            params = (annotation.type_var,)
-        elif getattr(annotation, '__args__', None) is not None:
-            params = annotation.__args__
-        elif hasattr(annotation, '__parameters__'):
-            params = annotation.__parameters__
-
-        if params:
-            extra = '\\[{}]'.format(', '.join(format_annotation(param) for param in params))
-
-        if not class_name:
-            class_name = annotation_cls.__qualname__.title()
-
-        return '{}`~{}.{}`{}'.format(prefix, module, class_name, extra)
-    elif annotation is Ellipsis:
-        return '...'
-    elif inspect.isclass(annotation) or inspect.isclass(getattr(annotation, '__origin__', None)):
-        if not inspect.isclass(annotation):
-            annotation_cls = annotation.__origin__
-
-        extra = ''
-        if Generic in annotation_cls.mro():
-            params = (getattr(annotation, '__parameters__', None) or
-                      getattr(annotation, '__args__', None))
-            extra = '\\[{}]'.format(', '.join(format_annotation(param) for param in params))
-
-        module = annotation.__module__.split('.')[0]    # hack to 'fix' class linking for Instaloader project
-        return ':py:class:`~{}.{}`{}'.format(module, annotation_cls.__qualname__,
-                                             extra)
-
-    return str(annotation)
-
-
-def process_signature(app, what: str, name: str, obj, options, signature, return_annotation):
-    if not callable(obj):
-        return
-
-    if what in ('class', 'exception'):
-        obj = getattr(obj, '__init__', getattr(obj, '__new__', None))
-
-    if not getattr(obj, '__annotations__', None):
-        return
-
-    obj = unwrap(obj)
-    try:
-        argspec = getargspec(obj)
-    except (TypeError, ValueError):
-        return
-
-    if argspec.args:
-        if what in ('class', 'exception'):
-            del argspec.args[0]
-        elif what == 'method':
-            outer = inspect.getmodule(obj)
-            for clsname in obj.__qualname__.split('.')[:-1]:
-                outer = getattr(outer, clsname)
-            method_object = outer.__dict__[obj.__name__]
-            if not isinstance(method_object, (classmethod, staticmethod)):
-                del argspec.args[0]
-
-    return formatargspec(obj, *argspec[:-1]), None
-
-
-def process_docstring(app, what, name, obj, options, lines):
-    if isinstance(obj, property):
-        obj = obj.fget
-
-    if callable(obj):
-        if what in ('class', 'exception'):
-            obj = getattr(obj, '__init__')
-
-        obj = unwrap(obj)
-        try:
-            type_hints = get_type_hints(obj)
-        except (AttributeError, TypeError):
-            # Introspecting a slot wrapper will raise TypeError
-            return
-
-        for argname, annotation in type_hints.items():
-            formatted_annotation = format_annotation(annotation)
-
-            if argname == 'return':
-                if what in ('class', 'exception'):
-                    # Don't add return type None from __init__()
-                    continue
-
-                insert_index = len(lines)
-                for i, line in enumerate(lines):
-                    if line.startswith(':rtype:'):
-                        insert_index = None
-                        break
-                    elif line.startswith(':return:') or line.startswith(':returns:'):
-                        insert_index = i
-
-                if insert_index is not None:
-                    if insert_index == len(lines):
-                        # Ensure that :rtype: doesn't get joined with a paragraph of text, which
-                        # prevents it being interpreted.
-                        lines.append('')
-                        insert_index += 1
-
-                    lines.insert(insert_index, ':rtype: {}'.format(formatted_annotation))
-            else:
-                searchfor = ':param {}:'.format(argname)
-                for i, line in enumerate(lines):
-                    if line.startswith(searchfor):
-                        lines.insert(i, ':type {}: {}'.format(argname, formatted_annotation))
-                        break

docs/sponsors.rst

@@ -0,0 +1,23 @@
+.. _sponsors:
+
+Sponsoring the development of Instaloader
+=========================================
+
+.. donations-start
+
+.. include:: ../README.rst
+   :start-after: current-sponsors-start
+   :end-before: current-sponsors-end
+
+It is a pleasure for us to share our Instaloader to the world, and we are proud
+to have attracted such an active and motivating community, with so many users
+who share their suggestions and ideas with us. Buying a community-sponsored beer
+or coffee from time to time is very likely to further raise our passion for the
+development of Instaloader.
+
+| For Donations, we provide GitHub Sponsors page, a PayPal.Me link and a Bitcoin address.
+|  GitHub Sponsors: `Sponsor @aandergr on GitHub Sponsors <https://github.com/sponsors/aandergr>`__
+|  PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
+|  BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
+
+.. donations-end

docs/troubleshooting.rst

@@ -0,0 +1,92 @@
+.. _troubleshooting:
+
+Troubleshooting
+===============
+
+.. highlight:: python
+
+429 Too Many Requests
+---------------------
+
+Instaloader has a logic to keep track of its requests to Instagram and to obey
+their rate limits. The rate controller assumes that
+
+- at one time, Instaloader is the only application that consumes requests, i.e.
+  neither the Instagram browser interface, nor a mobile app, nor another
+  Instaloader instance is running in parallel, and
+
+- no requests had been consumed when Instaloader starts.
+
+The latter one implies that restarting or reinstantiating Instaloader often
+within short time is prone to cause a 429.
+
+Since the behavior of the rate controller might change between different
+versions of Instaloader, make sure to use the current version of Instaloader,
+especially when encountering many 429 errors.
+
+If a request is denied with a 429,
+Instaloader retries the request as soon as the temporary ban is assumed to be
+expired. In case the retry continuously fails for some reason, which should not
+happen under normal conditions, consider adjusting the
+:option:`--max-connection-attempts` option.
+
+There have been observations that services, that in their nature offer
+promiscuous IP addresses, such as cloud, VPN and public proxy services, might be
+subject to significantly stricter limits for anonymous access. However,
+:ref:`logged-in accesses<login>` do not seem to be affected.
+
+Instaloader allows to adjust the rate controlling behavior by overriding
+:class:`instaloader.RateController`.
+
+Too many queries in the last time
+---------------------------------
+
+**"Too many queries in the last time"** is not an error. It is a notice that the
+rate limit has almost been reached, according to Instaloader's own rate
+accounting mechanism.
+
+Instaloader allows to adjust the rate controlling behavior by overriding
+:class:`instaloader.RateController`.
+
+Private but not followed
+------------------------
+
+You have to follow a private account to access most of its associated
+information.
+
+Login error
+-----------
+
+Instaloader's login *should* work fine, both with and without
+Two-Factor-Authentication. It also supports handling the *checkpoint challenge*,
+issued when Instagram suspects authentication activity on your account, by
+pointing the user to an URL to be opened in a browser.
+
+Nevertheless, in :issue:`92`, :issue:`615`, :issue:`1150` and :issue:`1217`,
+users reported problems with
+logging in. We recommend to always keep the session file which Instaloader
+creates when using :option:`--login`. If a session file is present,
+:option:`--login` does not make use of the failure-prone login procedure.
+Also, session files usually do not expire.
+
+If you do not have a session file present, you may use the following script
+(:example:`615_import_firefox_session.py`) to workaround login problems by
+importing the session cookies from Firefox and bypassing Instaloader's login and
+so still use Instaloader's logged-in functionality.
+
+.. literalinclude:: codesnippets/615_import_firefox_session.py
+
+To use this script,
+
+#. Download the script: :example:`615_import_firefox_session.py`,
+
+#. Login to Instagram in Firefox,
+
+#. Execute the snippet, e.g. with ``python 615_import_firefox_session.py``,
+
+#. Then, ``instaloader -l USERNAME`` should work fine.
+
+This script also supports specifying a cookie file path, which may be useful if
+you use multiple Firefox profiles or if your operating system has the directory
+structure differently set up. Also, you can specify an alternative session file
+path.

instaloader/__init__.py

@@ -1,12 +1,12 @@
 """Download pictures (or videos) along with their captions and other metadata from Instagram."""
 
 
-__version__ = '4.1b1'
+__version__ = '4.12'
 
 
 try:
     # pylint:disable=wrong-import-position
-    import win_unicode_console
+    import win_unicode_console  # type: ignore
 except ImportError:
     pass
 else:
@@ -14,6 +14,10 @@ else:
 
 from .exceptions import *
 from .instaloader import Instaloader
-from .instaloadercontext import InstaloaderContext
-from .structures import (Highlight, Post, PostSidecarNode, PostComment, PostLocation, Profile, Story, StoryItem,
-                         load_structure_from_file, save_structure_to_file)
+from .instaloadercontext import InstaloaderContext, RateController
+from .lateststamps import LatestStamps
+from .nodeiterator import NodeIterator, FrozenNodeIterator, resumable_iteration
+from .structures import (Hashtag, Highlight, Post, PostSidecarNode, PostComment, PostCommentAnswer, PostLocation,
+                         Profile, Story, StoryItem, TopSearchResults, TitlePic,
+                         load_structure_from_file, save_structure_to_file,
+                         load_structure, get_json_structure)

instaloader/__main__.py

@@ -3,27 +3,54 @@
 import ast
 import datetime
 import os
+import re
 import sys
-from argparse import ArgumentParser, SUPPRESS
+from argparse import ArgumentParser, ArgumentTypeError, SUPPRESS
+from enum import IntEnum
 from typing import List, Optional
 
-from . import (Instaloader, InstaloaderException, InvalidArgumentException, Post, Profile, ProfileNotExistsException,
-               StoryItem, __version__, load_structure_from_file)
-from .instaloader import get_default_session_filename
+from . import (AbortDownloadException, BadCredentialsException, Instaloader, InstaloaderException,
+               InvalidArgumentException, LoginException, Post, Profile, ProfileNotExistsException, StoryItem,
+               TwoFactorAuthRequiredException, __version__, load_structure_from_file)
+from .instaloader import (get_default_session_filename, get_default_stamps_filename)
 from .instaloadercontext import default_user_agent
+from .lateststamps import LatestStamps
+try:
+    import browser_cookie3
+    bc3_library = True
+except ImportError:
+    bc3_library = False
 
 
+class ExitCode(IntEnum):
+    SUCCESS = 0
+    NON_FATAL_ERROR = 1
+    INIT_FAILURE = 2
+    LOGIN_FAILURE = 3
+    DOWNLOAD_ABORTED = 4
+    USER_ABORTED = 5
+    UNEXPECTED_ERROR = 99
+
 def usage_string():
     # NOTE: duplicated in README.rst and docs/index.rst
     argv0 = os.path.basename(sys.argv[0])
     argv0 = "instaloader" if argv0 == "__main__.py" else argv0
     return """
-{0} [--comments] [--geotags] [--stories] [--highlights] [--tagged]
+{0} [--comments] [--geotags]
+{2:{1}} [--stories] [--highlights] [--tagged] [--igtv]
 {2:{1}} [--login YOUR-USERNAME] [--fast-update]
-{2:{1}} profile | "#hashtag" | :stories | :feed | :saved
+{2:{1}} profile | "#hashtag" | %%location_id | :stories | :feed | :saved
 {0} --help""".format(argv0, len(argv0), '')
 
 
+def http_status_code_list(code_list_str: str) -> List[int]:
+    codes = [int(s) for s in code_list_str.split(',')]
+    for code in codes:
+        if not 100 <= code <= 599:
+            raise ArgumentTypeError("Invalid HTTP status code: {}".format(code))
+    return codes
+
+
 def filterstr_to_filterfunc(filter_str: str, item_type: type):
     """Takes an --post-filter=... or --storyitem-filter=... filter
      specification and makes a filter_func Callable out of it."""
@@ -33,7 +60,6 @@ def filterstr_to_filterfunc(filter_str: str, item_type: type):
 
     class TransformFilterAst(ast.NodeTransformer):
         def visit_Name(self, node: ast.Name):
-            # pylint:disable=no-self-use
             if not isinstance(node.ctx, ast.Load):
                 raise InvalidArgumentException("Invalid filter: Modifying variables ({}) not allowed.".format(node.id))
             if node.id == "datetime":
@@ -56,14 +82,70 @@ def filterstr_to_filterfunc(filter_str: str, item_type: type):
     return filterfunc
 
 
+def get_cookies_from_instagram(domain, browser, cookie_file='', cookie_name=''):
+    supported_browsers = {
+        "brave": browser_cookie3.brave,
+        "chrome": browser_cookie3.chrome,
+        "chromium": browser_cookie3.chromium,
+        "edge": browser_cookie3.edge,
+        "firefox": browser_cookie3.firefox,
+        "librewolf": browser_cookie3.librewolf,
+        "opera": browser_cookie3.opera,
+        "opera_gx": browser_cookie3.opera_gx,
+        "safari": browser_cookie3.safari,
+        "vivaldi": browser_cookie3.vivaldi,
+    }
+
+    if browser not in supported_browsers:
+        raise InvalidArgumentException("Loading cookies from the specified browser failed\n"
+                                       "Supported browsers are Brave, Chrome, Chromium, Edge, Firefox, LibreWolf, "
+                                       "Opera, Opera_GX, Safari and Vivaldi")
+
+    cookies = {}
+    browser_cookies = list(supported_browsers[browser](cookie_file=cookie_file))
+
+    for cookie in browser_cookies:
+        if domain in cookie.domain:
+            cookies[cookie.name] = cookie.value
+
+    if cookies:
+        print(f"Cookies loaded successfully from {browser}")
+    else:
+        raise LoginException(f"No cookies found for Instagram in {browser}, "
+                             f"Are you logged in succesfully in {browser}?")
+
+    if cookie_name:
+        return cookies.get(cookie_name, {})
+    else:
+        return cookies
+
+
+def import_session(browser, instaloader, cookiefile):
+    cookie = get_cookies_from_instagram('instagram', browser, cookiefile)
+    if cookie is not None:
+        instaloader.context.update_cookies(cookie)
+        username = instaloader.test_login()
+        if not username:
+            raise LoginException(f"Not logged in. Are you logged in successfully in {browser}?")
+        instaloader.context.username = username
+        print(f"{username} has been successfully logged in.")
+        print(f"Next time use --login={username} to reuse the same session.")
+
+
 def _main(instaloader: Instaloader, targetlist: List[str],
           username: Optional[str] = None, password: Optional[str] = None,
           sessionfile: Optional[str] = None,
           download_profile_pic: bool = True, download_posts=True,
-          download_stories: bool = False, download_highlights: bool = False, download_tagged: bool = False,
+          download_stories: bool = False,
+          download_highlights: bool = False,
+          download_tagged: bool = False,
+          download_igtv: bool = False,
           fast_update: bool = False,
+          latest_stamps_file: Optional[str] = None,
           max_count: Optional[int] = None, post_filter_str: Optional[str] = None,
-          storyitem_filter_str: Optional[str] = None) -> None:
+          storyitem_filter_str: Optional[str] = None,
+          browser: Optional[str] = None,
+          cookiefile: Optional[str] = None) -> ExitCode:
     """Download set of profiles, hashtags etc. and handle logging in and session files if desired."""
     # Parse and generate filter function
     post_filter = None
@@ -74,8 +156,19 @@ def _main(instaloader: Instaloader, targetlist: List[str],
     if storyitem_filter_str is not None:
         storyitem_filter = filterstr_to_filterfunc(storyitem_filter_str, StoryItem)
         instaloader.context.log('Only download storyitems with property "{}".'.format(storyitem_filter_str))
+    latest_stamps = None
+    if latest_stamps_file is not None:
+        latest_stamps = LatestStamps(latest_stamps_file)
+        instaloader.context.log(f"Using latest stamps from {latest_stamps_file}.")
+    # load cookies if browser is not None
+    if browser and bc3_library:
+        import_session(browser.lower(), instaloader, cookiefile)
+    elif browser and not bc3_library:
+        raise InvalidArgumentException("browser_cookie3 library is needed to load cookies from browsers")
     # Login, if desired
     if username is not None:
+        if not re.match(r"^[A-Za-z0-9._]+$", username):
+            instaloader.context.error("Warning: Parameter \"{}\" for --login is not a valid username.".format(username))
         try:
             instaloader.load_session_from_file(username, sessionfile)
         except FileNotFoundError as err:
@@ -84,13 +177,35 @@ def _main(instaloader: Instaloader, targetlist: List[str],
             instaloader.context.log("Session file does not exist yet - Logging in.")
         if not instaloader.context.is_logged_in or username != instaloader.test_login():
             if password is not None:
-                instaloader.login(username, password)
+                try:
+                    instaloader.login(username, password)
+                except TwoFactorAuthRequiredException:
+                    # https://github.com/instaloader/instaloader/issues/1217
+                    instaloader.context.error("Warning: There have been reports of 2FA currently not working. "
+                                              "Consider importing session cookies from your browser with "
+                                              "--load-cookies.")
+                    while True:
+                        try:
+                            code = input("Enter 2FA verification code: ")
+                            instaloader.two_factor_login(code)
+                            break
+                        except BadCredentialsException as err:
+                            print(err, file=sys.stderr)
+                            pass
             else:
-                instaloader.interactive_login(username)
+                try:
+                    instaloader.interactive_login(username)
+                except KeyboardInterrupt:
+                    print("\nInterrupted by user.", file=sys.stderr)
+                    return ExitCode.USER_ABORTED
         instaloader.context.log("Logged in as %s." % username)
+    # since 4.2.9 login is required for geotags
+    if instaloader.download_geotags and not instaloader.context.is_logged_in:
+        instaloader.context.error("Warning: Login is required to download geotags of posts.")
     # Try block for KeyboardInterrupt (save session on ^C)
     profiles = set()
     anonymous_retry_profiles = set()
+    exit_code = ExitCode.SUCCESS
     try:
         # Generate set of profiles, already downloading non-profile targets
         for target in targetlist:
@@ -119,17 +234,21 @@ def _main(instaloader: Instaloader, targetlist: List[str],
             # strip '/' characters to be more shell-autocompletion-friendly
             target = target.rstrip('/')
             with instaloader.context.error_catcher(target):
-                if target[0] == '@':
+                if re.match(r"^@[A-Za-z0-9._]+$", target):
                     instaloader.context.log("Retrieving followees of %s..." % target[1:])
                     profile = Profile.from_username(instaloader.context, target[1:])
                     for followee in profile.get_followees():
                         instaloader.save_profile_id(followee)
                         profiles.add(followee)
-                elif target[0] == '#':
+                elif re.match(r"^#\w+$", target):
                     instaloader.download_hashtag(hashtag=target[1:], max_count=max_count, fast_update=fast_update,
-                                                 post_filter=post_filter)
-                elif target[0] == '-':
+                                                 post_filter=post_filter,
+                                                 profile_pic=download_profile_pic, posts=download_posts)
+                elif re.match(r"^-[A-Za-z0-9-_]+$", target):
                     instaloader.download_post(Post.from_shortcode(instaloader.context, target[1:]), target)
+                elif re.match(r"^%[0-9]+$", target):
+                    instaloader.download_location(location=target[1:], max_count=max_count, fast_update=fast_update,
+                                                  post_filter=post_filter)
                 elif target == ":feed":
                     instaloader.download_feed_posts(fast_update=fast_update, max_count=max_count,
                                                     post_filter=post_filter)
@@ -138,11 +257,12 @@ def _main(instaloader: Instaloader, targetlist: List[str],
                 elif target == ":saved":
                     instaloader.download_saved_posts(fast_update=fast_update, max_count=max_count,
                                                      post_filter=post_filter)
-                else:
+                elif re.match(r"^[A-Za-z0-9._]+$", target):
                     try:
-                        profile = instaloader.check_profile_id(target)
+                        profile = instaloader.check_profile_id(target, latest_stamps)
                         if instaloader.context.is_logged_in and profile.has_blocked_viewer:
-                            if download_profile_pic or ((download_posts or download_tagged) and not profile.is_private):
+                            if download_profile_pic or ((download_posts or download_tagged or download_igtv)
+                                                        and not profile.is_private):
                                 raise ProfileNotExistsException("{} blocked you; But we download her anonymously."
                                                                 .format(target))
                             else:
@@ -153,31 +273,48 @@ def _main(instaloader: Instaloader, targetlist: List[str],
                         # Not only our profile.has_blocked_viewer condition raises ProfileNotExistsException,
                         # check_profile_id() also does, since access to blocked profile may be responded with 404.
                         if instaloader.context.is_logged_in and (download_profile_pic or download_posts or
-                                                                 download_tagged):
+                                                                 download_tagged or download_igtv):
                             instaloader.context.log(err)
                             instaloader.context.log("Trying again anonymously, helps in case you are just blocked.")
                             with instaloader.anonymous_copy() as anonymous_loader:
                                 with instaloader.context.error_catcher():
-                                    anonymous_retry_profiles.add(anonymous_loader.check_profile_id(target))
+                                    anonymous_retry_profiles.add(anonymous_loader.check_profile_id(target,
+                                                                                                   latest_stamps))
                                     instaloader.context.error("Warning: {} will be downloaded anonymously (\"{}\")."
                                                               .format(target, err))
                         else:
                             raise
+                else:
+                    target_type = {
+                        '#': 'hashtag',
+                        '%': 'location',
+                        '-': 'shortcode',
+                    }.get(target[0], 'username')
+                    raise ProfileNotExistsException('Invalid {} {}'.format(target_type, target))
         if len(profiles) > 1:
             instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
                                                                          ' '.join([p.username for p in profiles])))
+        if instaloader.context.iphone_support and profiles and (download_profile_pic or download_posts) and \
+           not instaloader.context.is_logged_in:
+            instaloader.context.log("Hint: Login to download higher-quality versions of pictures.")
         instaloader.download_profiles(profiles,
-                                      download_profile_pic, download_posts, download_tagged, download_highlights,
-                                      download_stories, fast_update, post_filter, storyitem_filter)
+                                      download_profile_pic, download_posts, download_tagged, download_igtv,
+                                      download_highlights, download_stories,
+                                      fast_update, post_filter, storyitem_filter, latest_stamps=latest_stamps)
         if anonymous_retry_profiles:
             instaloader.context.log("Downloading anonymously: {}"
                                     .format(' '.join([p.username for p in anonymous_retry_profiles])))
             with instaloader.anonymous_copy() as anonymous_loader:
                 anonymous_loader.download_profiles(anonymous_retry_profiles,
-                                                   download_profile_pic, download_posts, download_tagged,
-                                                   fast_update=fast_update, post_filter=post_filter)
+                                                   download_profile_pic, download_posts, download_tagged, download_igtv,
+                                                   fast_update=fast_update, post_filter=post_filter,
+                                                   latest_stamps=latest_stamps)
     except KeyboardInterrupt:
         print("\nInterrupted by user.", file=sys.stderr)
+        exit_code = ExitCode.USER_ABORTED
+    except AbortDownloadException as exc:
+        print("\nDownload aborted: {}.".format(exc), file=sys.stderr)
+        exit_code = ExitCode.DOWNLOAD_ABORTED
     # Save session if it is useful
     if instaloader.context.is_logged_in:
         instaloader.save_session_to_file(sessionfile)
@@ -187,14 +324,15 @@ def _main(instaloader: Instaloader, targetlist: List[str],
             # Instaloader did at least save a session file
             instaloader.context.log("No targets were specified, thus nothing has been downloaded.")
         else:
-            # Instloader did not do anything
+            # Instaloader did not do anything
             instaloader.context.log("usage:" + usage_string())
+            exit_code = ExitCode.INIT_FAILURE
+    return exit_code
 
 
 def main():
     parser = ArgumentParser(description=__doc__, add_help=False, usage=usage_string(),
-                            epilog="Report issues at https://github.com/instaloader/instaloader/issues. "
-                                   "The complete documentation can be found at "
+                            epilog="The complete documentation can be found at "
                                    "https://instaloader.github.io/.",
                             fromfile_prefix_chars='+')
 
@@ -205,15 +343,17 @@ def main():
                            help="Download profile. If an already-downloaded profile has been renamed, Instaloader "
                                 "automatically finds it by its unique ID and renames the folder likewise.")
     g_targets.add_argument('_at_profile', nargs='*', metavar="@profile",
-                           help="Download all followees of profile. Requires --login. "
+                           help="Download all followees of profile. Requires login. "
                                 "Consider using :feed rather than @yourself.")
     g_targets.add_argument('_hashtag', nargs='*', metavar='"#hashtag"', help="Download #hashtag.")
+    g_targets.add_argument('_location', nargs='*', metavar='%location_id',
+                           help="Download %%location_id. Requires login.")
     g_targets.add_argument('_feed', nargs='*', metavar=":feed",
-                           help="Download pictures from your feed. Requires --login.")
+                           help="Download pictures from your feed. Requires login.")
     g_targets.add_argument('_stories', nargs='*', metavar=":stories",
-                           help="Download the stories of your followees. Requires --login.")
+                           help="Download the stories of your followees. Requires login.")
     g_targets.add_argument('_saved', nargs='*', metavar=":saved",
-                           help="Download the posts that you marked as saved. Requires --login.")
+                           help="Download the posts that you marked as saved. Requires login.")
     g_targets.add_argument('_singlepost', nargs='*', metavar="-- -shortcode",
                            help="Download the post with the given shortcode")
     g_targets.add_argument('_json', nargs='*', metavar="filename.json[.xz]",
@@ -231,6 +371,8 @@ def main():
                         help="Do not download regular posts.")
     g_prof.add_argument('--no-profile-pic', action='store_true',
                         help='Do not download profile picture.')
+    g_post.add_argument('--slide', action='store',
+                        help='Set what image/interval of a sidecar you want to download.')
     g_post.add_argument('--no-pictures', action='store_true',
                         help='Do not download post pictures. Cannot be used together with --fast-update. '
                              'Implies --no-video-thumbnails, does not imply --no-videos.')
@@ -242,11 +384,11 @@ def main():
                         help='Download geotags when available. Geotags are stored as a '
                              'text file with the location\'s name and a Google Maps link. '
                              'This requires an additional request to the Instagram '
-                             'server for each picture, which is why it is disabled by default.')
+                             'server for each picture. Requires login.')
     g_post.add_argument('-C', '--comments', action='store_true',
                         help='Download and update comments for each post. '
                              'This requires an additional request to the Instagram '
-                             'server for each post, which is why it is disabled by default.')
+                             'server for each post, which is why it is disabled by default. Requires login.')
     g_post.add_argument('--no-captions', action='store_true',
                         help='Do not create txt files.')
     g_post.add_argument('--post-metadata-txt', action='append',
@@ -260,20 +402,25 @@ def main():
     g_post.add_argument('--no-compress-json', action='store_true',
                         help='Do not xz compress JSON files, rather create pretty formatted JSONs.')
     g_prof.add_argument('-s', '--stories', action='store_true',
-                        help='Also download stories of each profile that is downloaded. Requires --login.')
+                        help='Also download stories of each profile that is downloaded. Requires login.')
     g_prof.add_argument('--stories-only', action='store_true',
                         help=SUPPRESS)
     g_prof.add_argument('--highlights', action='store_true',
-                        help='Also download highlights of each profile that is downloaded. Requires --login.')
+                        help='Also download highlights of each profile that is downloaded. Requires login.')
     g_prof.add_argument('--tagged', action='store_true',
                         help='Also download posts where each profile is tagged.')
+    g_prof.add_argument('--igtv', action='store_true',
+                        help='Also download IGTV videos.')
 
     g_cond = parser.add_argument_group("Which Posts to Download")
 
     g_cond.add_argument('-F', '--fast-update', action='store_true',
                         help='For each target, stop when encountering the first already-downloaded picture. This '
                              'flag is recommended when you use Instaloader to update your personal Instagram archive.')
-
+    g_cond.add_argument('--latest-stamps', nargs='?', metavar='STAMPSFILE', const=get_default_stamps_filename(),
+                        help='Store the timestamps of latest media scraped for each profile. This allows updating '
+                             'your personal Instagram archive even if you delete the destination directories. '
+                             'If STAMPSFILE is not provided, defaults to ' + get_default_stamps_filename())
     g_cond.add_argument('--post-filter', '--only-if', metavar='filter',
                         help='Expression that, if given, must evaluate to True for each post to be downloaded. Must be '
                              'a syntactically valid python expression. Variables are evaluated to '
@@ -285,15 +432,20 @@ def main():
 
     g_cond.add_argument('-c', '--count',
                         help='Do not attempt to download more than COUNT posts. '
-                             'Applies only to #hashtag and :feed.')
+                             'Applies to #hashtag, %%location_id, :feed, and :saved.')
 
     g_login = parser.add_argument_group('Login (Download Private Profiles)',
                                         'Instaloader can login to Instagram. This allows downloading private profiles. '
                                         'To login, pass the --login option. Your session cookie (not your password!) '
                                         'will be saved to a local file to be reused next time you want Instaloader '
-                                        'to login.')
+                                        'to login. Instead of --login, the --load-cookies option can be used to '
+                                        'import a session from a browser.')
     g_login.add_argument('-l', '--login', metavar='YOUR-USERNAME',
                          help='Login name (profile name) for your Instagram account.')
+    g_login.add_argument('-b', '--load-cookies', metavar='BROWSER-NAME',
+                         help='Browser name to load cookies from Instagram')
+    g_login.add_argument('-B', '--cookiefile', metavar='COOKIE-FILE',
+                         help='Cookie file of a profile to load cookies')
     g_login.add_argument('-f', '--sessionfile',
                          help='Path for loading and storing session key file. '
                               'Defaults to ' + get_default_session_filename("<login_name>"))
@@ -308,18 +460,40 @@ def main():
                             '{target} is replaced by the target you specified, i.e. either :feed, #hashtag or the '
                             'profile name. Defaults to \'{target}\'.')
     g_how.add_argument('--filename-pattern',
-                       help='Prefix of filenames, relative to the directory given with '
+                       help='Prefix of filenames for posts and stories, relative to the directory given with '
                             '--dirname-pattern. {profile} is replaced by the profile name,'
                             '{target} is replaced by the target you specified, i.e. either :feed'
                             '#hashtag or the profile name. Defaults to \'{date_utc}_UTC\'')
+    g_how.add_argument('--title-pattern',
+                       help='Prefix of filenames for profile pics, hashtag profile pics, and highlight covers. '
+                            'Defaults to \'{date_utc}_UTC_{typename}\' if --dirname-pattern contains \'{target}\' '
+                            'or \'{dirname}\', or if --dirname-pattern is not specified. Otherwise defaults to '
+                            '\'{target}_{date_utc}_UTC_{typename}\'.')
+    g_how.add_argument('--resume-prefix', metavar='PREFIX',
+                       help='Prefix for filenames that are used to save the information to resume an interrupted '
+                            'download.')
+    g_how.add_argument('--sanitize-paths', action='store_true',
+                       help='Sanitize paths so that the resulting file and directory names are valid on both '
+                            'Windows and Unix.')
+    g_how.add_argument('--no-resume', action='store_true',
+                       help='Do not resume a previously-aborted download iteration, and do not save such information '
+                            'when interrupted.')
+    g_how.add_argument('--use-aged-resume-files', action='store_true', help=SUPPRESS)
     g_how.add_argument('--user-agent',
                        help='User Agent to use for HTTP requests. Defaults to \'{}\'.'.format(default_user_agent()))
     g_how.add_argument('-S', '--no-sleep', action='store_true', help=SUPPRESS)
-    g_how.add_argument('--graphql-rate-limit', type=int, help=SUPPRESS)
     g_how.add_argument('--max-connection-attempts', metavar='N', type=int, default=3,
                        help='Maximum number of connection attempts until a request is aborted. Defaults to 3. If a '
                             'connection fails, it can be manually skipped by hitting CTRL+C. Set this to 0 to retry '
                             'infinitely.')
+    g_how.add_argument('--commit-mode', action='store_true', help=SUPPRESS)
+    g_how.add_argument('--request-timeout', metavar='N', type=float, default=300.0,
+                       help='Seconds to wait before timing out a connection request. Defaults to 300.')
+    g_how.add_argument('--abort-on', type=http_status_code_list, metavar="STATUS_CODES",
+                       help='Comma-separated list of HTTP status codes that cause Instaloader to abort, bypassing all '
+                            'retry logic.')
+    g_how.add_argument('--no-iphone', action='store_true',
+                        help='Do not attempt to download iPhone version of images and videos.')
 
     g_misc = parser.add_argument_group('Miscellaneous Options')
     g_misc.add_argument('-q', '--quiet', action='store_true',
@@ -332,15 +506,15 @@ def main():
 
     args = parser.parse_args()
     try:
-        if args.login is None and (args.stories or args.stories_only):
-            print("--login=USERNAME required to download stories.", file=sys.stderr)
+        if (args.login is None and args.load_cookies is None) and (args.stories or args.stories_only):
+            print("Login is required to download stories.", file=sys.stderr)
             args.stories = False
             if args.stories_only:
-                raise SystemExit(1)
+                raise InvalidArgumentException()
 
         if ':feed-all' in args.profile or ':feed-liked' in args.profile:
-            raise SystemExit(":feed-all and :feed-liked were removed. Use :feed as target and "
-                             "eventually --post-filter=viewer_has_liked.")
+            raise InvalidArgumentException(":feed-all and :feed-liked were removed. Use :feed as target and "
+                                           "eventually --post-filter=viewer_has_liked.")
 
         post_metadata_txt_pattern = '\n'.join(args.post_metadata_txt) if args.post_metadata_txt else None
         storyitem_metadata_txt_pattern = '\n'.join(args.storyitem_metadata_txt) if args.storyitem_metadata_txt else None
@@ -350,11 +524,18 @@ def main():
                 post_metadata_txt_pattern = ''
                 storyitem_metadata_txt_pattern = ''
             else:
-                raise SystemExit("--no-captions and --post-metadata-txt or --storyitem-metadata-txt given; "
-                                 "That contradicts.")
+                raise InvalidArgumentException("--no-captions and --post-metadata-txt or --storyitem-metadata-txt "
+                                               "given; That contradicts.")
+
+        if args.no_resume and args.resume_prefix:
+            raise InvalidArgumentException("--no-resume and --resume-prefix given; That contradicts.")
+        resume_prefix = (args.resume_prefix if args.resume_prefix else 'iterator') if not args.no_resume else None
 
         if args.no_pictures and args.fast_update:
-            raise SystemExit('--no-pictures and --fast-update cannot be used together.')
+            raise InvalidArgumentException('--no-pictures and --fast-update cannot be used together.')
+
+        if args.login and args.load_cookies:
+            raise InvalidArgumentException('--load-cookies and --login cannot be used together.')
 
         # Determine what to download
         download_profile_pic = not args.no_profile_pic or args.profile_pic_only
@@ -370,25 +551,46 @@ def main():
                              compress_json=not args.no_compress_json,
                              post_metadata_txt_pattern=post_metadata_txt_pattern,
                              storyitem_metadata_txt_pattern=storyitem_metadata_txt_pattern,
-                             graphql_rate_limit=args.graphql_rate_limit,
-                             max_connection_attempts=args.max_connection_attempts)
-        _main(loader,
-              args.profile,
-              username=args.login.lower() if args.login is not None else None,
-              password=args.password,
-              sessionfile=args.sessionfile,
-              download_profile_pic=download_profile_pic,
-              download_posts=download_posts,
-              download_stories=download_stories,
-              download_highlights=args.highlights,
-              download_tagged=args.tagged,
-              fast_update=args.fast_update,
-              max_count=int(args.count) if args.count is not None else None,
-              post_filter_str=args.post_filter,
-              storyitem_filter_str=args.storyitem_filter)
+                             max_connection_attempts=args.max_connection_attempts,
+                             request_timeout=args.request_timeout,
+                             resume_prefix=resume_prefix,
+                             check_resume_bbd=not args.use_aged_resume_files,
+                             slide=args.slide,
+                             fatal_status_codes=args.abort_on,
+                             iphone_support=not args.no_iphone,
+                             title_pattern=args.title_pattern,
+                             sanitize_paths=args.sanitize_paths)
+        exit_code = _main(loader,
+                          args.profile,
+                          username=args.login.lower() if args.login is not None else None,
+                          password=args.password,
+                          sessionfile=args.sessionfile,
+                          download_profile_pic=download_profile_pic,
+                          download_posts=download_posts,
+                          download_stories=download_stories,
+                          download_highlights=args.highlights,
+                          download_tagged=args.tagged,
+                          download_igtv=args.igtv,
+                          fast_update=args.fast_update,
+                          latest_stamps_file=args.latest_stamps,
+                          max_count=int(args.count) if args.count is not None else None,
+                          post_filter_str=args.post_filter,
+                          storyitem_filter_str=args.storyitem_filter,
+                          browser=args.load_cookies,
+                          cookiefile=args.cookiefile)
         loader.close()
+        if loader.has_stored_errors:
+            exit_code = ExitCode.NON_FATAL_ERROR
+    except InvalidArgumentException as err:
+        print(err, file=sys.stderr)
+        exit_code = ExitCode.INIT_FAILURE
+    except LoginException as err:
+        print(err, file=sys.stderr)
+        exit_code = ExitCode.LOGIN_FAILURE
     except InstaloaderException as err:
-        raise SystemExit("Fatal error: %s" % err)
+        print("Fatal error: %s" % err)
+        exit_code = ExitCode.UNEXPECTED_ERROR
+    sys.exit(exit_code)
 
 
 if __name__ == "__main__":

instaloader/exceptions.py

@@ -9,10 +9,6 @@ class QueryReturnedBadRequestException(InstaloaderException):
     pass
 
 
-class QueryReturnedNotFoundException(InstaloaderException):
-    pass
-
-
 class QueryReturnedForbiddenException(InstaloaderException):
     pass
 
@@ -22,6 +18,10 @@ class ProfileNotExistsException(InstaloaderException):
 
 
 class ProfileHasNoPicsException(InstaloaderException):
+    """
+    .. deprecated:: 4.2.2
+       Not raised anymore.
+    """
     pass
 
 
@@ -33,6 +33,14 @@ class LoginRequiredException(InstaloaderException):
     pass
 
 
+class LoginException(InstaloaderException):
+    pass
+
+
+class TwoFactorAuthRequiredException(LoginException):
+    pass
+
+
 class InvalidArgumentException(InstaloaderException):
     pass
 
@@ -41,7 +49,7 @@ class BadResponseException(InstaloaderException):
     pass
 
 
-class BadCredentialsException(InstaloaderException):
+class BadCredentialsException(LoginException):
     pass
 
 
@@ -49,5 +57,28 @@ class ConnectionException(InstaloaderException):
     pass
 
 
+class PostChangedException(InstaloaderException):
+    """.. versionadded:: 4.2.2"""
+    pass
+
+
+class QueryReturnedNotFoundException(ConnectionException):
+    pass
+
+
 class TooManyRequestsException(ConnectionException):
     pass
+
+class IPhoneSupportDisabledException(InstaloaderException):
+    pass
+
+class AbortDownloadException(Exception):
+    """
+    Exception that is not catched in the error catchers inside the download loop and so aborts the
+    download loop.
+
+    This exception is not a subclass of ``InstaloaderException``.
+
+    .. versionadded:: 4.7
+    """
+    pass

instaloader/instaloadercontext.py

@@ -1,35 +1,67 @@
 import hashlib
 import json
+import os
 import pickle
 import random
-import re
 import shutil
 import sys
 import textwrap
 import time
 import urllib.parse
-from contextlib import contextmanager
+import uuid
+from contextlib import contextmanager, suppress
 from datetime import datetime, timedelta
-from typing import Any, Callable, Dict, Iterator, Optional
+from functools import partial
+from typing import Any, Callable, Dict, Iterator, List, Optional, Union
 
 import requests
 import requests.utils
-import urllib3
 
 from .exceptions import *
 
 
-def copy_session(session: requests.Session) -> requests.Session:
+def copy_session(session: requests.Session, request_timeout: Optional[float] = None) -> requests.Session:
     """Duplicates a requests.Session."""
     new = requests.Session()
     new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
-    new.headers = session.headers.copy()
+    new.headers = session.headers.copy()  # type: ignore
+    # Override default timeout behavior.
+    # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+    new.request = partial(new.request, timeout=request_timeout)  # type: ignore
     return new
 
 
 def default_user_agent() -> str:
-    return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \
-           '(KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36'
+    return ('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 '
+            '(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36')
+
+
+def default_iphone_headers() -> Dict[str, Any]:
+    return {'User-Agent': 'Instagram 273.0.0.16.70 (iPad13,8; iOS 16_3; en_US; en-US; ' \
+                          'scale=2.00; 2048x2732; 452417278) AppleWebKit/420+',
+            'x-ads-opt-out': '1',
+            'x-bloks-is-panorama-enabled': 'true',
+            'x-bloks-version-id': '01507c21540f73e2216b6f62a11a5b5e51aa85491b72475c080da35b1228ddd6',
+            'x-fb-client-ip': 'True',
+            'x-fb-connection-type': 'wifi',
+            'x-fb-http-engine': 'Liger',
+            'x-fb-server-cluster': 'True',
+            'x-fb': '1',
+            'x-ig-abr-connection-speed-kbps': '2',
+            'x-ig-app-id': '124024574287414',
+            'x-ig-app-locale': 'en-US',
+            'x-ig-app-startup-country': 'US',
+            'x-ig-bandwidth-speed-kbps': '0.000',
+            'x-ig-capabilities': '36r/F/8=',
+            'x-ig-connection-speed': '{}kbps'.format(random.randint(1000, 20000)),
+            'x-ig-connection-type': 'WiFi',
+            'x-ig-device-locale': 'en-US',
+            'x-ig-mapped-locale': 'en-US',
+            'x-ig-timezone-offset': str((datetime.now().astimezone().utcoffset() or timedelta(seconds=0)).seconds),
+            'x-ig-www-claim': '0',
+            'x-pigeon-session-id': str(uuid.uuid4()),
+            'x-tigon-is-retry': 'False',
+            'x-whatsapp': '0'}
 
 
 class InstaloaderContext:
@@ -48,42 +80,57 @@ class InstaloaderContext:
     """
 
     def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
-                 graphql_count_per_slidingwindow: Optional[int] = None, max_connection_attempts: int = 3):
+                 max_connection_attempts: int = 3, request_timeout: float = 300.0,
+                 rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None,
+                 fatal_status_codes: Optional[List[int]] = None,
+                 iphone_support: bool = True):
 
         self.user_agent = user_agent if user_agent is not None else default_user_agent()
+        self.request_timeout = request_timeout
         self._session = self.get_anonymous_session()
         self.username = None
+        self.user_id = None
         self.sleep = sleep
         self.quiet = quiet
         self.max_connection_attempts = max_connection_attempts
         self._graphql_page_length = 50
-        self.graphql_count_per_slidingwindow = graphql_count_per_slidingwindow or 20
         self._root_rhx_gis = None
+        self.two_factor_auth_pending = None
+        self.iphone_support = iphone_support
+        self.iphone_headers = default_iphone_headers()
 
         # error log, filled with error() and printed at the end of Instaloader.main()
-        self.error_log = []
+        self.error_log: List[str] = []
 
-        # For the adaption of sleep intervals (rate control)
-        self.query_timestamps = list()
+        self._rate_controller = rate_controller(self) if rate_controller is not None else RateController(self)
 
         # Can be set to True for testing, disables supression of InstaloaderContext._error_catcher
         self.raise_all_errors = False
 
+        # HTTP status codes that should cause an AbortDownloadException
+        self.fatal_status_codes = fatal_status_codes or []
+
         # Cache profile from id (mapping from id to Profile)
-        self.profile_id_cache = dict()
+        self.profile_id_cache: Dict[int, Any] = dict()
 
     @contextmanager
     def anonymous_copy(self):
         session = self._session
         username = self.username
+        user_id = self.user_id
+        iphone_headers = self.iphone_headers
         self._session = self.get_anonymous_session()
         self.username = None
+        self.user_id = None
+        self.iphone_headers = default_iphone_headers()
         try:
             yield self
         finally:
             self._session.close()
             self.username = username
             self._session = session
+            self.user_id = user_id
+            self.iphone_headers = iphone_headers
 
     @property
     def is_logged_in(self) -> bool:
@@ -104,10 +151,17 @@ class InstaloaderContext:
         if repeat_at_end:
             self.error_log.append(msg)
 
+    @property
+    def has_stored_errors(self) -> bool:
+        """Returns whether any error has been reported and stored to be repeated at program termination.
+
+        .. versionadded: 4.12"""
+        return bool(self.error_log)
+
     def close(self):
         """Print error log and close session"""
         if self.error_log and not self.quiet:
-            print("\nErrors occured:", file=sys.stderr)
+            print("\nErrors or warnings occurred:", file=sys.stderr)
             for err in self.error_log:
                 print(err, file=sys.stderr)
         self._session.close()
@@ -143,7 +197,6 @@ class InstaloaderContext:
         if empty_session_only:
             del header['Host']
             del header['Origin']
-            del header['Referer']
             del header['X-Instagram-AJAX']
             del header['X-Requested-With']
         return header
@@ -155,76 +208,186 @@ class InstaloaderContext:
                                 'ig_vw': '1920', 'csrftoken': '',
                                 's_network': '', 'ds_user_id': ''})
         session.headers.update(self._default_http_header(empty_session_only=True))
+        # Override default timeout behavior.
+        # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+        session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
         return session
 
-    def save_session_to_file(self, sessionfile):
-        """Not meant to be used directly, use :meth:`Instaloader.save_session_to_file`."""
-        pickle.dump(requests.utils.dict_from_cookiejar(self._session.cookies), sessionfile)
+    def save_session(self):
+        """Not meant to be used directly, use :meth:`Instaloader.save_session`."""
+        return requests.utils.dict_from_cookiejar(self._session.cookies)
 
-    def load_session_from_file(self, username, sessionfile):
-        """Not meant to be used directly, use :meth:`Instaloader.load_session_from_file`."""
+    def update_cookies(self, cookie):
+        """.. versionadded:: 4.11"""
+        self._session.cookies.update(cookie)
+
+    def load_session(self, username, sessiondata):
+        """Not meant to be used directly, use :meth:`Instaloader.load_session`."""
         session = requests.Session()
-        session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile))
+        session.cookies = requests.utils.cookiejar_from_dict(sessiondata)
         session.headers.update(self._default_http_header())
         session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']})
+        # Override default timeout behavior.
+        # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+        session.request = partial(session.request, timeout=self.request_timeout)  # type: ignore
         self._session = session
         self.username = username
 
+    def save_session_to_file(self, sessionfile):
+        """Not meant to be used directly, use :meth:`Instaloader.save_session_to_file`."""
+        pickle.dump(self.save_session(), sessionfile)
+
+    def load_session_from_file(self, username, sessionfile):
+        """Not meant to be used directly, use :meth:`Instaloader.load_session_from_file`."""
+        self.load_session(username, pickle.load(sessionfile))
+
     def test_login(self) -> Optional[str]:
         """Not meant to be used directly, use :meth:`Instaloader.test_login`."""
-        data = self.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})
-        return data["data"]["user"]["username"] if data["data"]["user"] is not None else None
+        try:
+            data = self.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})
+            return data["data"]["user"]["username"] if data["data"]["user"] is not None else None
+        except (AbortDownloadException, ConnectionException) as err:
+            self.error(f"Error when checking if logged in: {err}")
+            return None
 
     def login(self, user, passwd):
         """Not meant to be used directly, use :meth:`Instaloader.login`.
 
-        :raises InvalidArgumentException: If the provided username does not exist.
         :raises BadCredentialsException: If the provided password is wrong.
-        :raises ConnectionException: If connection to Instagram failed."""
+        :raises TwoFactorAuthRequiredException: First step of 2FA login done, now call
+           :meth:`Instaloader.two_factor_login`.
+        :raises LoginException: An error happened during login (for example, and invalid response).
+           Or if the provided username does not exist.
+
+        .. versionchanged:: 4.12
+           Raises LoginException instead of ConnectionException when an error happens.
+           Raises LoginException instead of InvalidArgumentException when the username does not exist.
+        """
+        # pylint:disable=import-outside-toplevel
         import http.client
         # pylint:disable=protected-access
         http.client._MAXHEADERS = 200
         session = requests.Session()
         session.cookies.update({'sessionid': '', 'mid': '', 'ig_pr': '1',
-                                'ig_vw': '1920', 'csrftoken': '',
+                                'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '',
                                 's_network': '', 'ds_user_id': ''})
         session.headers.update(self._default_http_header())
-        session.headers.update({'X-CSRFToken': self.get_json('', {})['config']['csrf_token']})
-        # Not using self.get_json() here, because we need to access csrftoken cookie
-        self._sleep()
-        login = session.post('https://www.instagram.com/accounts/login/ajax/',
-                             data={'password': passwd, 'username': user}, allow_redirects=True)
-        if login.status_code != 200:
-            raise ConnectionException("Login error: {} {}".format(login.status_code, login.reason))
-        resp_json = login.json()
+        # Override default timeout behavior.
+        # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
+        session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
+
+        # Make a request to Instagram's root URL, which will set the session's csrftoken cookie
+        # Not using self.get_json() here, because we need to access the cookie
+        session.get('https://www.instagram.com/')
+        # Add session's csrftoken cookie to session headers
+        csrf_token = session.cookies.get_dict()['csrftoken']
+        session.headers.update({'X-CSRFToken': csrf_token})
+
+        self.do_sleep()
+        # Workaround credits to pgrimaud.
+        # See: https://github.com/pgrimaud/instagram-user-feed/commit/96ad4cf54d1ad331b337f325c73e664999a6d066
+        enc_password = '#PWD_INSTAGRAM_BROWSER:0:{}:{}'.format(int(datetime.now().timestamp()), passwd)
+        login = session.post('https://www.instagram.com/api/v1/web/accounts/login/ajax/',
+                             data={'enc_password': enc_password, 'username': user}, allow_redirects=True)
+        try:
+            resp_json = login.json()
+
+        except json.decoder.JSONDecodeError as err:
+            raise LoginException(
+                "Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason)
+            ) from err
+        if resp_json.get('two_factor_required'):
+            two_factor_session = copy_session(session, self.request_timeout)
+            two_factor_session.headers.update({'X-CSRFToken': csrf_token})
+            two_factor_session.cookies.update({'csrftoken': csrf_token})
+            self.two_factor_auth_pending = (two_factor_session,
+                                            user,
+                                            resp_json['two_factor_info']['two_factor_identifier'])
+            raise TwoFactorAuthRequiredException("Login error: two-factor authentication required.")
+        if resp_json.get('checkpoint_url'):
+            raise LoginException(
+                f"Login: Checkpoint required. Point your browser to {resp_json.get('checkpoint_url')} - "
+                f"follow the instructions, then retry."
+            )
         if resp_json['status'] != 'ok':
             if 'message' in resp_json:
-                raise ConnectionException("Login error: \"{}\" status, message \"{}\".".format(resp_json['status'],
-                                                                                               resp_json['message']))
+                raise LoginException("Login error: \"{}\" status, message \"{}\".".format(resp_json['status'],
+                                                                                          resp_json['message']))
             else:
-                raise ConnectionException("Login error: \"{}\" status.".format(resp_json['status']))
+                raise LoginException("Login error: \"{}\" status.".format(resp_json['status']))
+        if 'authenticated' not in resp_json:
+            # Issue #472
+            if 'message' in resp_json:
+                raise LoginException("Login error: Unexpected response, \"{}\".".format(resp_json['message']))
+            else:
+                raise LoginException("Login error: Unexpected response, this might indicate a blocked IP.")
         if not resp_json['authenticated']:
             if resp_json['user']:
                 # '{"authenticated": false, "user": true, "status": "ok"}'
                 raise BadCredentialsException('Login error: Wrong password.')
             else:
                 # '{"authenticated": false, "user": false, "status": "ok"}'
-                # Raise InvalidArgumentException rather than BadCredentialException, because BadCredentialException
+                # Raise LoginException rather than BadCredentialException, because BadCredentialException
                 # triggers re-asking of password in Instaloader.interactive_login(), which makes no sense if the
                 # username is invalid.
-                raise InvalidArgumentException('Login error: User {} does not exist.'.format(user))
+                raise LoginException('Login error: User {} does not exist.'.format(user))
         # '{"authenticated": true, "user": true, "userId": ..., "oneTapPrompt": false, "status": "ok"}'
         session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
         self._session = session
         self.username = user
+        self.user_id = resp_json['userId']
 
-    def _sleep(self):
+    def two_factor_login(self, two_factor_code):
+        """Second step of login if 2FA is enabled.
+        Not meant to be used directly, use :meth:`Instaloader.two_factor_login`.
+
+        :raises InvalidArgumentException: No two-factor authentication pending.
+        :raises BadCredentialsException: 2FA verification code invalid.
+
+        .. versionadded:: 4.2"""
+        if not self.two_factor_auth_pending:
+            raise InvalidArgumentException("No two-factor authentication pending.")
+        (session, user, two_factor_id) = self.two_factor_auth_pending
+
+        login = session.post('https://www.instagram.com/accounts/login/ajax/two_factor/',
+                             data={'username': user, 'verificationCode': two_factor_code, 'identifier': two_factor_id},
+                             allow_redirects=True)
+        resp_json = login.json()
+        if resp_json['status'] != 'ok':
+            if 'message' in resp_json:
+                raise BadCredentialsException("2FA error: {}".format(resp_json['message']))
+            else:
+                raise BadCredentialsException("2FA error: \"{}\" status.".format(resp_json['status']))
+        session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
+        self._session = session
+        self.username = user
+        self.two_factor_auth_pending = None
+
+    def do_sleep(self):
         """Sleep a short time if self.sleep is set. Called before each request to instagram.com."""
         if self.sleep:
-            time.sleep(min(random.expovariate(0.6), 5.0))
+            time.sleep(min(random.expovariate(0.6), 15.0))
+
+    @staticmethod
+    def _response_error(resp: requests.Response) -> str:
+        extra_from_json: Optional[str] = None
+        with suppress(json.decoder.JSONDecodeError):
+            resp_json = resp.json()
+            if "status" in resp_json:
+                extra_from_json = (
+                    f"\"{resp_json['status']}\" status, message \"{resp_json['message']}\""
+                    if "message" in resp_json
+                    else f"\"{resp_json['status']}\" status"
+                )
+        return (
+            f"{resp.status_code} {resp.reason}"
+            f"{f' - {extra_from_json}' if extra_from_json is not None else ''}"
+            f" when accessing {resp.url}"
+        )
 
     def get_json(self, path: str, params: Dict[str, Any], host: str = 'www.instagram.com',
-                 session: Optional[requests.Session] = None, _attempt=1) -> Dict[str, Any]:
+                 session: Optional[requests.Session] = None, _attempt=1,
+                 response_headers: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
         """JSON request to Instagram.
 
         :param path: URL, relative to the given domain which defaults to www.instagram.com/
@@ -236,84 +399,75 @@ class InstaloaderContext:
         :raises QueryReturnedNotFoundException: When the server responds with a 404.
         :raises ConnectionException: When query repeatedly failed.
         """
-        def graphql_query_waittime(untracked_queries: bool = False) -> int:
-            sliding_window = 660
-            if not self.query_timestamps:
-                return sliding_window if untracked_queries else 0
-            current_time = time.monotonic()
-            self.query_timestamps = list(filter(lambda t: t > current_time - sliding_window, self.query_timestamps))
-            if len(self.query_timestamps) < self.graphql_count_per_slidingwindow and not untracked_queries:
-                return 0
-            return round(min(self.query_timestamps) + sliding_window - current_time) + 6
         is_graphql_query = 'query_hash' in params and 'graphql/query' in path
-        # some queries are not rate limited if invoked anonymously:
-        query_not_limited = is_graphql_query and not self.is_logged_in \
-                            and params['query_hash'] in ['9ca88e465c3f866a76f7adee3871bdd8',
-                                                         '472f257a40c653c64c666ce877d59d2b']
-        if is_graphql_query and not query_not_limited:
-            waittime = graphql_query_waittime()
-            if waittime > 0:
-                self.log('\nToo many queries in the last time. Need to wait {} seconds, until {:%H:%M}.'
-                         .format(waittime, datetime.now() + timedelta(seconds=waittime)))
-                time.sleep(waittime)
-            if self.query_timestamps is not None:
-                self.query_timestamps.append(time.monotonic())
-            else:
-                self.query_timestamps = [time.monotonic()]
+        is_iphone_query = host == 'i.instagram.com'
+        is_other_query = not is_graphql_query and host == "www.instagram.com"
         sess = session if session else self._session
         try:
-            self._sleep()
+            self.do_sleep()
+            if is_graphql_query:
+                self._rate_controller.wait_before_query(params['query_hash'])
+            if is_iphone_query:
+                self._rate_controller.wait_before_query('iphone')
+            if is_other_query:
+                self._rate_controller.wait_before_query('other')
             resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False)
+            if resp.status_code in self.fatal_status_codes:
+                redirect = " redirect to {}".format(resp.headers['location']) if 'location' in resp.headers else ""
+                body = ""
+                if resp.headers['Content-Type'].startswith('application/json'):
+                    body = ': ' + resp.text[:500] + ('…' if len(resp.text) > 501 else '')
+                raise AbortDownloadException("Query to https://{}/{} responded with \"{} {}\"{}{}".format(
+                    host, path, resp.status_code, resp.reason, redirect, body
+                ))
             while resp.is_redirect:
                 redirect_url = resp.headers['location']
                 self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url))
+                if (redirect_url.startswith('https://www.instagram.com/accounts/login') or
+                    redirect_url.startswith('https://i.instagram.com/accounts/login')):
+                    if not self.is_logged_in:
+                        raise LoginRequiredException("Redirected to login page. Use --login or --load-cookies.")
+                    raise AbortDownloadException("Redirected to login page. You've been logged out, please wait " +
+                                                 "some time, recreate the session and try again")
                 if redirect_url.startswith('https://{}/'.format(host)):
                     resp = sess.get(redirect_url if redirect_url.endswith('/') else redirect_url + '/',
                                     params=params, allow_redirects=False)
                 else:
                     break
+            if response_headers is not None:
+                response_headers.clear()
+                response_headers.update(resp.headers)
             if resp.status_code == 400:
-                raise QueryReturnedBadRequestException("400 Bad Request")
+                raise QueryReturnedBadRequestException(self._response_error(resp))
             if resp.status_code == 404:
-                raise QueryReturnedNotFoundException("404 Not Found")
+                raise QueryReturnedNotFoundException(self._response_error(resp))
             if resp.status_code == 429:
-                raise TooManyRequestsException("429 Too Many Requests")
+                raise TooManyRequestsException(self._response_error(resp))
             if resp.status_code != 200:
-                raise ConnectionException("HTTP error code {}.".format(resp.status_code))
-            is_html_query = not is_graphql_query and not "__a" in params and host == "www.instagram.com"
-            if is_html_query:
-                match = re.search(r'window\._sharedData = (.*);</script>', resp.text)
-                if match is None:
-                    raise ConnectionException("Could not find \"window._sharedData\" in html response.")
-                return json.loads(match.group(1))
+                raise ConnectionException(self._response_error(resp))
             else:
                 resp_json = resp.json()
             if 'status' in resp_json and resp_json['status'] != "ok":
-                if 'message' in resp_json:
-                    raise ConnectionException("Returned \"{}\" status, message \"{}\".".format(resp_json['status'],
-                                                                                               resp_json['message']))
-                else:
-                    raise ConnectionException("Returned \"{}\" status.".format(resp_json['status']))
+                raise ConnectionException(self._response_error(resp))
             return resp_json
         except (ConnectionException, json.decoder.JSONDecodeError, requests.exceptions.RequestException) as err:
             error_string = "JSON Query to {}: {}".format(path, err)
             if _attempt == self.max_connection_attempts:
-                raise ConnectionException(error_string) from err
+                if isinstance(err, QueryReturnedNotFoundException):
+                    raise QueryReturnedNotFoundException(error_string) from err
+                else:
+                    raise ConnectionException(error_string) from err
             self.error(error_string + " [retrying; skip with ^C]", repeat_at_end=False)
-            text_for_429 = ("HTTP error code 429 was returned because too many queries occured in the last time. "
-                            "Please do not use Instagram in your browser or run multiple instances of Instaloader "
-                            "in parallel.")
             try:
                 if isinstance(err, TooManyRequestsException):
-                    print(textwrap.fill(text_for_429), file=sys.stderr)
                     if is_graphql_query:
-                        waittime = graphql_query_waittime(untracked_queries=True)
-                        if waittime > 0:
-                            self.log('The request will be retried in {} seconds, at {:%H:%M}.'
-                                     .format(waittime, datetime.now() + timedelta(seconds=waittime)))
-                            time.sleep(waittime)
-                self._sleep()
-                return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1)
+                        self._rate_controller.handle_429(params['query_hash'])
+                    if is_iphone_query:
+                        self._rate_controller.handle_429('iphone')
+                    if is_other_query:
+                        self._rate_controller.handle_429('other')
+                return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1,
+                                     response_headers=response_headers)
             except KeyboardInterrupt:
                 self.error("[skipped by user]", repeat_at_end=False)
                 raise ConnectionException(error_string) from err
@@ -329,7 +483,7 @@ class InstaloaderContext:
         :param rhx_gis: 'rhx_gis' variable as somewhere returned by Instagram, needed to 'sign' request
         :return: The server's response dictionary.
         """
-        with copy_session(self._session) as tmpsession:
+        with copy_session(self._session, self.request_timeout) as tmpsession:
             tmpsession.headers.update(self._default_http_header(empty_session_only=True))
             del tmpsession.headers['Connection']
             del tmpsession.headers['Content-Length']
@@ -360,7 +514,12 @@ class InstaloaderContext:
                           edge_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
                           rhx_gis: Optional[str] = None,
                           first_data: Optional[Dict[str, Any]] = None) -> Iterator[Dict[str, Any]]:
-        """Retrieve a list of GraphQL nodes."""
+        """
+        Retrieve a list of GraphQL nodes.
+
+        .. deprecated:: 4.5
+           Use :class:`NodeIterator` instead, which provides more functionality.
+        """
 
         def _query():
             query_variables['first'] = self._graphql_page_length
@@ -386,39 +545,133 @@ class InstaloaderContext:
             data = _query()
             yield from (edge['node'] for edge in data['edges'])
 
-    def get_and_write_raw(self, url: str, filename: str, _attempt=1) -> None:
-        """Downloads raw data.
+    def get_iphone_json(self, path: str, params: Dict[str, Any]) -> Dict[str, Any]:
+        """JSON request to ``i.instagram.com``.
+
+        :param path: URL, relative to ``i.instagram.com/``
+        :param params: GET parameters
+        :return: Decoded response dictionary
+        :raises QueryReturnedBadRequestException: When the server responds with a 400.
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises ConnectionException: When query repeatedly failed.
+
+        .. versionadded:: 4.2.1"""
+        with copy_session(self._session, self.request_timeout) as tempsession:
+            # Set headers to simulate an API request from iPad
+            tempsession.headers['ig-intended-user-id'] = str(self.user_id)
+            tempsession.headers['x-pigeon-rawclienttime'] = '{:.6f}'.format(time.time())
+
+            # Add headers obtained from previous iPad request
+            tempsession.headers.update(self.iphone_headers)
+
+            # Extract key information from cookies if we haven't got it already from a previous request
+            header_cookies_mapping = {'x-mid': 'mid',
+                                     'ig-u-ds-user-id': 'ds_user_id',
+                                     'x-ig-device-id': 'ig_did',
+                                     'x-ig-family-device-id': 'ig_did',
+                                     'family_device_id': 'ig_did'}
+
+            # Map the cookie value to the matching HTTP request header
+            cookies = tempsession.cookies.get_dict().copy()
+            for key, value in header_cookies_mapping.items():
+                if value in cookies:
+                    if key not in tempsession.headers:
+                        tempsession.headers[key] = cookies[value]
+                    else:
+                        # Remove the cookie value if it's already specified as a header
+                        tempsession.cookies.pop(value, None)
+
+            # Edge case for ig-u-rur header due to special string encoding in cookie
+            if 'rur' in cookies:
+                if 'ig-u-rur' not in tempsession.headers:
+                    tempsession.headers['ig-u-rur'] = cookies['rur'].strip('\"').encode('utf-8') \
+                                                                    .decode('unicode_escape')
+                else:
+                    tempsession.cookies.pop('rur', None)
+
+            # Remove headers specific to Desktop version
+            for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With', 'Referer']:
+                tempsession.headers.pop(header, None)
+
+            # No need for cookies if we have a bearer token
+            if 'authorization' in tempsession.headers:
+                tempsession.cookies.clear()
+
+            response_headers = dict()    # type: Dict[str, Any]
+            response = self.get_json(path, params, 'i.instagram.com', tempsession, response_headers=response_headers)
+
+            # Extract the ig-set-* headers and use them in the next request
+            for key, value in response_headers.items():
+                if key.startswith('ig-set-'):
+                    self.iphone_headers[key.replace('ig-set-', '')] = value
+                elif key.startswith('x-ig-set-'):
+                    self.iphone_headers[key.replace('x-ig-set-', 'x-ig-')] = value
+
+            return response
+
+    def write_raw(self, resp: Union[bytes, requests.Response], filename: str) -> None:
+        """Write raw response data into a file.
+
+        .. versionadded:: 4.2.1"""
+        self.log(filename, end=' ', flush=True)
+        with open(filename + '.temp', 'wb') as file:
+            if isinstance(resp, requests.Response):
+                shutil.copyfileobj(resp.raw, file)
+            else:
+                file.write(resp)
+        os.replace(filename + '.temp', filename)
+
+    def get_raw(self, url: str, _attempt=1) -> requests.Response:
+        """Downloads a file anonymously.
+
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises QueryReturnedForbiddenException: When the server responds with a 403.
+        :raises ConnectionException: When download failed.
+
+        .. versionadded:: 4.2.1"""
+        with self.get_anonymous_session() as anonymous_session:
+            resp = anonymous_session.get(url, stream=True)
+        if resp.status_code == 200:
+            resp.raw.decode_content = True
+            return resp
+        else:
+            if resp.status_code == 403:
+                # suspected invalid URL signature
+                raise QueryReturnedForbiddenException(self._response_error(resp))
+            if resp.status_code == 404:
+                # 404 not worth retrying.
+                raise QueryReturnedNotFoundException(self._response_error(resp))
+            raise ConnectionException(self._response_error(resp))
+
+    def get_and_write_raw(self, url: str, filename: str) -> None:
+        """Downloads and writes anonymously-requested raw data into a file.
 
         :raises QueryReturnedNotFoundException: When the server responds with a 404.
         :raises QueryReturnedForbiddenException: When the server responds with a 403.
         :raises ConnectionException: When download repeatedly failed."""
-        try:
-            with self.get_anonymous_session() as anonymous_session:
-                resp = anonymous_session.get(url, stream=True)
-            if resp.status_code == 200:
-                self.log(filename, end=' ', flush=True)
-                with open(filename, 'wb') as file:
-                    resp.raw.decode_content = True
-                    shutil.copyfileobj(resp.raw, file)
-            else:
-                if resp.status_code == 403:
-                    # suspected invalid URL signature
-                    raise QueryReturnedForbiddenException("403 when accessing {}.".format(url))
-                if resp.status_code == 404:
-                    # 404 not worth retrying.
-                    raise QueryReturnedNotFoundException("404 when accessing {}.".format(url))
-                raise ConnectionException("HTTP error code {}.".format(resp.status_code))
-        except (urllib3.exceptions.HTTPError, requests.exceptions.RequestException, ConnectionException) as err:
-            error_string = "URL {}: {}".format(url, err)
-            if _attempt == self.max_connection_attempts:
-                raise ConnectionException(error_string) from err
-            self.error(error_string + " [retrying; skip with ^C]", repeat_at_end=False)
-            try:
-                self._sleep()
-                self.get_and_write_raw(url, filename, _attempt + 1)
-            except KeyboardInterrupt:
-                self.error("[skipped by user]", repeat_at_end=False)
-                raise ConnectionException(error_string) from err
+        self.write_raw(self.get_raw(url), filename)
+
+    def head(self, url: str, allow_redirects: bool = False) -> requests.Response:
+        """HEAD a URL anonymously.
+
+        :raises QueryReturnedNotFoundException: When the server responds with a 404.
+        :raises QueryReturnedForbiddenException: When the server responds with a 403.
+        :raises ConnectionException: When request failed.
+
+        .. versionadded:: 4.7.6
+        """
+        with self.get_anonymous_session() as anonymous_session:
+            resp = anonymous_session.head(url, allow_redirects=allow_redirects)
+        if resp.status_code == 200:
+            return resp
+        else:
+            if resp.status_code == 403:
+                # suspected invalid URL signature
+                raise QueryReturnedForbiddenException(self._response_error(resp))
+            if resp.status_code == 404:
+                # 404 not worth retrying.
+                raise QueryReturnedNotFoundException(self._response_error(resp))
+            raise ConnectionException(self._response_error(resp))
 
     @property
     def root_rhx_gis(self) -> Optional[str]:
@@ -427,6 +680,164 @@ class InstaloaderContext:
             # At the moment, rhx_gis seems to be required for anonymous requests only. By returning None when logged
             # in, we can save the root_rhx_gis lookup query.
             return None
-        if not self._root_rhx_gis:
-            self._root_rhx_gis = self.get_json('', {})['rhx_gis']
-        return self._root_rhx_gis
+        if self._root_rhx_gis is None:
+            self._root_rhx_gis = self.get_json('', {}).get('rhx_gis', '')
+        return self._root_rhx_gis or None
+
+
+class RateController:
+    """
+    Class providing request tracking and rate controlling to stay within rate limits.
+
+    It can be overridden to change Instaloader's behavior regarding rate limits, for example to raise a custom
+    exception when the rate limit is hit::
+
+       import instaloader
+
+       class MyRateController(instaloader.RateController):
+           def sleep(self, secs):
+               raise MyCustomException()
+
+       L = instaloader.Instaloader(rate_controller=lambda ctx: MyRateController(ctx))
+    """
+
+    def __init__(self, context: InstaloaderContext):
+        self._context = context
+        self._query_timestamps: Dict[str, List[float]] = dict()
+        self._earliest_next_request_time = 0.0
+        self._iphone_earliest_next_request_time = 0.0
+
+    def sleep(self, secs: float):
+        """Wait given number of seconds."""
+        # Not static, to allow for the behavior of this method to depend on context-inherent properties, such as
+        # whether we are logged in.
+        time.sleep(secs)
+
+    def _dump_query_timestamps(self, current_time: float, failed_query_type: str):
+        windows = [10, 11, 20, 22, 30, 60]
+        self._context.error("Number of requests within last {} minutes grouped by type:"
+                            .format('/'.join(str(w) for w in windows)),
+                            repeat_at_end=False)
+        for query_type, times in self._query_timestamps.items():
+            reqs_in_sliding_window = [sum(t > current_time - w * 60 for t in times) for w in windows]
+            self._context.error(" {} {:>32}: {}".format(
+                "*" if query_type == failed_query_type else " ",
+                query_type,
+                " ".join("{:4}".format(reqs) for reqs in reqs_in_sliding_window)
+            ), repeat_at_end=False)
+
+    def count_per_sliding_window(self, query_type: str) -> int:
+        """Return how many requests of the given type can be done within a sliding window of 11 minutes.
+
+        This is called by :meth:`RateController.query_waittime` and allows to simply customize wait times before queries
+        at query_type granularity. Consider overriding :meth:`RateController.query_waittime` directly if you need more
+        control."""
+        # Not static, to allow for the count_per_sliding_window to depend on context-inherent properties, such as
+        # whether we are logged in.
+        return 75 if query_type == 'other' else 200
+
+    def _reqs_in_sliding_window(self, query_type: Optional[str], current_time: float, window: float) -> List[float]:
+        if query_type is not None:
+            # timestamps of type query_type
+            relevant_timestamps = self._query_timestamps[query_type]
+        else:
+            # all GraphQL queries, i.e. not 'iphone' or 'other'
+            graphql_query_timestamps = filter(lambda tp: tp[0] not in ['iphone', 'other'],
+                                              self._query_timestamps.items())
+            relevant_timestamps = [t for times in (tp[1] for tp in graphql_query_timestamps) for t in times]
+        return list(filter(lambda t: t > current_time - window, relevant_timestamps))
+
+    def query_waittime(self, query_type: str, current_time: float, untracked_queries: bool = False) -> float:
+        """Calculate time needed to wait before query can be executed."""
+        per_type_sliding_window = 660
+        iphone_sliding_window = 1800
+        if query_type not in self._query_timestamps:
+            self._query_timestamps[query_type] = []
+        self._query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60,
+                                                         self._query_timestamps[query_type]))
+
+        def per_type_next_request_time():
+            reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, per_type_sliding_window)
+            if len(reqs_in_sliding_window) < self.count_per_sliding_window(query_type):
+                return 0.0
+            else:
+                return min(reqs_in_sliding_window) + per_type_sliding_window + 6
+
+        def gql_accumulated_next_request_time():
+            if query_type in ['iphone', 'other']:
+                return 0.0
+            gql_accumulated_sliding_window = 600
+            gql_accumulated_max_count = 275
+            reqs_in_sliding_window = self._reqs_in_sliding_window(None, current_time, gql_accumulated_sliding_window)
+            if len(reqs_in_sliding_window) < gql_accumulated_max_count:
+                return 0.0
+            else:
+                return min(reqs_in_sliding_window) + gql_accumulated_sliding_window
+
+        def untracked_next_request_time():
+            if untracked_queries:
+                if query_type == "iphone":
+                    reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
+                                                                          iphone_sliding_window)
+                    self._iphone_earliest_next_request_time = min(reqs_in_sliding_window) + iphone_sliding_window + 18
+                else:
+                    reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
+                                                                          per_type_sliding_window)
+                    self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6
+            return max(self._iphone_earliest_next_request_time, self._earliest_next_request_time)
+
+        def iphone_next_request():
+            if query_type == "iphone":
+                reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, iphone_sliding_window)
+                if len(reqs_in_sliding_window) >= 199:
+                    return min(reqs_in_sliding_window) + iphone_sliding_window + 18
+            return 0.0
+
+        return max(0.0,
+                   max(
+                       per_type_next_request_time(),
+                       gql_accumulated_next_request_time(),
+                       untracked_next_request_time(),
+                       iphone_next_request(),
+                   ) - current_time)
+
+    def wait_before_query(self, query_type: str) -> None:
+        """This method is called before a query to Instagram.
+
+        It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
+        :meth:`RateController.sleep` to wait until the request can be made."""
+        waittime = self.query_waittime(query_type, time.monotonic(), False)
+        assert waittime >= 0
+        if waittime > 15:
+            formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
+                                  "{} minutes".format(round(waittime / 60)))
+            self._context.log("\nToo many queries in the last time. Need to wait {}, until {:%H:%M}."
+                              .format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)))
+        if waittime > 0:
+            self.sleep(waittime)
+        if query_type not in self._query_timestamps:
+            self._query_timestamps[query_type] = [time.monotonic()]
+        else:
+            self._query_timestamps[query_type].append(time.monotonic())
+
+    def handle_429(self, query_type: str) -> None:
+        """This method is called to handle a 429 Too Many Requests response.
+
+        It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
+        :meth:`RateController.sleep` to wait until we can repeat the same request."""
+        current_time = time.monotonic()
+        waittime = self.query_waittime(query_type, current_time, True)
+        assert waittime >= 0
+        self._dump_query_timestamps(current_time, query_type)
+        text_for_429 = ("Instagram responded with HTTP error \"429 - Too Many Requests\". Please do not run multiple "
+                        "instances of Instaloader in parallel or within short sequence. Also, do not use any Instagram "
+                        "App while Instaloader is running.")
+        self._context.error(textwrap.fill(text_for_429), repeat_at_end=False)
+        if waittime > 1.5:
+            formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
+                                  "{} minutes".format(round(waittime / 60)))
+            self._context.error("The request will be retried in {}, at {:%H:%M}."
+                                .format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)),
+                                repeat_at_end=False)
+        if waittime > 0:
+            self.sleep(waittime)

instaloader/lateststamps.py

@@ -0,0 +1,117 @@
+import configparser
+from datetime import datetime, timezone
+from typing import Optional
+from os.path import dirname
+from os import makedirs
+
+
+class LatestStamps:
+    """LatestStamps class.
+
+    Convenience class for retrieving and storing data from the :option:`--latest-stamps` file.
+
+    :param latest_stamps_file: path to file.
+
+    .. versionadded:: 4.8"""
+    PROFILE_ID = 'profile-id'
+    PROFILE_PIC = 'profile-pic'
+    POST_TIMESTAMP = 'post-timestamp'
+    TAGGED_TIMESTAMP = 'tagged-timestamp'
+    IGTV_TIMESTAMP = 'igtv-timestamp'
+    STORY_TIMESTAMP = 'story-timestamp'
+    ISO_FORMAT = '%Y-%m-%dT%H:%M:%S.%f%z'
+
+    def __init__(self, latest_stamps_file):
+        self.file = latest_stamps_file
+        self.data = configparser.ConfigParser()
+        self.data.read(latest_stamps_file)
+
+    def _save(self):
+        if dn := dirname(self.file):
+            makedirs(dn, exist_ok=True)
+        with open(self.file, 'w') as f:
+            self.data.write(f)
+
+    def _ensure_section(self, section: str):
+        if not self.data.has_section(section):
+            self.data.add_section(section)
+
+    def get_profile_id(self, profile_name: str) -> Optional[int]:
+        """Returns stored ID of profile."""
+        try:
+            return self.data.getint(profile_name, self.PROFILE_ID)
+        except (configparser.Error, ValueError):
+            return None
+
+    def save_profile_id(self, profile_name: str, profile_id: int):
+        """Stores ID of profile."""
+        self._ensure_section(profile_name)
+        self.data.set(profile_name, self.PROFILE_ID, str(profile_id))
+        self._save()
+
+    def rename_profile(self, old_profile: str, new_profile: str):
+        """Renames a profile."""
+        self._ensure_section(new_profile)
+        for option in [self.PROFILE_ID, self.PROFILE_PIC, self.POST_TIMESTAMP,
+                       self.TAGGED_TIMESTAMP, self.IGTV_TIMESTAMP, self.STORY_TIMESTAMP]:
+            if self.data.has_option(old_profile, option):
+                value = self.data.get(old_profile, option)
+                self.data.set(new_profile, option, value)
+        self.data.remove_section(old_profile)
+        self._save()
+
+    def _get_timestamp(self, section: str, key: str) -> datetime:
+        try:
+            return datetime.strptime(self.data.get(section, key), self.ISO_FORMAT)
+        except (configparser.Error, ValueError):
+            return datetime.fromtimestamp(0, timezone.utc)
+
+    def _set_timestamp(self, section: str, key: str, timestamp: datetime):
+        self._ensure_section(section)
+        self.data.set(section, key, timestamp.strftime(self.ISO_FORMAT))
+        self._save()
+
+    def get_last_post_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's posts."""
+        return self._get_timestamp(profile_name, self.POST_TIMESTAMP)
+
+    def set_last_post_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's posts."""
+        self._set_timestamp(profile_name, self.POST_TIMESTAMP, timestamp)
+
+    def get_last_tagged_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's tagged posts."""
+        return self._get_timestamp(profile_name, self.TAGGED_TIMESTAMP)
+
+    def set_last_tagged_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's tagged posts."""
+        self._set_timestamp(profile_name, self.TAGGED_TIMESTAMP, timestamp)
+
+    def get_last_igtv_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's igtv posts."""
+        return self._get_timestamp(profile_name, self.IGTV_TIMESTAMP)
+
+    def set_last_igtv_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's igtv posts."""
+        self._set_timestamp(profile_name, self.IGTV_TIMESTAMP, timestamp)
+
+    def get_last_story_timestamp(self, profile_name: str) -> datetime:
+        """Returns timestamp of last download of a profile's stories."""
+        return self._get_timestamp(profile_name, self.STORY_TIMESTAMP)
+
+    def set_last_story_timestamp(self, profile_name: str, timestamp: datetime):
+        """Sets timestamp of last download of a profile's stories."""
+        self._set_timestamp(profile_name, self.STORY_TIMESTAMP, timestamp)
+
+    def get_profile_pic(self, profile_name: str) -> str:
+        """Returns filename of profile's last downloaded profile pic."""
+        try:
+            return self.data.get(profile_name, self.PROFILE_PIC)
+        except configparser.Error:
+            return ""
+
+    def set_profile_pic(self, profile_name: str, profile_pic: str):
+        """Sets filename of profile's last downloaded profile pic."""
+        self._ensure_section(profile_name)
+        self.data.set(profile_name, self.PROFILE_PIC, profile_pic)
+        self._save()

instaloader/nodeiterator.py

@@ -0,0 +1,308 @@
+import base64
+import hashlib
+import json
+import os
+from contextlib import contextmanager
+from datetime import datetime, timedelta
+from lzma import LZMAError
+from typing import Any, Callable, Dict, Iterable, Iterator, NamedTuple, Optional, Tuple, TypeVar
+
+from .exceptions import AbortDownloadException, InvalidArgumentException, QueryReturnedBadRequestException
+from .instaloadercontext import InstaloaderContext
+
+class FrozenNodeIterator(NamedTuple):
+    query_hash: str
+    query_variables: Dict
+    query_referer: Optional[str]
+    context_username: Optional[str]
+    total_index: int
+    best_before: Optional[float]
+    remaining_data: Optional[Dict]
+    first_node: Optional[Dict]
+FrozenNodeIterator.query_hash.__doc__ = """The GraphQL ``query_hash`` parameter."""
+FrozenNodeIterator.query_variables.__doc__ = """The GraphQL ``query_variables`` parameter."""
+FrozenNodeIterator.query_referer.__doc__ = """The HTTP referer used for the GraphQL query."""
+FrozenNodeIterator.context_username.__doc__ = """The username who created the iterator, or ``None``."""
+FrozenNodeIterator.total_index.__doc__ = """Number of items that have already been returned."""
+FrozenNodeIterator.best_before.__doc__ = """Date when parts of the stored nodes might have expired."""
+FrozenNodeIterator.remaining_data.__doc__ = \
+    """The already-retrieved, yet-unprocessed ``edges`` and the ``page_info`` at time of freezing."""
+FrozenNodeIterator.first_node.__doc__ = """Node data of the first item, if an item has been produced."""
+
+T = TypeVar('T')
+
+
+class NodeIterator(Iterator[T]):
+    """
+    Iterate the nodes within edges in a GraphQL pagination. Instances of this class are returned by many (but not all)
+    of Instaloader's :class:`Post`-returning functions (such as :meth:`Profile.get_posts` etc.).
+
+    What makes this iterator special is its ability to freeze/store its current state, e.g. to interrupt an iteration,
+    and later thaw/resume from where it left off.
+
+    You can freeze a NodeIterator with :meth:`NodeIterator.freeze`::
+
+       post_iterator = profile.get_posts()
+       try:
+           for post in post_iterator:
+               do_something_with(post)
+       except KeyboardInterrupt:
+           save("resume_information.json", post_iterator.freeze())
+
+    and later reuse it with :meth:`NodeIterator.thaw` on an equally-constructed NodeIterator::
+
+       post_iterator = profile.get_posts()
+       post_iterator.thaw(load("resume_information.json"))
+
+    (an appropriate method to load and save the :class:`FrozenNodeIterator` is e.g.
+    :func:`load_structure_from_file` and :func:`save_structure_to_file`.)
+
+    A :class:`FrozenNodeIterator` can only be thawn with a matching NodeIterator, i.e. a NodeIterator instance that has
+    been constructed with the same parameters as the instance that is represented by the :class:`FrozenNodeIterator` in
+    question. This is to ensure that an iteration cannot be resumed in a wrong, unmatching loop. As a quick way to
+    distinguish iterators that are saved e.g. in files, there is the :attr:`NodeIterator.magic` string: Two
+    NodeIterators are matching if and only if they have the same magic.
+
+    See also :func:`resumable_iteration` for a high-level context manager that handles a resumable iteration.
+    """
+
+    _graphql_page_length = 50
+    _shelf_life = timedelta(days=29)
+
+    def __init__(self,
+                 context: InstaloaderContext,
+                 query_hash: str,
+                 edge_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
+                 node_wrapper: Callable[[Dict], T],
+                 query_variables: Optional[Dict[str, Any]] = None,
+                 query_referer: Optional[str] = None,
+                 first_data: Optional[Dict[str, Any]] = None,
+                 is_first: Optional[Callable[[T, Optional[T]], bool]] = None):
+        self._context = context
+        self._query_hash = query_hash
+        self._edge_extractor = edge_extractor
+        self._node_wrapper = node_wrapper
+        self._query_variables = query_variables if query_variables is not None else {}
+        self._query_referer = query_referer
+        self._page_index = 0
+        self._total_index = 0
+        if first_data is not None:
+            self._data = first_data
+            self._best_before = datetime.now() + NodeIterator._shelf_life
+        else:
+            self._data = self._query()
+        self._first_node: Optional[Dict] = None
+        self._is_first = is_first
+
+    def _query(self, after: Optional[str] = None) -> Dict:
+        pagination_variables: Dict[str, Any] = {'first': NodeIterator._graphql_page_length}
+        if after is not None:
+            pagination_variables['after'] = after
+        try:
+            data = self._edge_extractor(
+                self._context.graphql_query(
+                    self._query_hash, {**self._query_variables, **pagination_variables}, self._query_referer
+                )
+            )
+            self._best_before = datetime.now() + NodeIterator._shelf_life
+            return data
+        except QueryReturnedBadRequestException:
+            new_page_length = int(NodeIterator._graphql_page_length / 2)
+            if new_page_length >= 12:
+                NodeIterator._graphql_page_length = new_page_length
+                self._context.error("HTTP Error 400 (Bad Request) on GraphQL Query. Retrying with shorter page length.",
+                                    repeat_at_end=False)
+                return self._query(after)
+            else:
+                raise
+
+    def __iter__(self):
+        return self
+
+    def __next__(self) -> T:
+        if self._page_index < len(self._data['edges']):
+            node = self._data['edges'][self._page_index]['node']
+            page_index, total_index = self._page_index, self._total_index
+            try:
+                self._page_index += 1
+                self._total_index += 1
+            except KeyboardInterrupt:
+                self._page_index, self._total_index = page_index, total_index
+                raise
+            item = self._node_wrapper(node)
+            if self._is_first is not None:
+                if self._is_first(item, self.first_item):
+                    self._first_node = node
+            else:
+                if self._first_node is None:
+                    self._first_node = node
+            return item
+        if self._data.get('page_info', {}).get('has_next_page'):
+            query_response = self._query(self._data['page_info']['end_cursor'])
+            if self._data['edges'] != query_response['edges'] and len(query_response['edges']) > 0:
+                page_index, data = self._page_index, self._data
+                try:
+                    self._page_index = 0
+                    self._data = query_response
+                except KeyboardInterrupt:
+                    self._page_index, self._data = page_index, data
+                    raise
+                return self.__next__()
+        raise StopIteration()
+
+    @property
+    def count(self) -> Optional[int]:
+        """The ``count`` as returned by Instagram. This is not always the total count this iterator will yield."""
+        return self._data.get('count') if self._data is not None else None
+
+    @property
+    def total_index(self) -> int:
+        """Number of items that have already been returned."""
+        return self._total_index
+
+    @property
+    def magic(self) -> str:
+        """Magic string for easily identifying a matching iterator file for resuming (hash of some parameters)."""
+        magic_hash = hashlib.blake2b(digest_size=6)
+        magic_hash.update(json.dumps(
+            [self._query_hash, self._query_variables, self._query_referer, self._context.username]
+        ).encode())
+        return base64.urlsafe_b64encode(magic_hash.digest()).decode()
+
+    @property
+    def first_item(self) -> Optional[T]:
+        """
+        If this iterator has produced any items, returns the first item produced.
+
+        It is possible to override what is considered the first item (for example, to consider the
+        newest item in case items are not in strict chronological order) by passing a callback
+        function as the `is_first` parameter when creating the class.
+
+        .. versionadded:: 4.8
+        .. versionchanged:: 4.9.2
+           What is considered the first item can be overridden.
+        """
+        return self._node_wrapper(self._first_node) if self._first_node is not None else None
+
+    @staticmethod
+    def page_length() -> int:
+        return NodeIterator._graphql_page_length
+
+    def freeze(self) -> FrozenNodeIterator:
+        """Freeze the iterator for later resuming."""
+        remaining_data = None
+        if self._data is not None:
+            remaining_data = {**self._data,
+                              'edges': (self._data['edges'][(max(self._page_index - 1, 0)):])}
+        return FrozenNodeIterator(
+            query_hash=self._query_hash,
+            query_variables=self._query_variables,
+            query_referer=self._query_referer,
+            context_username=self._context.username,
+            total_index=max(self.total_index - 1, 0),
+            best_before=self._best_before.timestamp() if self._best_before else None,
+            remaining_data=remaining_data,
+            first_node=self._first_node,
+        )
+
+    def thaw(self, frozen: FrozenNodeIterator) -> None:
+        """
+        Use this iterator for resuming from earlier iteration.
+
+        :raises InvalidArgumentException:
+           If
+
+           - the iterator on which this method is called has already been used, or
+           - the given :class:`FrozenNodeIterator` does not match, i.e. belongs to a different iteration.
+        """
+        if self._total_index or self._page_index:
+            raise InvalidArgumentException("thaw() called on already-used iterator.")
+        if (self._query_hash != frozen.query_hash or
+                self._query_variables != frozen.query_variables or
+                self._query_referer != frozen.query_referer or
+                self._context.username != frozen.context_username):
+            raise InvalidArgumentException("Mismatching resume information.")
+        if not frozen.best_before:
+            raise InvalidArgumentException("\"best before\" date missing.")
+        if frozen.remaining_data is None:
+            raise InvalidArgumentException("\"remaining_data\" missing.")
+        self._total_index = frozen.total_index
+        self._best_before = datetime.fromtimestamp(frozen.best_before)
+        self._data = frozen.remaining_data
+        if frozen.first_node is not None:
+            self._first_node = frozen.first_node
+
+
+@contextmanager
+def resumable_iteration(context: InstaloaderContext,
+                        iterator: Iterable,
+                        load: Callable[[InstaloaderContext, str], Any],
+                        save: Callable[[FrozenNodeIterator, str], None],
+                        format_path: Callable[[str], str],
+                        check_bbd: bool = True,
+                        enabled: bool = True) -> Iterator[Tuple[bool, int]]:
+    """
+    High-level context manager to handle a resumable iteration that can be interrupted
+    with a :class:`KeyboardInterrupt` or an :class:`AbortDownloadException`.
+
+    It can be used as follows to automatically load a previously-saved state into the iterator, save the iterator's
+    state when interrupted, and delete the resume file upon completion::
+
+       post_iterator = profile.get_posts()
+       with resumable_iteration(
+               context=L.context,
+               iterator=post_iterator,
+               load=lambda _, path: FrozenNodeIterator(**json.load(open(path))),
+               save=lambda fni, path: json.dump(fni._asdict(), open(path, 'w')),
+               format_path=lambda magic: "resume_info_{}.json".format(magic)
+       ) as (is_resuming, start_index):
+           for post in post_iterator:
+               do_something_with(post)
+
+    It yields a tuple (is_resuming, start_index).
+
+    When the passed iterator is not a :class:`NodeIterator`, it behaves as if ``resumable_iteration`` was not used,
+    just executing the inner body.
+
+    :param context: The :class:`InstaloaderContext`.
+    :param iterator: The fresh :class:`NodeIterator`.
+    :param load: Loads a FrozenNodeIterator from given path. The object is ignored if it has a different type.
+    :param save: Saves the given FrozenNodeIterator to the given path.
+    :param format_path: Returns the path to the resume file for the given magic.
+    :param check_bbd: Whether to check the best before date and reject an expired FrozenNodeIterator.
+    :param enabled: Set to False to disable all functionality and simply execute the inner body.
+
+    .. versionchanged:: 4.7
+       Also interrupt on :class:`AbortDownloadException`.
+    """
+    if not enabled or not isinstance(iterator, NodeIterator):
+        yield False, 0
+        return
+    is_resuming = False
+    start_index = 0
+    resume_file_path = format_path(iterator.magic)
+    resume_file_exists = os.path.isfile(resume_file_path)
+    if resume_file_exists:
+        try:
+            fni = load(context, resume_file_path)
+            if not isinstance(fni, FrozenNodeIterator):
+                raise InvalidArgumentException("Invalid type.")
+            if check_bbd and fni.best_before and datetime.fromtimestamp(fni.best_before) < datetime.now():
+                raise InvalidArgumentException("\"Best before\" date exceeded.")
+            iterator.thaw(fni)
+            is_resuming = True
+            start_index = iterator.total_index
+            context.log("Resuming from {}.".format(resume_file_path))
+        except (InvalidArgumentException, LZMAError, json.decoder.JSONDecodeError, EOFError) as exc:
+            context.error("Warning: Not resuming from {}: {}".format(resume_file_path, exc))
+    try:
+        yield is_resuming, start_index
+    except (KeyboardInterrupt, AbortDownloadException):
+        if os.path.dirname(resume_file_path):
+            os.makedirs(os.path.dirname(resume_file_path), exist_ok=True)
+        save(iterator.freeze(), resume_file_path)
+        context.log("\nSaved resume information to {}.".format(resume_file_path))
+        raise
+    if resume_file_exists:
+        os.unlink(resume_file_path)
+        context.log("Iteration complete, deleted resume information file {}.".format(resume_file_path))

instaloader/sectioniterator.py

@@ -0,0 +1,46 @@
+from typing import Any, Callable, Dict, Iterator, Optional, TypeVar
+
+from .instaloadercontext import InstaloaderContext
+
+T = TypeVar('T')
+
+
+class SectionIterator(Iterator[T]):
+    """Iterator for the new 'sections'-style responses.
+
+    .. versionadded:: 4.9"""
+    def __init__(self,
+                 context: InstaloaderContext,
+                 sections_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
+                 media_wrapper: Callable[[Dict], T],
+                 query_path: str,
+                 first_data: Optional[Dict[str, Any]] = None):
+        self._context = context
+        self._sections_extractor = sections_extractor
+        self._media_wrapper = media_wrapper
+        self._query_path = query_path
+        self._data = first_data or self._query()
+        self._page_index = 0
+        self._section_index = 0
+
+    def __iter__(self):
+        return self
+
+    def _query(self, max_id: Optional[str] = None) -> Dict[str, Any]:
+        pagination_variables = {"max_id": max_id} if max_id is not None else {}
+        return self._sections_extractor(
+            self._context.get_json(self._query_path, params={"__a": 1, "__d": "dis", **pagination_variables})
+        )
+
+    def __next__(self) -> T:
+        if self._page_index < len(self._data['sections']):
+            media = self._data['sections'][self._page_index]['layout_content']['medias'][self._section_index]['media']
+            self._section_index += 1
+            if self._section_index >= len(self._data['sections'][self._page_index]['layout_content']['medias']):
+                self._section_index = 0
+                self._page_index += 1
+            return self._media_wrapper(media)
+        if self._data['more_available']:
+            self._page_index, self._section_index, self._data = 0, 0, self._query(self._data["next_max_id"])
+            return self.__next__()
+        raise StopIteration()

setup.py

@@ -3,7 +3,6 @@
 import re
 import sys
 import os
-import platform
 from setuptools import setup
 
 
@@ -19,13 +18,13 @@ def get_version():
     raise SystemExit("Could not find version string.")
 
 
-if sys.version_info < (3, 5):
-    sys.exit('Instaloader requires Python >= 3.5.')
+if sys.version_info < (3, 8):
+    sys.exit('Instaloader requires Python >= 3.8.')
 
 requirements = ['requests>=2.4']
-
-if platform.system() == 'Windows' and sys.version_info < (3, 6):
-    requirements.append('win_unicode_console')
+optional_requirements = {
+    'browser_cookie3': ['browser_cookie3>=0.19.1'],
+}
 
 keywords = (['instagram', 'instagram-scraper', 'instagram-client', 'instagram-feed', 'downloader', 'videos', 'photos',
              'pictures', 'instagram-user-photos', 'instagram-photos', 'instagram-metadata', 'instagram-downloader',
@@ -38,6 +37,7 @@ setup(
     name='instaloader',
     version=get_version(),
     packages=['instaloader'],
+    package_data={'instaloader': ['py.typed']},
     url='https://instaloader.github.io/',
     license='MIT',
     author='Alexander Graf, André Koch-Kramer',
@@ -46,9 +46,10 @@ setup(
                 'from Instagram.',
     long_description=open(os.path.join(SRC, 'README.rst')).read(),
     install_requires=requirements,
-    python_requires='>=3.5',
+    python_requires='>=3.8',
+    extras_require=optional_requirements,
     entry_points={'console_scripts': ['instaloader=instaloader.__main__:main']},
-    zip_safe=True,
+    zip_safe=False,
     keywords=keywords,
     classifiers=[
         'Development Status :: 5 - Production/Stable',
@@ -57,9 +58,11 @@ setup(
         'Intended Audience :: Developers',
         'License :: OSI Approved :: MIT License',
         'Operating System :: OS Independent',
-        'Programming Language :: Python :: 3.5',
-        'Programming Language :: Python :: 3.6',
-        'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
+        'Programming Language :: Python :: 3.9',
+        'Programming Language :: Python :: 3.10',
+        'Programming Language :: Python :: 3.11',
+        'Programming Language :: Python :: 3.12',
         'Programming Language :: Python :: 3 :: Only',
         'Topic :: Internet',
         'Topic :: Multimedia :: Graphics'

test/instaloader_unittests.py

@@ -5,20 +5,25 @@ import shutil
 import tempfile
 import unittest
 from itertools import islice
+from typing import Optional
 
 import instaloader
 
 PROFILE_WITH_HIGHLIGHTS = 325732271
 PUBLIC_PROFILE = "selenagomez"
 PUBLIC_PROFILE_ID = 460563723
+PUBLIC_PROFILE_WITH_IGTV = "natgeo"
 HASHTAG = "kitten"
+LOCATION = "362629379"
 OWN_USERNAME = "aandergr"
 NORMAL_MAX_COUNT = 2
 PAGING_MAX_COUNT = 15
 PRIVATE_PROFILE = "aandergr"
+PRIVATE_PROFILE_ID = 1706625676
+EMPTY_PROFILE = "not_public"
+EMPTY_PROFILE_ID = 1928659031
 
-# Preserve query timestamps (rate control) between tests to not get rate limited
-instaloadercontext_query_timestamps = list()
+ratecontroller: Optional[instaloader.RateController] = None
 
 
 class TestInstaloaderAnonymously(unittest.TestCase):
@@ -31,11 +36,15 @@ class TestInstaloaderAnonymously(unittest.TestCase):
                                          download_comments=True,
                                          save_metadata=True)
         self.L.context.raise_all_errors = True
-        self.L.context.query_timestamps = instaloadercontext_query_timestamps
+        if ratecontroller is not None:
+            # pylint:disable=protected-access
+            ratecontroller._context = self.L.context
+            self.L.context._rate_controller = ratecontroller
 
     def tearDown(self):
-        global instaloadercontext_query_timestamps
-        instaloadercontext_query_timestamps = self.L.context.query_timestamps
+        # pylint:disable=global-statement,protected-access
+        global ratecontroller
+        ratecontroller = self.L.context._rate_controller
         self.L.close()
         os.chdir('/')
         print("Removing {}".format(self.dir))
@@ -49,28 +58,47 @@ class TestInstaloaderAnonymously(unittest.TestCase):
                 self.assertTrue(post.date_utc < previous_post.date_utc)
             previous_post = post
 
-    @unittest.SkipTest
+    @unittest.skip('')
     def test_public_profile_download(self):
-        self.L.download_profile(PUBLIC_PROFILE, profile_pic=False, fast_update=True)
-        self.L.download_profile(PUBLIC_PROFILE, profile_pic=False, fast_update=True)
+        profiles = {self.L.check_profile_id(PUBLIC_PROFILE)}
+        self.L.download_profiles(profiles, profile_pic=False, fast_update=True, raise_errors=True)
+        self.L.download_profiles(profiles, profile_pic=False, fast_update=True, raise_errors=True)
 
     def test_public_profile_paging(self):
         self.post_paging_test(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts())
 
     def test_profile_pic_download(self):
-        self.L.download_profile(PUBLIC_PROFILE, profile_pic_only=True)
+        self.L.download_profiles({self.L.check_profile_id(PUBLIC_PROFILE)}, posts=False, raise_errors=True)
 
     def test_hashtag_download(self):
         self.L.download_hashtag(HASHTAG, NORMAL_MAX_COUNT)
 
+    def test_hashtag_paging(self):
+        for count, post in enumerate(instaloader.Hashtag.from_name(self.L.context, HASHTAG).get_all_posts()):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
     def test_get_id_by_username(self):
         self.assertEqual(PUBLIC_PROFILE_ID,
                          instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).userid)
 
-    def test_get_username_by_id(self):
+    def test_get_username_by_id_private(self):
+        self.assertEqual(PRIVATE_PROFILE.lower(),
+                         instaloader.Profile.from_id(self.L.context, PRIVATE_PROFILE_ID).username)
+
+    def test_get_username_by_id_public(self):
         self.assertEqual(PUBLIC_PROFILE.lower(),
                          instaloader.Profile.from_id(self.L.context, PUBLIC_PROFILE_ID).username)
 
+    def test_get_username_by_id_empty(self):
+        self.assertEqual(EMPTY_PROFILE.lower(),
+                         instaloader.Profile.from_id(self.L.context, EMPTY_PROFILE_ID).username)
+
+    def test_get_username_by_name_empty(self):
+        self.assertEqual(EMPTY_PROFILE_ID,
+                         instaloader.Profile.from_username(self.L.context, EMPTY_PROFILE).userid)
+
     def test_post_from_mediaid(self):
         for post in instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts():
             post2 = instaloader.Post.from_mediaid(self.L.context, post.mediaid)
@@ -82,6 +110,11 @@ class TestInstaloaderAnonymously(unittest.TestCase):
                            PAGING_MAX_COUNT):
             print(post)
 
+    def test_public_profile_igtv(self):
+        for post in islice(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE_WITH_IGTV).get_igtv_posts(),
+                           PAGING_MAX_COUNT):
+            print(post)
+
 
 class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
 
@@ -89,13 +122,13 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
         super().setUp()
         self.L.load_session_from_file(OWN_USERNAME)
 
-    @unittest.SkipTest
+    @unittest.skip('')
     def test_stories_download(self):
         self.L.download_stories()
 
-    @unittest.SkipTest
+    @unittest.skip('')
     def test_private_profile_download(self):
-        self.L.download_profile(PRIVATE_PROFILE, download_stories=True)
+        self.L.download_profiles({self.L.check_profile_id(PRIVATE_PROFILE)}, stories=True, raise_errors=True)
 
     def test_stories_paging(self):
         for user_story in self.L.get_stories():
@@ -137,8 +170,8 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
 
     def test_followees_and_stories(self):
         profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
-        for f in islice(profile.get_followees(), PAGING_MAX_COUNT):
-            self.L.download_profile(f.username, profile_pic=False, download_stories_only=True)
+        followees = set(islice(profile.get_followees(), PAGING_MAX_COUNT))
+        self.L.download_profiles(followees, profile_pic=False, stories=True, posts=False, raise_errors=True)
 
     def test_get_followees(self):
         profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
@@ -150,9 +183,10 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
         for f in profile.get_followers():
             print(f.username)
 
-    def test_get_username_by_id(self):
-        self.assertEqual(PUBLIC_PROFILE.lower(),
-                         instaloader.Profile.from_id(self.L.context, PUBLIC_PROFILE_ID).username)
+    def test_get_similar_accounts(self):
+        profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
+        for f in profile.get_similar_accounts():
+            print(f.username)
 
     def test_get_likes(self):
         for post in instaloader.Profile.from_username(self.L.context, OWN_USERNAME).get_posts():
@@ -166,6 +200,15 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
             if count == PAGING_MAX_COUNT:
                 break
 
+    def test_location_download(self):
+        self.L.download_location(LOCATION, NORMAL_MAX_COUNT)
+
+    def test_location_paging(self):
+        for count, post in enumerate(self.L.get_location_posts(LOCATION)):
+            print(post)
+            if count == PAGING_MAX_COUNT:
+                break
+
 
 if __name__ == '__main__':
     unittest.main()