Compare commits

...
43 Commits
Author SHA1 Message Date
Alexander Graf 52f258e89b Release of Version 4.7 2021-03-27 15:19:12 +01:00
Alexander Graf e9888e72fd Second Release Candidate for Version 4.7 2021-03-20 17:44:04 +01:00
Alexander Graf c72b2a8fc2 Default download_comments and download_geotags to False 2021-03-20 17:39:37 +01:00
Alexander Graf e2cec0a0bf Update dependencies; test on py3.9; drop python 3.5 support 2021-03-20 17:30:15 +01:00
Alexander Graf 08eaa7d474 always keep exception chains 2021-03-20 17:17:08 +01:00
Alexander Graf f70355e803 Update default user agent 2021-03-20 17:15:05 +01:00
Alexander Graf bba7d5131c First Release Candidate for Version 4.7 2021-03-13 11:24:53 +01:00
Alexander GrafandAndré Koch-Kramer f6731566cd Support resuming of downloading comments
Co-Authored-By: André Koch-Kramer <koch-kramer@web.de>
2021-03-13 11:17:56 +01:00
Alexander Graf cd13211603 Minor optimization in get_sidecar_nodes() 2021-02-24 16:35:08 +01:00
Alexander Graf 8635a576f0 Fix line-too-long pylint complaint 2021-02-24 16:25:56 +01:00
Alexander Graf 8f594a3a30 Merge branch 'master' into upcoming/v4.7 2021-02-24 16:10:27 +01:00
Alexander Graf 0f11ef1bdd Release of version 4.6.2 2021-02-24 16:01:33 +01:00
Alexander Graf a4bb353421 docs: Enhance description of --resume-prefix 2021-02-24 16:01:22 +01:00
Alexander Graf dfdd744804 Minor fixes to top-level README.md 2021-02-24 16:00:28 +01:00
Thomas 9fb112faa8 Fix issue 892 Save content using original filenames (PR 893) (#1025)
- When a post contained sidecars, the filename for caption, json etc. was changed. It was no longer the original filename like before.
- If the post contains sidecars, a local variable is used to build the filenames for the sidecar media.
2021-02-20 10:17:57 +01:00
Alexander Graf f85e49d521 Spare unnecessary full metadata fetch
This fetch was done for sidecar posts that contain a video when going
through a profile. The fetched information is already present with the
new profile query introduced in the last commit, making this full
metadata fetch query unnecessary. Instaloader now better evaluates if
that fetch must be done or not.

The fetch was also (eventually unnecessarily) made when accessing
get_sidecar_posts() on a Post that has been loaded with
load_structure_from_file().
2021-02-17 16:40:54 +01:00
Thomas 6c4da39372 Update the query_hash for post downloads (#1026)
- It's necessary because for the old query_hash IG isn't returning the full structure any more.
- So the old query_hash is replaced with the current one.
2021-02-17 16:07:18 +01:00
Alexander Graf a2d756b177 Flag --abort-on to abort on given status codes
Closes #920.
2021-02-13 19:23:14 +01:00
Alexander Graf e15d67c065 Change login hint from "Warning" to "Hint" 2021-02-13 19:12:13 +01:00
Alexander Graf 640066dd60 Merge branch 'master' into upcoming/v4.7 2021-01-30 17:07:21 +01:00
Alexander GrafandAndré Koch-Kramer 672b0b1840 Fix efficiency bug regarding re-downloading targets
Remove unnecessary queries when a file in a target does not have to be
re-downloaded.

This significantly speeds up the download in cases where the majority of media
is already present locally.

Co-Authored-By: André Koch-Kramer <koch-kramer@web.de>
2021-01-30 16:54:35 +01:00
Alexander Graf 828fa71ede Release of version 4.6.1 2021-01-25 16:04:30 +01:00
Alexander Graf 4fd57f5598 Adjust user agent string and sleep behavior 2021-01-25 15:53:49 +01:00
Alexander Graf c5b686a213 Update RateController to current observations 2021-01-25 15:46:05 +01:00
43c521afa9 Add method to get title of post (#976)
Co-authored-by: Tejas Chauhan <tejas.chauhan@kevit.io>
Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
2021-01-16 18:50:27 +01:00
André Koch-Kramer 3955540080 Release of version 4.6 2021-01-09 16:23:49 +01:00
André Koch-Kramer 3fe6b1e139 Merge remote-tracking branch 'origin/upcoming/v4.6' 2021-01-09 16:11:05 +01:00
Alexander Graf a2ce906fd0 Fix efficiency bug regarding {filename} in filename-pattern
Avoid evaluating 'url' when preparing the filename if the
filename-pattern does not contain the "{filename}" token. This avoids
the URL to be queried even though it is not needed (e.g. with
--no-pictures).
2021-01-04 17:37:19 +01:00
Alexander Graf 850cd39be4 First Release Candidate for Version 4.6 2021-01-02 18:42:49 +01:00
Alexander Graf 227efa7b68 Improvements to contributing.md, other minor changes 2021-01-02 17:41:19 +01:00
Alexander Graf d7f7cad4a2 docs: Link to examples in master branch 2020-12-23 19:39:18 +01:00
Alexander Graf 2f69ea62a1 First Alpha Release for Version 4.6 2020-12-14 22:16:22 +01:00
Alexander Graf 0f258b6118 Merge branch 'master' into upcoming/v4.6 2020-12-14 22:14:01 +01:00
AndyRandAlexander Graf b31f279527 Allow selecting range to download from an sidecar (#749)
Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
2020-12-14 22:11:33 +01:00
Alexander GrafandAndré Koch-Kramer ed55ce140d Fix efficiency bug in download_profiles()
Fixes efficiency of the download_profiles() function when called with
--no-posts --no-profile-pic by reordering an if statement.

This inefficiency has been reported in #724.

Co-authored-by: André Koch-Kramer <koch-kramer@web.de>
2020-12-12 16:49:38 +01:00
Alexander Graf dcc7ab89c1 615_import_firefox_session.py: Open database with immutable=1
Allows to open the firefox cookie database even though Firefox is
holding a lock.

Fixes the "Cookie import failed: database is locked" message when using
the script while Firefox was running.

This has been reported in #877.
2020-12-06 08:43:04 +01:00
Alexander Graf 610756d342 Improvements to docs/contributing.rst 2020-12-04 17:08:42 +01:00
Alexander Graf 097bf7fecc Merge branch 'master' into upcoming/v4.6 2020-12-04 14:10:18 +01:00
Thomas a045168c79 Save content using original filenames (#893)
Add feature from issue #892 (Save content using original filenames)

- The new feature would download posts and save the images/videos with the original filenames.
- The implementation follows the other options for the filename-pattern parameter. As filename is related to a single file and not the whole post, for sidecar nodes the filename is calculated again.
- Using this option media from one post is only grouped together by the file modification timestamp.
2020-12-02 11:22:55 +01:00
Alexander Graf 4c02a186d3 Merge branch 'master' into upcoming/v4.6 2020-11-28 19:00:49 +01:00
Alexander Graf e11b88d44b Advertise PEP 561 compliance 2020-11-19 10:32:46 +01:00
Alexander Graf 1c2e226630 Enable default request timeout of 300 seconds (#870)
Sets a default request timeout of 300 seconds to fix Instaloader hanging indefinitely when used on an unstable internet connection without using --request-timeout, such as in #810.
2020-11-14 18:47:37 +01:00
fireattack b57bbe2af0 Get higher Post video and StoryItem image quality if logged-in (#712) 2020-08-21 19:37:38 +02:00
28 changed files with 748 additions and 297 deletions
+6 -1
View File
@@ -24,4 +24,9 @@ If applicable, add error messages and tracebacks to help explain your problem.
**Additional context** **Additional context**
Add any other context about the problem here. Add any other context about the problem here.
<!-- please also see https://instaloader.github.io/contributing.html for how to report a bug --> <!--
Also see https://instaloader.github.io/contributing.html for how to report a bug.
Please understand that a bug issue that lacks the required information cannot be
processed.
-->
+8 -2
View File
@@ -7,6 +7,12 @@ labels: question
Your question here... Your question here...
<!-- Please see our documentation: https://instaloader.github.io/ --> <!--
People will be better able and more motivated to provide help if your question
<!-- Basic questions regarding Instaloader's usage might be more appropriate to ask on Stack Overflow --> - has a concise and meaningful title,
- is not covered by the documentation: https://instaloader.github.io/,
- is actually related to Instaloader (this is not a Python help forum),
- has not already been answered here or on other sites such as Stack Overflow: https://stackoverflow.com/questions/tagged/instaloader,
- is written in a polite and welcoming tone.
-->
+1
View File
@@ -0,0 +1 @@
custom: "https://instaloader.github.io/contributing.html#donations"
+2 -1
View File
@@ -1,7 +1,8 @@
<!-- <!--
Please describe: Thanks for your willingness to contribute to Instaloader! Please briefly
describe
- A motivation for this change, e.g. - A motivation for this change, e.g.
- Fixes # . - Fixes # .
+3 -2
View File
@@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
python-version: [3.8] python-version: [3.8, 3.9]
steps: steps:
- name: Checkout Instaloader Repository - name: Checkout Instaloader Repository
uses: actions/checkout@v2 uses: actions/checkout@v2
@@ -20,11 +20,12 @@ jobs:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Dependencies - name: Install Dependencies
run: | run: |
python -m pip install pipenv==2020.6.2 python -m pip install pipenv==2020.11.15
pipenv sync --dev pipenv sync --dev
- name: PyLint - name: PyLint
run: pipenv run pylint instaloader run: pipenv run pylint instaloader
- name: MyPy - name: MyPy
run: pipenv run mypy -m instaloader run: pipenv run mypy -m instaloader
- name: Build Documentation - name: Build Documentation
if: matrix.python-version == '3.8'
run: pipenv run make -C docs html SPHINXOPTS="-W -n" run: pipenv run make -C docs html SPHINXOPTS="-W -n"
+1 -1
View File
@@ -12,7 +12,7 @@ jobs:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@v1 uses: actions/setup-python@v1
with: with:
python-version: 3.8 python-version: 3.9
architecture: x64 architecture: x64
- name: Get the tagged version - name: Get the tagged version
id: get_version id: get_version
-1
View File
@@ -1,7 +1,6 @@
dist: xenial dist: xenial
language: python language: python
python: python:
- "3.5"
- "3.6" - "3.6"
- "3.7" - "3.7"
install: install:
+1
View File
@@ -11,6 +11,7 @@ pyinstaller = "*"
pefile = "*" pefile = "*"
pywin32-ctypes = "*" pywin32-ctypes = "*"
psutil = "*" psutil = "*"
typing-extensions = "*"
[packages] [packages]
requests = "*" requests = "*"
Generated
+239 -141
View File
@@ -1,7 +1,7 @@
{ {
"_meta": { "_meta": {
"hash": { "hash": {
"sha256": "58caf1be034bed9aa7ebb8c78e75ffa7d7abe820d014b6398c444a5db566b306" "sha256": "9704331c2f9e76af9e5ee386bc359a46ee66d1e2f93aa880d9d148da7ba0e6f5"
}, },
"pipfile-spec": 6, "pipfile-spec": 6,
"requires": {}, "requires": {},
@@ -16,39 +16,42 @@
"default": { "default": {
"certifi": { "certifi": {
"hashes": [ "hashes": [
"sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3", "sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41" "sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
], ],
"version": "==2020.6.20" "version": "==2020.12.5"
}, },
"chardet": { "chardet": {
"hashes": [ "hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae", "sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691" "sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
], ],
"version": "==3.0.4" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==4.0.0"
}, },
"idna": { "idna": {
"hashes": [ "hashes": [
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6", "sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0" "sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.10" "version": "==2.10"
}, },
"requests": { "requests": {
"hashes": [ "hashes": [
"sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b", "sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898" "sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
], ],
"index": "pypi", "index": "pypi",
"version": "==2.24.0" "version": "==2.25.1"
}, },
"urllib3": { "urllib3": {
"hashes": [ "hashes": [
"sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a", "sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df",
"sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461" "sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937"
], ],
"version": "==1.25.10" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.4"
} }
}, },
"develop": { "develop": {
@@ -68,43 +71,48 @@
}, },
"astroid": { "astroid": {
"hashes": [ "hashes": [
"sha256:2f4078c2a41bf377eea06d71c9d2ba4eb8f6b1af2135bec27bbbb7d8f12bb703", "sha256:21d735aab248253531bb0f1e1e6d068f0ee23533e18ae8a6171ff892b98297cf",
"sha256:bc58d83eb610252fd8de6363e39d4f1d0619c894b0ed24603b881c02e64c7386" "sha256:cfc35498ee64017be059ceffab0a25bedf7548ab76f2bea691c5565896e7128d"
], ],
"version": "==2.4.2" "markers": "python_version >= '3.6'",
"version": "==2.5.1"
}, },
"babel": { "babel": {
"hashes": [ "hashes": [
"sha256:1aac2ae2d0d8ea368fa90906567f5c08463d98ade155c0c4bfedd6a0f7160e38", "sha256:9d35c22fcc79893c3ecc85ac4a56cde1ecf3f19c540bba0922308a6c06ca6fa5",
"sha256:d670ea0b10f8b723672d3a6abeb87b565b244da220d76b4dba1b66269ec152d4" "sha256:da031ab54472314f210b0adcff1588ee5d1d1d0ba4dbd07b94dba82bde791e05"
], ],
"version": "==2.8.0" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.9.0"
}, },
"certifi": { "certifi": {
"hashes": [ "hashes": [
"sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3", "sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41" "sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
], ],
"version": "==2020.6.20" "version": "==2020.12.5"
}, },
"chardet": { "chardet": {
"hashes": [ "hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae", "sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691" "sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
], ],
"version": "==3.0.4" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==4.0.0"
}, },
"docutils": { "docutils": {
"hashes": [ "hashes": [
"sha256:0c5b78adfbf7762415433f5515cd5c9e762339e23369dbe8000d84a4bf4ab3af", "sha256:0c5b78adfbf7762415433f5515cd5c9e762339e23369dbe8000d84a4bf4ab3af",
"sha256:c2de3a60e9e7d07be26b7f2b00ca0309c207e06c100f9cc2a94931fc75a478fc" "sha256:c2de3a60e9e7d07be26b7f2b00ca0309c207e06c100f9cc2a94931fc75a478fc"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==0.16" "version": "==0.16"
}, },
"future": { "future": {
"hashes": [ "hashes": [
"sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d" "sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d"
], ],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.18.2" "version": "==0.18.2"
}, },
"idna": { "idna": {
@@ -112,6 +120,7 @@
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6", "sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0" "sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.10" "version": "==2.10"
}, },
"imagesize": { "imagesize": {
@@ -119,47 +128,62 @@
"sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1", "sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1",
"sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1" "sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.2.0" "version": "==1.2.0"
}, },
"importlib-metadata": {
"hashes": [
"sha256:742add720a20d0467df2f444ae41704000f50e1234f46174b51f9c6031a1bd71",
"sha256:b74159469b464a99cb8cc3e21973e4d96e05d3024d337313fedb618a6e86e6f4"
],
"markers": "python_version < '3.8'",
"version": "==3.7.3"
},
"isort": { "isort": {
"hashes": [ "hashes": [
"sha256:54da7e92468955c4fceacd0c86bd0ec997b0e1ee80d97f67c35a78b719dccab1", "sha256:c729845434366216d320e936b8ad6f9d681aab72dc7cbc2d51bedc3582f3ad1e",
"sha256:6e811fcb295968434526407adb8796944f1988c5b65e8139058f2014cbe100fd" "sha256:fff4f0c04e1825522ce6949973e83110a6e907750cd92d128b0d14aaaadbffdc"
], ],
"version": "==4.3.21" "markers": "python_version >= '3.6' and python_version < '4.0'",
"version": "==5.7.0"
}, },
"jinja2": { "jinja2": {
"hashes": [ "hashes": [
"sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0", "sha256:03e47ad063331dd6a3f04a43eddca8a966a26ba0c5b7207a9a9e4e08f1b29419",
"sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035" "sha256:a6d58433de0ae800347cab1fa3043cebbabe8baa9d29e668f1c768cb87a333c6"
], ],
"version": "==2.11.2" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==2.11.3"
}, },
"lazy-object-proxy": { "lazy-object-proxy": {
"hashes": [ "hashes": [
"sha256:0c4b206227a8097f05c4dbdd323c50edf81f15db3b8dc064d08c62d37e1a504d", "sha256:1d33d6f789697f401b75ce08e73b1de567b947740f768376631079290118ad39",
"sha256:194d092e6f246b906e8f70884e620e459fc54db3259e60cf69a4d66c3fda3449", "sha256:2f2de8f8ac0be3e40d17730e0600619d35c78c13a099ea91ef7fb4ad944ce694",
"sha256:1be7e4c9f96948003609aa6c974ae59830a6baecc5376c25c92d7d697e684c08", "sha256:3782931963dc89e0e9a0ae4348b44762e868ea280e4f8c233b537852a8996ab9",
"sha256:4677f594e474c91da97f489fea5b7daa17b5517190899cf213697e48d3902f5a", "sha256:37d9c34b96cca6787fe014aeb651217944a967a5b165e2cacb6b858d2997ab84",
"sha256:48dab84ebd4831077b150572aec802f303117c8cc5c871e182447281ebf3ac50", "sha256:38c3865bd220bd983fcaa9aa11462619e84a71233bafd9c880f7b1cb753ca7fa",
"sha256:5541cada25cd173702dbd99f8e22434105456314462326f06dba3e180f203dfd", "sha256:429c4d1862f3fc37cd56304d880f2eae5bd0da83bdef889f3bd66458aac49128",
"sha256:59f79fef100b09564bc2df42ea2d8d21a64fdcda64979c0fa3db7bdaabaf6239", "sha256:522b7c94b524389f4a4094c4bf04c2b02228454ddd17c1a9b2801fac1d754871",
"sha256:8d859b89baf8ef7f8bc6b00aa20316483d67f0b1cbf422f5b4dc56701c8f2ffb", "sha256:57fb5c5504ddd45ed420b5b6461a78f58cbb0c1b0cbd9cd5a43ad30a4a3ee4d0",
"sha256:9254f4358b9b541e3441b007a0ea0764b9d056afdeafc1a5569eee1cc6c1b9ea", "sha256:5944a9b95e97de1980c65f03b79b356f30a43de48682b8bdd90aa5089f0ec1f4",
"sha256:9651375199045a358eb6741df3e02a651e0330be090b3bc79f6d0de31a80ec3e", "sha256:6f4e5e68b7af950ed7fdb594b3f19a0014a3ace0fedb86acb896e140ffb24302",
"sha256:97bb5884f6f1cdce0099f86b907aa41c970c3c672ac8b9c8352789e103cf3156", "sha256:71a1ef23f22fa8437974b2d60fedb947c99a957ad625f83f43fd3de70f77f458",
"sha256:9b15f3f4c0f35727d3a0fba4b770b3c4ebbb1fa907dbcc046a1d2799f3edd142", "sha256:8a44e9901c0555f95ac401377032f6e6af66d8fc1fbfad77a7a8b1a826e0b93c",
"sha256:a2238e9d1bb71a56cd710611a1614d1194dc10a175c1e08d75e1a7bcc250d442", "sha256:b6577f15d5516d7d209c1a8cde23062c0f10625f19e8dc9fb59268859778d7d7",
"sha256:a6ae12d08c0bf9909ce12385803a543bfe99b95fe01e752536a60af2b7797c62", "sha256:c8fe2d6ff0ff583784039d0255ea7da076efd08507f2be6f68583b0da32e3afb",
"sha256:ca0a928a3ddbc5725be2dd1cf895ec0a254798915fb3a36af0964a0a4149e3db", "sha256:cadfa2c2cf54d35d13dc8d231253b7985b97d629ab9ca6e7d672c35539d38163",
"sha256:cb2c7c57005a6804ab66f106ceb8482da55f5314b7fcb06551db1edae4ad1531", "sha256:cd1bdace1a8762534e9a36c073cd54e97d517a17d69a17985961265be6d22847",
"sha256:d74bb8693bf9cf75ac3b47a54d716bbb1a92648d5f781fc799347cfc95952383", "sha256:ddbdcd10eb999d7ab292677f588b658372aadb9a52790f82484a37127a390108",
"sha256:d945239a5639b3ff35b70a88c5f2f491913eb94871780ebfabb2568bd58afc5a", "sha256:e7273c64bccfd9310e9601b8f4511d84730239516bada26a0c9846c9697617ef",
"sha256:eba7011090323c1dadf18b3b689845fd96a61ba0a1dfbd7f24b921398affc357", "sha256:e7428977763150b4cf83255625a80a23dfdc94d43be7791ce90799d446b4e26f",
"sha256:efa1909120ce98bbb3777e8b6f92237f5d5c8ea6758efea36a473e1d38f7d3e4", "sha256:e960e8be509e8d6d618300a6c189555c24efde63e85acaf0b14b2cd1ac743315",
"sha256:f3900e8a5de27447acbf900b4750b0ddfd7ec1ea7fbaf11dfa911141bc522af0" "sha256:ecb5dd5990cec6e7f5c9c1124a37cb2c710c6d69b0c1a5c4aa4b35eba0ada068",
"sha256:ef3f5e288aa57b73b034ce9c1f1ac753d968f9069cd0742d1d69c698a0167166",
"sha256:fa5b2dee0e231fa4ad117be114251bdfe6afe39213bd629d43deb117b6a6c40a",
"sha256:fa7fb7973c622b9e725bee1db569d2c2ee64d2f9a089201c5e8185d482c7352d"
], ],
"version": "==1.4.3" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==1.5.2"
}, },
"markupsafe": { "markupsafe": {
"hashes": [ "hashes": [
@@ -168,8 +192,12 @@
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235", "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5", "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
"sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42", "sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
"sha256:195d7d2c4fbb0ee8139a6cf67194f3973a6b3042d742ebe0a9ed36d8b6f0c07f",
"sha256:22c178a091fc6630d0d045bdb5992d2dfe14e3259760e713c490da5323866c39",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff", "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b", "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
"sha256:2beec1e0de6924ea551859edb9e7679da6e4870d32cb766240ce17e0a0ba2014",
"sha256:3b8a6499709d29c2e2399569d96719a1b21dcd94410a586a18526b143ec8470f",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1", "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e", "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183", "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
@@ -178,25 +206,41 @@
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1", "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
"sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15", "sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1", "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
"sha256:6f1e273a344928347c1290119b493a1f0303c52f5a5eae5f16d74f48c15d4a85",
"sha256:6fffc775d90dcc9aed1b89219549b329a9250d918fd0b8fa8d93d154918422e1",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e", "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b", "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905", "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
"sha256:7fed13866cf14bba33e7176717346713881f56d9d2bcebab207f7a036f41b850",
"sha256:84dee80c15f1b560d55bcfe6d47b27d070b4681c699c572af2e3c7cc90a3b8e0",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735", "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d", "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
"sha256:98bae9582248d6cf62321dcb52aaf5d9adf0bad3b40582925ef7c7f0ed85fceb",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e", "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d", "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c", "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
"sha256:a6a744282b7718a2a62d2ed9d993cad6f5f585605ad352c11de459f4108df0a1",
"sha256:acf08ac40292838b3cbbb06cfe9b2cb9ec78fce8baca31ddb87aaac2e2dc3bc2",
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21", "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2", "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5", "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b1dba4527182c95a0db8b6060cc98ac49b9e2f5e64320e2b56e47cb2831978c7",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b", "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:b7d644ddb4dbd407d31ffb699f1d140bc35478da613b441c582aeb7c43838dd8",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6", "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:bf5aa3cbcfdf57fa2ee9cd1822c862ef23037f5c832ad09cfea57fa846dec193",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f", "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:caabedc8323f1e93231b52fc32bdcde6db817623d33e100708d9a68e1f53b26b",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f", "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2", "sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
"sha256:d53bc011414228441014aa71dbec320c66468c1030aae3a6e29778a3382d96e5",
"sha256:d73a845f227b0bfe8a7455ee623525ee656a9e2e749e4742706d80a6065d5e2c",
"sha256:d9be0ba6c527163cbed5e0857c451fcd092ce83947944d6c14bc95441203f032",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7", "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be" "sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be",
"sha256:feb7b34d6325451ef96bc0e36e1a6c0c1c64bc1fbec4b854f4529e51887b1621"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.1.1" "version": "==1.1.1"
}, },
"mccabe": { "mccabe": {
@@ -208,23 +252,31 @@
}, },
"mypy": { "mypy": {
"hashes": [ "hashes": [
"sha256:2c6cde8aa3426c1682d35190b59b71f661237d74b053822ea3d748e2c9578a7c", "sha256:0d0a87c0e7e3a9becdfbe936c981d32e5ee0ccda3e0f07e1ef2c3d1a817cf73e",
"sha256:3fdda71c067d3ddfb21da4b80e2686b71e9e5c72cca65fa216d207a358827f86", "sha256:25adde9b862f8f9aac9d2d11971f226bd4c8fbaa89fb76bdadb267ef22d10064",
"sha256:5dd13ff1f2a97f94540fd37a49e5d255950ebcdf446fb597463a40d0df3fac8b", "sha256:28fb5479c494b1bab244620685e2eb3c3f988d71fd5d64cc753195e8ed53df7c",
"sha256:6731603dfe0ce4352c555c6284c6db0dc935b685e9ce2e4cf220abe1e14386fd", "sha256:2f9b3407c58347a452fc0736861593e105139b905cca7d097e413453a1d650b4",
"sha256:6bb93479caa6619d21d6e7160c552c1193f6952f0668cdda2f851156e85186fc", "sha256:33f159443db0829d16f0a8d83d94df3109bb6dd801975fe86bacb9bf71628e97",
"sha256:81c7908b94239c4010e16642c9102bfc958ab14e36048fa77d0be3289dda76ea", "sha256:3f2aca7f68580dc2508289c729bd49ee929a436208d2b2b6aab15745a70a57df",
"sha256:9c7a9a7ceb2871ba4bac1cf7217a7dd9ccd44c27c2950edbc6dc08530f32ad4e", "sha256:499c798053cdebcaa916eef8cd733e5584b5909f789de856b482cd7d069bdad8",
"sha256:a4a2cbcfc4cbf45cd126f531dedda8485671545b43107ded25ce952aac6fb308", "sha256:4eec37370483331d13514c3f55f446fc5248d6373e7029a29ecb7b7494851e7a",
"sha256:b7fbfabdbcc78c4f6fc4712544b9b0d6bf171069c6e0e3cb82440dd10ced3406", "sha256:552a815579aa1e995f39fd05dde6cd378e191b063f031f2acfe73ce9fb7f9e56",
"sha256:c05b9e4fb1d8a41d41dec8786c94f3b95d3c5f528298d769eb8e73d293abc48d", "sha256:5873888fff1c7cf5b71efbe80e0e73153fe9212fafdf8e44adfe4c20ec9f82d7",
"sha256:d7df6eddb6054d21ca4d3c6249cae5578cb4602951fd2b6ee2f5510ffb098707", "sha256:61a3d5b97955422964be6b3baf05ff2ce7f26f52c85dd88db11d5e03e146a3a6",
"sha256:e0b61738ab504e656d1fe4ff0c0601387a5489ca122d55390ade31f9ca0e252d", "sha256:674e822aa665b9fd75130c6c5f5ed9564a38c6cea6a6432ce47eafb68ee578c5",
"sha256:eff7d4a85e9eea55afa34888dfeaccde99e7520b51f867ac28a48492c0b1130c", "sha256:7ce3175801d0ae5fdfa79b4f0cfed08807af4d075b402b7e294e6aa72af9aa2a",
"sha256:f05644db6779387ccdb468cc47a44b4356fc2ffa9287135d05b70a98dc83b89a" "sha256:9743c91088d396c1a5a3c9978354b61b0382b4e3c440ce83cf77994a43e8c521",
"sha256:9f94aac67a2045ec719ffe6111df543bac7874cee01f41928f6969756e030564",
"sha256:a26f8ec704e5a7423c8824d425086705e381b4f1dfdef6e3a1edab7ba174ec49",
"sha256:abf7e0c3cf117c44d9285cc6128856106183938c68fd4944763003decdcfeb66",
"sha256:b09669bcda124e83708f34a94606e01b614fa71931d356c1f1a5297ba11f110a",
"sha256:cd07039aa5df222037005b08fbbfd69b3ab0b0bd7a07d7906de75ae52c4e3119",
"sha256:d23e0ea196702d918b60c8288561e722bf437d82cb7ef2edcd98cfa38905d506",
"sha256:d65cc1df038ef55a99e617431f0553cd77763869eebdf9042403e16089fe746c",
"sha256:d7da2e1d5f558c37d6e8c1246f1aec1e7349e4913d8fb3cb289a35de573fe2eb"
], ],
"index": "pypi", "index": "pypi",
"version": "==0.782" "version": "==0.812"
}, },
"mypy-extensions": { "mypy-extensions": {
"hashes": [ "hashes": [
@@ -235,10 +287,11 @@
}, },
"packaging": { "packaging": {
"hashes": [ "hashes": [
"sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8", "sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
"sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181" "sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
], ],
"version": "==20.4" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==20.9"
}, },
"pefile": { "pefile": {
"hashes": [ "hashes": [
@@ -249,56 +302,82 @@
}, },
"psutil": { "psutil": {
"hashes": [ "hashes": [
"sha256:0ee3c36428f160d2d8fce3c583a0353e848abb7de9732c50cf3356dd49ad63f8", "sha256:0066a82f7b1b37d334e68697faba68e5ad5e858279fd6351c8ca6024e8d6ba64",
"sha256:10512b46c95b02842c225f58fa00385c08fa00c68bac7da2d9a58ebe2c517498", "sha256:02b8292609b1f7fcb34173b25e48d0da8667bc85f81d7476584d889c6e0f2131",
"sha256:4080869ed93cce662905b029a1770fe89c98787e543fa7347f075ade761b19d6", "sha256:0ae6f386d8d297177fd288be6e8d1afc05966878704dad9847719650e44fc49c",
"sha256:5e9d0f26d4194479a13d5f4b3798260c20cecf9ac9a461e718eb59ea520a360c", "sha256:0c9ccb99ab76025f2f0bbecf341d4656e9c1351db8cc8a03ccd62e318ab4b5c6",
"sha256:66c18ca7680a31bf16ee22b1d21b6397869dda8059dbdb57d9f27efa6615f195", "sha256:0dd4465a039d343925cdc29023bb6960ccf4e74a65ad53e768403746a9207023",
"sha256:68d36986ded5dac7c2dcd42f2682af1db80d4bce3faa126a6145c1637e1b559f", "sha256:12d844996d6c2b1d3881cfa6fa201fd635971869a9da945cf6756105af73d2df",
"sha256:90990af1c3c67195c44c9a889184f84f5b2320dce3ee3acbd054e3ba0b4a7beb", "sha256:1bff0d07e76114ec24ee32e7f7f8d0c4b0514b3fae93e3d2aaafd65d22502394",
"sha256:a5b120bb3c0c71dfe27551f9da2f3209a8257a178ed6c628a819037a8df487f1", "sha256:245b5509968ac0bd179287d91210cd3f37add77dad385ef238b275bad35fa1c4",
"sha256:d8a82162f23c53b8525cf5f14a355f5d1eea86fa8edde27287dd3a98399e4fdf", "sha256:28ff7c95293ae74bf1ca1a79e8805fcde005c18a122ca983abf676ea3466362b",
"sha256:f2018461733b23f308c298653c8903d32aaad7873d25e1d228765e91ae42c3f2", "sha256:36b3b6c9e2a34b7d7fbae330a85bf72c30b1c827a4366a07443fc4b6270449e2",
"sha256:ff1977ba1a5f71f89166d5145c3da1cea89a0fdb044075a12c720ee9123ec818" "sha256:52de075468cd394ac98c66f9ca33b2f54ae1d9bff1ef6b67a212ee8f639ec06d",
"sha256:5da29e394bdedd9144c7331192e20c1f79283fb03b06e6abd3a8ae45ffecee65",
"sha256:61f05864b42fedc0771d6d8e49c35f07efd209ade09a5afe6a5059e7bb7bf83d",
"sha256:6223d07a1ae93f86451d0198a0c361032c4c93ebd4bf6d25e2fb3edfad9571ef",
"sha256:6323d5d845c2785efb20aded4726636546b26d3b577aded22492908f7c1bdda7",
"sha256:6ffe81843131ee0ffa02c317186ed1e759a145267d54fdef1bc4ea5f5931ab60",
"sha256:74f2d0be88db96ada78756cb3a3e1b107ce8ab79f65aa885f76d7664e56928f6",
"sha256:74fb2557d1430fff18ff0d72613c5ca30c45cdbfcddd6a5773e9fc1fe9364be8",
"sha256:90d4091c2d30ddd0a03e0b97e6a33a48628469b99585e2ad6bf21f17423b112b",
"sha256:90f31c34d25b1b3ed6c40cdd34ff122b1887a825297c017e4cbd6796dd8b672d",
"sha256:99de3e8739258b3c3e8669cb9757c9a861b2a25ad0955f8e53ac662d66de61ac",
"sha256:c6a5fd10ce6b6344e616cf01cc5b849fa8103fbb5ba507b6b2dee4c11e84c935",
"sha256:ce8b867423291cb65cfc6d9c4955ee9bfc1e21fe03bb50e177f2b957f1c2469d",
"sha256:d225cd8319aa1d3c85bf195c4e07d17d3cd68636b8fc97e6cf198f782f99af28",
"sha256:ea313bb02e5e25224e518e4352af4bf5e062755160f77e4b1767dd5ccb65f876",
"sha256:ea372bcc129394485824ae3e3ddabe67dc0b118d262c568b4d2602a7070afdb0",
"sha256:f4634b033faf0d968bb9220dd1c793b897ab7f1189956e1aa9eae752527127d3",
"sha256:fcc01e900c1d7bee2a37e5d6e4f9194760a93597c97fee89c4ae51701de03563"
], ],
"index": "pypi", "index": "pypi",
"version": "==5.7.2" "version": "==5.8.0"
}, },
"pygments": { "pygments": {
"hashes": [ "hashes": [
"sha256:647344a061c249a3b74e230c739f434d7ea4d8b1d5f3721bc0f3558049b38f44", "sha256:2656e1a6edcdabf4275f9a3640db59fd5de107d88e8663c5d4e9a0fa62f77f94",
"sha256:ff7a40b4860b727ab48fad6360eb351cc1b33cbf9b15a0f689ca5353e9463324" "sha256:534ef71d539ae97d4c3a4cf7d6f110f214b0e687e92f9cb9d2a3b0d3101289c8"
], ],
"version": "==2.6.1" "markers": "python_version >= '3.5'",
"version": "==2.8.1"
}, },
"pyinstaller": { "pyinstaller": {
"hashes": [ "hashes": [
"sha256:3730fa80d088f8bb7084d32480eb87cbb4ddb64123363763cf8f2a1378c1c4b7" "sha256:f5c0eeb2aa663cce9a5404292c0195011fa500a6501c873a466b2e8cad3c950c"
], ],
"index": "pypi", "index": "pypi",
"version": "==3.6" "version": "==4.2"
},
"pyinstaller-hooks-contrib": {
"hashes": [
"sha256:27558072021857d89524c42136feaa2ffe4f003f1bdf0278f9b24f6902c1759c",
"sha256:892310e6363655838485ee748bf1c5e5cade7963686d9af8650ee218a3e0b031"
],
"version": "==2021.1"
}, },
"pylint": { "pylint": {
"hashes": [ "hashes": [
"sha256:7dd78437f2d8d019717dbf287772d0b2dbdfd13fc016aa7faa08d67bccc46adc", "sha256:0e21d3b80b96740909d77206d741aa3ce0b06b41be375d92e1f3244a274c1f8a",
"sha256:d0ece7d223fe422088b0e8f13fa0a1e8eb745ebffcb8ed53d3e95394b6101a1c" "sha256:d09b0b07ba06bcdff463958f53f23df25e740ecd81895f7d2699ec04bbd8dc3b"
], ],
"index": "pypi", "index": "pypi",
"version": "==2.5.3" "version": "==2.7.2"
}, },
"pyparsing": { "pyparsing": {
"hashes": [ "hashes": [
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1", "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b" "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
], ],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.4.7" "version": "==2.4.7"
}, },
"pytz": { "pytz": {
"hashes": [ "hashes": [
"sha256:a494d53b6d39c3c6e44c3bec237336e14305e4f29bbf800b599253057fbb79ed", "sha256:83a4a90894bf38e243cf052c8b58f381bfe9a7a483f6a9cab140bc7f702ac4da",
"sha256:c35965d010ce31b23eeb663ed3cc8c906275d6be1a34393a1d73a41febf4a048" "sha256:eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798"
], ],
"version": "==2020.1" "version": "==2021.1"
}, },
"pywin32-ctypes": { "pywin32-ctypes": {
"hashes": [ "hashes": [
@@ -310,25 +389,18 @@
}, },
"requests": { "requests": {
"hashes": [ "hashes": [
"sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b", "sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898" "sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
], ],
"index": "pypi", "index": "pypi",
"version": "==2.24.0" "version": "==2.25.1"
},
"six": {
"hashes": [
"sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
"sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
],
"version": "==1.15.0"
}, },
"snowballstemmer": { "snowballstemmer": {
"hashes": [ "hashes": [
"sha256:209f257d7533fdb3cb73bdbd24f436239ca3b2fa67d56f6ff88e86be08cc5ef0", "sha256:b51b447bea85f9968c13b650126a888aabd4cb4463fca868ec596826325dedc2",
"sha256:df3bac3df4c2c01363f3dd2cfa78cce2840a79b9f1c2d2de9ce8d31683992f52" "sha256:e997baa4f2e9139951b6f4c631bad912dfd3c792467e2f03d7239464af90e914"
], ],
"version": "==2.0.0" "version": "==2.1.0"
}, },
"sphinx": { "sphinx": {
"hashes": [ "hashes": [
@@ -343,6 +415,7 @@
"sha256:806111e5e962be97c29ec4c1e7fe277bfd19e9652fb1a4392105b43e01af885a", "sha256:806111e5e962be97c29ec4c1e7fe277bfd19e9652fb1a4392105b43e01af885a",
"sha256:a072735ec80e7675e3f432fcae8610ecf509c5f1869d17e2eecff44389cdbc58" "sha256:a072735ec80e7675e3f432fcae8610ecf509c5f1869d17e2eecff44389cdbc58"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.2" "version": "==1.0.2"
}, },
"sphinxcontrib-devhelp": { "sphinxcontrib-devhelp": {
@@ -350,6 +423,7 @@
"sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e", "sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e",
"sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4" "sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.2" "version": "==1.0.2"
}, },
"sphinxcontrib-htmlhelp": { "sphinxcontrib-htmlhelp": {
@@ -357,6 +431,7 @@
"sha256:3c0bc24a2c41e340ac37c85ced6dafc879ab485c095b1d65d2461ac2f7cca86f", "sha256:3c0bc24a2c41e340ac37c85ced6dafc879ab485c095b1d65d2461ac2f7cca86f",
"sha256:e8f5bb7e31b2dbb25b9cc435c8ab7a79787ebf7f906155729338f3156d93659b" "sha256:e8f5bb7e31b2dbb25b9cc435c8ab7a79787ebf7f906155729338f3156d93659b"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.3" "version": "==1.0.3"
}, },
"sphinxcontrib-jsmath": { "sphinxcontrib-jsmath": {
@@ -364,6 +439,7 @@
"sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178",
"sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8" "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.1" "version": "==1.0.1"
}, },
"sphinxcontrib-qthelp": { "sphinxcontrib-qthelp": {
@@ -371,6 +447,7 @@
"sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72", "sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72",
"sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6" "sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.3" "version": "==1.0.3"
}, },
"sphinxcontrib-serializinghtml": { "sphinxcontrib-serializinghtml": {
@@ -378,62 +455,83 @@
"sha256:eaa0eccc86e982a9b939b2b82d12cc5d013385ba5eadcc7e4fed23f4405f77bc", "sha256:eaa0eccc86e982a9b939b2b82d12cc5d013385ba5eadcc7e4fed23f4405f77bc",
"sha256:f242a81d423f59617a8e5cf16f5d4d74e28ee9a66f9e5b637a18082991db5a9a" "sha256:f242a81d423f59617a8e5cf16f5d4d74e28ee9a66f9e5b637a18082991db5a9a"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.1.4" "version": "==1.1.4"
}, },
"toml": { "toml": {
"hashes": [ "hashes": [
"sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f", "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b",
"sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88" "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
], ],
"version": "==0.10.1" "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.10.2"
}, },
"typed-ast": { "typed-ast": {
"hashes": [ "hashes": [
"sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355", "sha256:07d49388d5bf7e863f7fa2f124b1b1d89d8aa0e2f7812faff0a5658c01c59aa1",
"sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919", "sha256:14bf1522cdee369e8f5581238edac09150c765ec1cb33615855889cf33dcb92d",
"sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa", "sha256:240296b27397e4e37874abb1df2a608a92df85cf3e2a04d0d4d61055c8305ba6",
"sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652", "sha256:36d829b31ab67d6fcb30e185ec996e1f72b892255a745d3a82138c97d21ed1cd",
"sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75", "sha256:37f48d46d733d57cc70fd5f30572d11ab8ed92da6e6b28e024e4a3edfb456e37",
"sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01", "sha256:4c790331247081ea7c632a76d5b2a265e6d325ecd3179d06e9cf8d46d90dd151",
"sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d", "sha256:5dcfc2e264bd8a1db8b11a892bd1647154ce03eeba94b461effe68790d8b8e07",
"sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1", "sha256:7147e2a76c75f0f64c4319886e7639e490fee87c9d25cb1d4faef1d8cf83a440",
"sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907", "sha256:7703620125e4fb79b64aa52427ec192822e9f45d37d4b6625ab37ef403e1df70",
"sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c", "sha256:8368f83e93c7156ccd40e49a783a6a6850ca25b556c0fa0240ed0f659d2fe496",
"sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3", "sha256:84aa6223d71012c68d577c83f4e7db50d11d6b1399a9c779046d75e24bed74ea",
"sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b", "sha256:85f95aa97a35bdb2f2f7d10ec5bbdac0aeb9dafdaf88e17492da0504de2e6400",
"sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614", "sha256:8db0e856712f79c45956da0c9a40ca4246abc3485ae0d7ecc86a20f5e4c09abc",
"sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb", "sha256:9044ef2df88d7f33692ae3f18d3be63dec69c4fb1b5a4a9ac950f9b4ba571606",
"sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b", "sha256:963c80b583b0661918718b095e02303d8078950b26cc00b5e5ea9ababe0de1fc",
"sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41", "sha256:987f15737aba2ab5f3928c617ccf1ce412e2e321c77ab16ca5a293e7bbffd581",
"sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6", "sha256:9ec45db0c766f196ae629e509f059ff05fc3148f9ffd28f3cfe75d4afb485412",
"sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34", "sha256:9fc0b3cb5d1720e7141d103cf4819aea239f7d136acf9ee4a69b047b7986175a",
"sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe", "sha256:a2c927c49f2029291fbabd673d51a2180038f8cd5a5b2f290f78c4516be48be2",
"sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4", "sha256:a38878a223bdd37c9709d07cd357bb79f4c760b29210e14ad0fb395294583787",
"sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7" "sha256:b4fcdcfa302538f70929eb7b392f536a237cbe2ed9cba88e3bf5027b39f5f77f",
"sha256:c0c74e5579af4b977c8b932f40a5464764b2f86681327410aa028a22d2f54937",
"sha256:c1c876fd795b36126f773db9cbb393f19808edd2637e00fd6caba0e25f2c7b64",
"sha256:c9aadc4924d4b5799112837b226160428524a9a45f830e0d0f184b19e4090487",
"sha256:cc7b98bf58167b7f2db91a4327da24fb93368838eb84a44c472283778fc2446b",
"sha256:cf54cfa843f297991b7388c281cb3855d911137223c6b6d2dd82a47ae5125a41",
"sha256:d003156bb6a59cda9050e983441b7fa2487f7800d76bdc065566b7d728b4581a",
"sha256:d175297e9533d8d37437abc14e8a83cbc68af93cc9c1c59c2c292ec59a0697a3",
"sha256:d746a437cdbca200622385305aedd9aef68e8a645e385cc483bdc5e488f07166",
"sha256:e683e409e5c45d5c9082dc1daf13f6374300806240719f95dc783d1fc942af10"
], ],
"markers": "implementation_name == 'cpython' and python_version < '3.8'", "markers": "python_version < '3.8' and implementation_name == 'cpython'",
"version": "==1.4.1" "version": "==1.4.2"
}, },
"typing-extensions": { "typing-extensions": {
"hashes": [ "hashes": [
"sha256:6e95524d8a547a91e08f404ae485bbb71962de46967e1b71a0cb89af24e761c5", "sha256:7cb407020f00f7bfc3cb3e7881628838e69d8f3fcab2f64742a5e76b2f841918",
"sha256:79ee589a3caca649a9bfd2a8de4709837400dfa00b6cc81962a1e6a1815969ae", "sha256:99d4073b617d30288f569d3f13d2bd7548c3a7e4c8de87db09a9d29bb3a4a60c",
"sha256:f8d2bd89d25bc39dabe7d23df520442fa1d8969b82544370e03d88b5a591c392" "sha256:dafc7639cde7f1b6e1acc0f457842a83e722ccca8eef5270af2d74792619a89f"
], ],
"version": "==3.7.4.2" "index": "pypi",
"version": "==3.7.4.3"
}, },
"urllib3": { "urllib3": {
"hashes": [ "hashes": [
"sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a", "sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df",
"sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461" "sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937"
], ],
"version": "==1.25.10" "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.4"
}, },
"wrapt": { "wrapt": {
"hashes": [ "hashes": [
"sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7" "sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7"
], ],
"version": "==1.12.1" "version": "==1.12.1"
},
"zipp": {
"hashes": [
"sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76",
"sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098"
],
"markers": "python_version >= '3.6'",
"version": "==3.4.1"
} }
} }
} }
+4 -2
View File
@@ -29,7 +29,7 @@
.. |downloads| image:: https://pepy.tech/badge/instaloader/month .. |downloads| image:: https://pepy.tech/badge/instaloader/month
:alt: PyPI Download Count :alt: PyPI Download Count
:target: https://pepy.tech/project/instaloader/month :target: https://pepy.tech/project/instaloader
.. badges-end .. badges-end
@@ -84,7 +84,9 @@ To later **update your local copy** of that profiles, you may run
instaloader --fast-update profile [profile ...] instaloader --fast-update profile [profile ...]
If ``--fast-update`` is given, Instaloader stops when arriving at the If ``--fast-update`` is given, Instaloader stops when arriving at the
first already-downloaded picture. When updating profiles, Instaloader first already-downloaded picture.
When updating profiles, Instaloader
automatically **detects profile name changes** and renames the target directory automatically **detects profile name changes** and renames the target directory
accordingly. accordingly.
+1 -1
View File
@@ -8,7 +8,7 @@ arch=('any')
url="https://instaloader.github.io/" url="https://instaloader.github.io/"
license=('MIT') license=('MIT')
groups=() groups=()
depends=('python>=3.5' 'python-requests>=2.4') depends=('python>=3.6' 'python-requests>=2.4')
makedepends=('python-setuptools') makedepends=('python-setuptools')
options=('!emptydirs') options=('!emptydirs')
source=($pkgname-$pkgver.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v$pkgver) source=($pkgname-$pkgver.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v$pkgver)
+1 -1
View File
@@ -44,7 +44,7 @@ with open('__main__.py', 'w+') as f:
f.writelines(lines) f.writelines(lines)
# install dependencies and invoke PyInstaller # install dependencies and invoke PyInstaller
commands = ["pip install pipenv==2020.6.2", commands = ["pip install pipenv==2020.11.15",
"pipenv sync --dev", "pipenv sync --dev",
"pipenv run pyinstaller --log-level=DEBUG instaloader.spec"] "pipenv run pyinstaller --log-level=DEBUG instaloader.spec"]
+3
View File
@@ -171,6 +171,9 @@ dirname pattern. The following tokens are defined for usage with
- ``{mediaid}`` - ``{mediaid}``
Integer representation of shortcode. Integer representation of shortcode.
- ``{filename}``
Instagram's internal filename.
- ``{date_utc}`` (same as ``{date}``) - ``{date_utc}`` (same as ``{date}``)
Creation time in UTC timezone. Creation time in UTC timezone.
`strftime()-style formatting options <https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>`__ `strftime()-style formatting options <https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>`__
+37 -5
View File
@@ -82,6 +82,16 @@ What to Download of each Post
Template to write in txt file for each StoryItem. See Template to write in txt file for each StoryItem. See
:ref:`metadata-text-files`. :ref:`metadata-text-files`.
.. option:: --slide
Download only selected images of a sidecar. You can select single images using their
index in the sidecar starting with the leftmost or you can specify a range of images
with the following syntax: ``start_index-end_index``. Example:
``--slide 1`` will select only the first image, ``--slide last`` only the last one and ``--slide 1-3`` will select only
the first three images.
.. versionadded:: 4.6
.. option:: --no-metadata-json .. option:: --no-metadata-json
Do not create a JSON file containing the metadata of each post. Do not create a JSON file containing the metadata of each post.
@@ -210,12 +220,20 @@ How to Download
used to save the information to resume an interrupted download. The default used to save the information to resume an interrupted download. The default
prefix is ``iterator``. prefix is ``iterator``.
Resuming an interrupted download is supported for most, but not all targets. Resuming an interrupted download is supported for the following targets:
- Profile posts,
- Profile IGTV posts (:option:`--igtv`),
- Profile tagged posts (:option:`--tagged`),
- Saved posts (``:saved``).
This feature is enabled by default for targets where it is supported;
:option:`--resume-prefix` only changes the name of the iterator files.
To turn this feature off, use :option:`--no-resume`.
JSON files with resume information are always compressed, regardless of JSON files with resume information are always compressed, regardless of
:option:`--no-compress-json`. :option:`--no-compress-json`.
This feature is turned off entirely with :option:`--no-resume`.
.. versionadded:: 4.5 .. versionadded:: 4.5
.. option:: --no-resume .. option:: --no-resume
@@ -228,7 +246,7 @@ How to Download
.. option:: --user-agent USER_AGENT .. option:: --user-agent USER_AGENT
User Agent to use for HTTP requests. Per default, Instaloader pretends being User Agent to use for HTTP requests. Per default, Instaloader pretends being
Chrome/51. Chrome/89 on Linux.
.. option:: --max-connection-attempts N .. option:: --max-connection-attempts N
@@ -238,10 +256,24 @@ How to Download
.. option:: --request-timeout N .. option:: --request-timeout N
Seconds to wait before timing out a connection request. Seconds to wait before timing out a connection request. Defaults to 300.
.. versionadded:: 4.3 .. versionadded:: 4.3
.. versionchanged:: 4.6
Enabled this option by default with a timeout of 300 seconds.
.. option:: --abort-on STATUS_CODE_LIST
Comma-separated list of HTTP status codes that cause Instaloader to abort,
bypassing all retry logic.
For example, with ``--abort-on=302,400,429``, Instaloader will stop if a
request is responded with a 302 redirect, a Bad Request error, or a Too Many
Requests error.
.. versionadded:: 4.7
Miscellaneous Options Miscellaneous Options
^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^
@@ -23,7 +23,7 @@ def get_cookiefile():
def import_session(cookiefile, sessionfile): def import_session(cookiefile, sessionfile):
print("Using cookies from {}.".format(cookiefile)) print("Using cookies from {}.".format(cookiefile))
conn = connect(cookiefile) conn = connect(f"file:{cookiefile}?immutable=1", uri=True)
try: try:
cookie_data = conn.execute( cookie_data = conn.execute(
"SELECT name, value FROM moz_cookies WHERE baseDomain='instagram.com'" "SELECT name, value FROM moz_cookies WHERE baseDomain='instagram.com'"
+1 -1
View File
@@ -65,7 +65,7 @@ extlinks = {
'Issue #' 'Issue #'
), ),
'example': ( 'example': (
'https://raw.githubusercontent.com/instaloader/instaloader/v' + current_release + '/docs/codesnippets/%s', 'https://raw.githubusercontent.com/instaloader/instaloader/master/docs/codesnippets/%s',
'Example ' 'Example '
), ),
} }
+47 -13
View File
@@ -11,6 +11,23 @@ Instaloader's development is organized on
`GitHub <https://github.com/instaloader/instaloader>`__, where Issues and Pull `GitHub <https://github.com/instaloader/instaloader>`__, where Issues and Pull
Requests are discussed. Requests are discussed.
Contributing to an open source project is an unmatchable opportunity to improve
and manifest programming skills. For issues that are good starting points to get
involved in Instaloader development, as they do not require deep insight into
Instaloader's code base, have a look at our
`Issues labelled 'good first issue' <https://github.com/instaloader/instaloader/contribute>`__.
This document covers several ways of how you can contribute back to Instaloader.
Answering Questions
-------------------
The easiest way to help out is to answer questions. If you are interested in
answering questions regarding Instaloader, good places to begin are
- `Questions tagged 'instaloader' on Stack Overflow <https://stackoverflow.com/questions/tagged/instaloader>`__,
- `Instaloader Issues labeled 'question' <https://github.com/instaloader/instaloader/issues?q=is%3Aissue+is%3Aopen+label%3Aquestion>`__.
Reporting Bugs Reporting Bugs
-------------- --------------
@@ -41,22 +58,39 @@ reporting a problem, please keep the following in mind:
Writing Code or Improving the Documentation Writing Code or Improving the Documentation
------------------------------------------- -------------------------------------------
Changes of the Instaloader source can be proposed as a Improvements of the Instaloader source or its documentation can be proposed as a
`Pull Request <https://github.com/instaloader/instaloader/pulls>`__. There are only `Pull Request <https://github.com/instaloader/instaloader/pulls>`__.
few things to consider:
- Base your Pull Request on the ``master`` branch if it fixes a bug, - Please base your Pull Request on
or the ``upcoming/v4.X`` branch (if it exists at the moment of submitting the PR)
otherwise.
- We use `Pylint <https://www.pylint.org/>`__ for error and syntax checking of - ``master``, which will be released with the next minor release, if it is
the source and `MyPy <https://github.com/python/mypy>`__ for type checking.
Beware that sometimes it might be better to disable a warning rather than
adapting the code to a tool's desires.
- The documentation source is located in the ``docs`` folder. The file - a bug fix that does not require extensive testing,
``cli-options.rst`` is merely an RST-formatted copy of ``instaloader --help`` - an improvement to the documentation,
output, of which the source is in ``instaloader/__main__.py``.
- ``upcoming/v4.X``, if it is
- a new feature,
- a bug fix that does require thorough testing before being released to a
final version.
- All Pull Requests are analyzed by `Pylint <https://www.pylint.org/>`__ for
error and syntax checking of the source and
`Mypy <https://github.com/python/mypy>`__ for type checking. You can run them
locally with::
pylint instaloader
mypy -m instaloader
- Improvements to the documentation are very welcome. The documentation is
created with `Sphinx <https://www.sphinx-doc.org/en/2.0/>`__, version 2,
and can be build locally using::
make -C docs html
- Feel free to create an issue to make sure someone from the Instaloader team
agrees that the change might be an improvement, or if you want to discuss
your basic proposal, before working on a pull request.
Proposing Features Proposing Features
------------------ ------------------
+1 -1
View File
@@ -8,7 +8,7 @@ Install Instaloader
To **install Instaloader**, To **install Instaloader**,
#. Ensure that you have `Python <https://www.python.org/>`__, at least #. Ensure that you have `Python <https://www.python.org/>`__, at least
version 3.5, and `pip <https://pypi.python.org/pypi/pip>`__ version 3.6, and `pip <https://pypi.python.org/pypi/pip>`__
installed. installed.
#. Then, install Instaloader using:: #. Then, install Instaloader using::
+2
View File
@@ -47,3 +47,5 @@ Exceptions
.. autoexception:: TooManyRequestsException .. autoexception:: TooManyRequestsException
.. autoexception:: AbortDownloadException
+1 -1
View File
@@ -1,7 +1,7 @@
"""Download pictures (or videos) along with their captions and other metadata from Instagram.""" """Download pictures (or videos) along with their captions and other metadata from Instagram."""
__version__ = '4.5.5' __version__ = '4.7'
try: try:
+26 -9
View File
@@ -5,12 +5,12 @@ import datetime
import os import os
import re import re
import sys import sys
from argparse import ArgumentParser, SUPPRESS from argparse import ArgumentParser, ArgumentTypeError, SUPPRESS
from typing import List, Optional from typing import List, Optional
from . import (Instaloader, InstaloaderException, InvalidArgumentException, Post, Profile, ProfileNotExistsException, from . import (AbortDownloadException, BadCredentialsException, Instaloader, InstaloaderException,
StoryItem, __version__, load_structure_from_file, TwoFactorAuthRequiredException, InvalidArgumentException, Post, Profile, ProfileNotExistsException, StoryItem,
BadCredentialsException) TwoFactorAuthRequiredException, __version__, load_structure_from_file)
from .instaloader import get_default_session_filename from .instaloader import get_default_session_filename
from .instaloadercontext import default_user_agent from .instaloadercontext import default_user_agent
@@ -27,6 +27,14 @@ def usage_string():
{0} --help""".format(argv0, len(argv0), '') {0} --help""".format(argv0, len(argv0), '')
def http_status_code_list(code_list_str: str) -> List[int]:
codes = [int(s) for s in code_list_str.split(',')]
for code in codes:
if not 100 <= code <= 599:
raise ArgumentTypeError("Invalid HTTP status code: {}".format(code))
return codes
def filterstr_to_filterfunc(filter_str: str, item_type: type): def filterstr_to_filterfunc(filter_str: str, item_type: type):
"""Takes an --post-filter=... or --storyitem-filter=... filter """Takes an --post-filter=... or --storyitem-filter=... filter
specification and makes a filter_func Callable out of it.""" specification and makes a filter_func Callable out of it."""
@@ -199,7 +207,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
instaloader.context.log("Downloading {} profiles: {}".format(len(profiles), instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
' '.join([p.username for p in profiles]))) ' '.join([p.username for p in profiles])))
if profiles and (download_profile_pic or download_posts) and not instaloader.context.is_logged_in: if profiles and (download_profile_pic or download_posts) and not instaloader.context.is_logged_in:
instaloader.context.error("Warning: Use --login to download higher-quality versions of pictures.") instaloader.context.log("Hint: Use --login to download higher-quality versions of pictures.")
instaloader.download_profiles(profiles, instaloader.download_profiles(profiles,
download_profile_pic, download_posts, download_tagged, download_igtv, download_profile_pic, download_posts, download_tagged, download_igtv,
download_highlights, download_stories, download_highlights, download_stories,
@@ -213,6 +221,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
fast_update=fast_update, post_filter=post_filter) fast_update=fast_update, post_filter=post_filter)
except KeyboardInterrupt: except KeyboardInterrupt:
print("\nInterrupted by user.", file=sys.stderr) print("\nInterrupted by user.", file=sys.stderr)
except AbortDownloadException as exc:
print("\nDownload aborted: {}.".format(exc), file=sys.stderr)
# Save session if it is useful # Save session if it is useful
if instaloader.context.is_logged_in: if instaloader.context.is_logged_in:
instaloader.save_session_to_file(sessionfile) instaloader.save_session_to_file(sessionfile)
@@ -267,6 +277,8 @@ def main():
help="Do not download regular posts.") help="Do not download regular posts.")
g_prof.add_argument('--no-profile-pic', action='store_true', g_prof.add_argument('--no-profile-pic', action='store_true',
help='Do not download profile picture.') help='Do not download profile picture.')
g_post.add_argument('--slide', action='store',
help='Set what image/interval of a sidecar you want to download.')
g_post.add_argument('--no-pictures', action='store_true', g_post.add_argument('--no-pictures', action='store_true',
help='Do not download post pictures. Cannot be used together with --fast-update. ' help='Do not download post pictures. Cannot be used together with --fast-update. '
'Implies --no-video-thumbnails, does not imply --no-videos.') 'Implies --no-video-thumbnails, does not imply --no-videos.')
@@ -365,8 +377,11 @@ def main():
'connection fails, it can be manually skipped by hitting CTRL+C. Set this to 0 to retry ' 'connection fails, it can be manually skipped by hitting CTRL+C. Set this to 0 to retry '
'infinitely.') 'infinitely.')
g_how.add_argument('--commit-mode', action='store_true', help=SUPPRESS) g_how.add_argument('--commit-mode', action='store_true', help=SUPPRESS)
g_how.add_argument('--request-timeout', metavar='N', type=float, g_how.add_argument('--request-timeout', metavar='N', type=float, default=300.0,
help='seconds to wait before timing out a connection request') help='Seconds to wait before timing out a connection request. Defaults to 300.')
g_how.add_argument('--abort-on', type=http_status_code_list, metavar="STATUS_CODES",
help='Comma-separated list of HTTP status codes that cause Instaloader to abort, bypassing all '
'retry logic.')
g_misc = parser.add_argument_group('Miscellaneous Options') g_misc = parser.add_argument_group('Miscellaneous Options')
g_misc.add_argument('-q', '--quiet', action='store_true', g_misc.add_argument('-q', '--quiet', action='store_true',
@@ -424,7 +439,9 @@ def main():
max_connection_attempts=args.max_connection_attempts, max_connection_attempts=args.max_connection_attempts,
request_timeout=args.request_timeout, request_timeout=args.request_timeout,
resume_prefix=resume_prefix, resume_prefix=resume_prefix,
check_resume_bbd=not args.use_aged_resume_files) check_resume_bbd=not args.use_aged_resume_files,
slide=args.slide,
fatal_status_codes=args.abort_on)
_main(loader, _main(loader,
args.profile, args.profile,
username=args.login.lower() if args.login is not None else None, username=args.login.lower() if args.login is not None else None,
@@ -442,7 +459,7 @@ def main():
storyitem_filter_str=args.storyitem_filter) storyitem_filter_str=args.storyitem_filter)
loader.close() loader.close()
except InstaloaderException as err: except InstaloaderException as err:
raise SystemExit("Fatal error: %s" % err) raise SystemExit("Fatal error: %s" % err) from err
if __name__ == "__main__": if __name__ == "__main__":
+12
View File
@@ -64,3 +64,15 @@ class QueryReturnedNotFoundException(ConnectionException):
class TooManyRequestsException(ConnectionException): class TooManyRequestsException(ConnectionException):
pass pass
class AbortDownloadException(Exception):
"""
Exception that is not catched in the error catchers inside the download loop and so aborts the
download loop.
This exception is not a subclass of ``InstaloaderException``.
.. versionadded:: 4.7
"""
pass
+168 -35
View File
@@ -14,6 +14,7 @@ from hashlib import md5
from io import BytesIO from io import BytesIO
from pathlib import Path from pathlib import Path
from typing import Any, Callable, IO, Iterator, List, Optional, Set, Union, cast from typing import Any, Callable, IO, Iterator, List, Optional, Set, Union, cast
from urllib.parse import urlparse
import requests import requests
import urllib3 # type: ignore import urllib3 # type: ignore
@@ -22,7 +23,7 @@ from .exceptions import *
from .instaloadercontext import InstaloaderContext, RateController from .instaloadercontext import InstaloaderContext, RateController
from .nodeiterator import NodeIterator, resumable_iteration from .nodeiterator import NodeIterator, resumable_iteration
from .structures import (Hashtag, Highlight, JsonExportable, Post, PostLocation, Profile, Story, StoryItem, from .structures import (Hashtag, Highlight, JsonExportable, Post, PostLocation, Profile, Story, StoryItem,
load_structure_from_file, save_structure_to_file) load_structure_from_file, save_structure_to_file, PostSidecarNode)
def get_default_session_filename(username: str) -> str: def get_default_session_filename(username: str) -> str:
@@ -101,6 +102,8 @@ class _ArbitraryItemFormatter(string.Formatter):
def get_value(self, key, args, kwargs): def get_value(self, key, args, kwargs):
"""Override to substitute {ATTRIBUTE} by attributes of our _item.""" """Override to substitute {ATTRIBUTE} by attributes of our _item."""
if key == 'filename' and isinstance(self._item, (Post, StoryItem, PostSidecarNode)):
return "{filename}"
if hasattr(self._item, key): if hasattr(self._item, key):
return getattr(self._item, key) return getattr(self._item, key)
return super().get_value(key, args, kwargs) return super().get_value(key, args, kwargs)
@@ -157,6 +160,8 @@ class Instaloader:
:param rate_controller: Generator for a :class:`RateController` to override rate controlling behavior :param rate_controller: Generator for a :class:`RateController` to override rate controlling behavior
:param resume_prefix: :option:`--resume-prefix`, or None for :option:`--no-resume`. :param resume_prefix: :option:`--resume-prefix`, or None for :option:`--no-resume`.
:param check_resume_bbd: Whether to check the date of expiry of resume files and reject them if expired. :param check_resume_bbd: Whether to check the date of expiry of resume files and reject them if expired.
:param slide: :option:`--slide`
:param fatal_status_codes: :option:`--abort-on`
.. attribute:: context .. attribute:: context
@@ -172,20 +177,22 @@ class Instaloader:
download_pictures=True, download_pictures=True,
download_videos: bool = True, download_videos: bool = True,
download_video_thumbnails: bool = True, download_video_thumbnails: bool = True,
download_geotags: bool = True, download_geotags: bool = False,
download_comments: bool = True, download_comments: bool = False,
save_metadata: bool = True, save_metadata: bool = True,
compress_json: bool = True, compress_json: bool = True,
post_metadata_txt_pattern: str = None, post_metadata_txt_pattern: str = None,
storyitem_metadata_txt_pattern: str = None, storyitem_metadata_txt_pattern: str = None,
max_connection_attempts: int = 3, max_connection_attempts: int = 3,
request_timeout: Optional[float] = None, request_timeout: float = 300.0,
rate_controller: Optional[Callable[[InstaloaderContext], RateController]] = None, rate_controller: Optional[Callable[[InstaloaderContext], RateController]] = None,
resume_prefix: Optional[str] = "iterator", resume_prefix: Optional[str] = "iterator",
check_resume_bbd: bool = True): check_resume_bbd: bool = True,
slide: Optional[str] = None,
fatal_status_codes: Optional[List[int]] = None):
self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts, self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts,
request_timeout, rate_controller) request_timeout, rate_controller, fatal_status_codes)
# configuration parameters # configuration parameters
self.dirname_pattern = dirname_pattern or "{target}" self.dirname_pattern = dirname_pattern or "{target}"
@@ -204,6 +211,31 @@ class Instaloader:
self.resume_prefix = resume_prefix self.resume_prefix = resume_prefix
self.check_resume_bbd = check_resume_bbd self.check_resume_bbd = check_resume_bbd
self.slide = slide or ""
self.slide_start = 0
self.slide_end = -1
if self.slide != "":
splitted = self.slide.split('-')
if len(splitted) == 1:
if splitted[0] == 'last':
# download only last image of a sidecar
self.slide_start = -1
else:
if int(splitted[0]) > 0:
self.slide_start = self.slide_end = int(splitted[0])-1
else:
raise InvalidArgumentException("--slide parameter must be greater than 0.")
elif len(splitted) == 2:
if splitted[1] == 'last':
self.slide_start = int(splitted[0])-1
elif 0 < int(splitted[0]) < int(splitted[1]):
self.slide_start = int(splitted[0])-1
self.slide_end = int(splitted[1])-1
else:
raise InvalidArgumentException("Invalid data for --slide parameter.")
else:
raise InvalidArgumentException("Invalid data for --slide parameter.")
@contextmanager @contextmanager
def anonymous_copy(self): def anonymous_copy(self):
"""Yield an anonymous, otherwise equally-configured copy of an Instaloader instance; Then copy its error log.""" """Yield an anonymous, otherwise equally-configured copy of an Instaloader instance; Then copy its error log."""
@@ -225,7 +257,9 @@ class Instaloader:
max_connection_attempts=self.context.max_connection_attempts, max_connection_attempts=self.context.max_connection_attempts,
request_timeout=self.context.request_timeout, request_timeout=self.context.request_timeout,
resume_prefix=self.resume_prefix, resume_prefix=self.resume_prefix,
check_resume_bbd=self.check_resume_bbd) check_resume_bbd=self.check_resume_bbd,
slide=self.slide,
fatal_status_codes=self.context.fatal_status_codes)
yield new_loader yield new_loader
self.context.error_log.extend(new_loader.context.error_log) self.context.error_log.extend(new_loader.context.error_log)
new_loader.context.error_log = [] # avoid double-printing of errors new_loader.context.error_log = [] # avoid double-printing of errors
@@ -301,18 +335,47 @@ class Instaloader:
combined_answers.extend(y['answers']) combined_answers.extend(y['answers'])
unique_comments_list[-1]['answers'] = get_unique_comments(combined_answers) unique_comments_list[-1]['answers'] = get_unique_comments(combined_answers)
return unique_comments_list return unique_comments_list
def get_new_comments(new_comments, start):
for idx, comment in enumerate(new_comments, start=start+1):
if idx % 250 == 0:
self.context.log('{}'.format(idx), end='…', flush=True)
yield comment
def save_comments(extended_comments):
unique_comments = get_unique_comments(extended_comments, combine_answers=True)
answer_ids = set(int(answer['id']) for comment in unique_comments for answer in comment.get('answers', []))
with open(filename, 'w') as file:
file.write(json.dumps(list(filter(lambda t: int(t['id']) not in answer_ids, unique_comments)),
indent=4))
base_filename = filename
filename += '_comments.json' filename += '_comments.json'
try: try:
with open(filename) as fp: with open(filename) as fp:
comments = json.load(fp) comments = json.load(fp)
except (FileNotFoundError, json.decoder.JSONDecodeError): except (FileNotFoundError, json.decoder.JSONDecodeError):
comments = list() comments = list()
comments.extend(_postcomment_asdict(comment) for comment in post.get_comments())
comments_iterator = post.get_comments()
try:
with resumable_iteration(
context=self.context,
iterator=comments_iterator,
load=load_structure_from_file,
save=save_structure_to_file,
format_path=lambda magic: "{}_{}_{}.json.xz".format(base_filename, self.resume_prefix, magic),
check_bbd=self.check_resume_bbd,
enabled=self.resume_prefix is not None
) as (_is_resuming, start_index):
comments.extend(_postcomment_asdict(comment)
for comment in get_new_comments(comments_iterator, start_index))
except (KeyboardInterrupt, AbortDownloadException):
if comments:
save_comments(comments)
raise
if comments: if comments:
comments = get_unique_comments(comments, combine_answers=True) save_comments(comments)
answer_ids = set(int(answer['id']) for comment in comments for answer in comment.get('answers', []))
with open(filename, 'w') as file:
file.write(json.dumps(list(filter(lambda t: int(t['id']) not in answer_ids, comments)), indent=4))
self.context.log('comments', end=' ', flush=True) self.context.log('comments', end=' ', flush=True)
def save_caption(self, filename: str, mtime: datetime, caption: str) -> None: def save_caption(self, filename: str, mtime: datetime, caption: str) -> None:
@@ -492,7 +555,20 @@ class Instaloader:
.. versionadded:: 4.2""" .. versionadded:: 4.2"""
self.context.two_factor_login(two_factor_code) self.context.two_factor_login(two_factor_code)
def format_filename(self, item: Union[Post, StoryItem], target: Optional[Union[str, Path]] = None): @staticmethod
def __prepare_filename(filename_template: str, url: Callable[[], str]) -> str:
"""Replace filename token inside filename_template with url's filename and assure the directories exist.
.. versionadded:: 4.6"""
if "{filename}" in filename_template:
filename = filename_template.replace("{filename}",
os.path.splitext(os.path.basename(urlparse(url()).path))[0])
else:
filename = filename_template
os.makedirs(os.path.dirname(filename), exist_ok=True)
return filename
def format_filename(self, item: Union[Post, StoryItem, PostSidecarNode], target: Optional[Union[str, Path]] = None):
"""Format filename of a :class:`Post` or :class:`StoryItem` according to ``filename-pattern`` parameter. """Format filename of a :class:`Post` or :class:`StoryItem` according to ``filename-pattern`` parameter.
.. versionadded:: 4.1""" .. versionadded:: 4.1"""
@@ -507,32 +583,76 @@ class Instaloader:
:return: True if something was downloaded, False otherwise, i.e. file was already there :return: True if something was downloaded, False otherwise, i.e. file was already there
""" """
def _already_downloaded(path: str) -> bool:
if not os.path.isfile(path):
return False
else:
self.context.log(path + ' exists', end=' ', flush=True)
return True
def _all_already_downloaded(path_base, is_videos_enumerated) -> bool:
if '{filename}' in self.filename_pattern:
# full URL needed to evaluate actual filename, cannot determine at
# this point if all sidecar nodes were already downloaded.
return False
for idx, is_video in is_videos_enumerated:
if self.download_pictures and (not is_video or self.download_video_thumbnails):
if not _already_downloaded("{0}_{1}.jpg".format(path_base, idx)):
return False
if is_video and self.download_videos:
if not _already_downloaded("{0}_{1}.mp4".format(path_base, idx)):
return False
return True
dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target) dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target)
filename = os.path.join(dirname, self.format_filename(post, target=target)) filename_template = os.path.join(dirname, self.format_filename(post, target=target))
os.makedirs(os.path.dirname(filename), exist_ok=True) filename = self.__prepare_filename(filename_template, lambda: post.url)
# Download the image(s) / video thumbnail and videos within sidecars if desired # Download the image(s) / video thumbnail and videos within sidecars if desired
downloaded = True downloaded = True
if post.typename == 'GraphSidecar': if post.typename == 'GraphSidecar':
if self.download_pictures or self.download_videos: if self.download_pictures or self.download_videos:
for edge_number, sidecar_node in enumerate(post.get_sidecar_nodes(), start=1): if not _all_already_downloaded(
if self.download_pictures and (not sidecar_node.is_video or self.download_video_thumbnails): filename_template, enumerate(
# Download sidecar picture or video thumbnail (--no-pictures implies --no-video-thumbnails) (post.get_is_videos()[i]
downloaded &= self.download_pic(filename=filename, url=sidecar_node.display_url, for i in range(self.slide_start % post.mediacount, self.slide_end % post.mediacount + 1)),
mtime=post.date_local, filename_suffix=str(edge_number)) start=self.slide_start % post.mediacount + 1
if sidecar_node.is_video and self.download_videos: )
# Download sidecar video if desired ):
downloaded &= self.download_pic(filename=filename, url=sidecar_node.video_url, for edge_number, sidecar_node in enumerate(
mtime=post.date_local, filename_suffix=str(edge_number)) post.get_sidecar_nodes(self.slide_start, self.slide_end),
start=self.slide_start % post.mediacount + 1
):
suffix = str(edge_number) # type: Optional[str]
if '{filename}' in self.filename_pattern:
suffix = None
if self.download_pictures and (not sidecar_node.is_video or self.download_video_thumbnails):
# pylint:disable=cell-var-from-loop
sidecar_filename = self.__prepare_filename(filename_template,
lambda: sidecar_node.display_url)
# Download sidecar picture or video thumbnail (--no-pictures implies --no-video-thumbnails)
downloaded &= self.download_pic(filename=sidecar_filename, url=sidecar_node.display_url,
mtime=post.date_local, filename_suffix=suffix)
if sidecar_node.is_video and self.download_videos:
# pylint:disable=cell-var-from-loop
sidecar_filename = self.__prepare_filename(filename_template,
lambda: sidecar_node.video_url)
# Download sidecar video if desired
downloaded &= self.download_pic(filename=sidecar_filename, url=sidecar_node.video_url,
mtime=post.date_local, filename_suffix=suffix)
else:
downloaded = False
elif post.typename == 'GraphImage': elif post.typename == 'GraphImage':
# Download picture # Download picture
if self.download_pictures: if self.download_pictures:
downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local) downloaded = (not _already_downloaded(filename + ".jpg") and
self.download_pic(filename=filename, url=post.url, mtime=post.date_local))
elif post.typename == 'GraphVideo': elif post.typename == 'GraphVideo':
# Download video thumbnail (--no-pictures implies --no-video-thumbnails) # Download video thumbnail (--no-pictures implies --no-video-thumbnails)
if self.download_pictures and self.download_video_thumbnails: if self.download_pictures and self.download_video_thumbnails:
with self.context.error_catcher("Video thumbnail of {}".format(post)): with self.context.error_catcher("Video thumbnail of {}".format(post)):
downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local) downloaded = (not _already_downloaded(filename + ".jpg") and
self.download_pic(filename=filename, url=post.url, mtime=post.date_local))
else: else:
self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename)) self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename))
@@ -543,7 +663,8 @@ class Instaloader:
# Download video if desired # Download video if desired
if post.is_video and self.download_videos: if post.is_video and self.download_videos:
downloaded &= self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local) downloaded &= (not _already_downloaded(filename + ".mp4") and
self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local))
# Download geotags if desired # Download geotags if desired
if self.download_geotags and post.location: if self.download_geotags and post.location:
@@ -636,16 +757,25 @@ class Instaloader:
:return: True if something was downloaded, False otherwise, i.e. file was already there :return: True if something was downloaded, False otherwise, i.e. file was already there
""" """
def _already_downloaded(path: str) -> bool:
if not os.path.isfile(path):
return False
else:
self.context.log(path + ' exists', end=' ', flush=True)
return True
date_local = item.date_local date_local = item.date_local
dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target) dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target)
filename = os.path.join(dirname, self.format_filename(item, target=target)) filename_template = os.path.join(dirname, self.format_filename(item, target=target))
os.makedirs(os.path.dirname(filename), exist_ok=True) filename = self.__prepare_filename(filename_template, lambda: item.url)
downloaded = False downloaded = False
if not item.is_video or self.download_video_thumbnails is True: if not item.is_video or self.download_video_thumbnails is True:
url = item.url downloaded = (not _already_downloaded(filename + ".jpg") and
downloaded = self.download_pic(filename=filename, url=url, mtime=date_local) self.download_pic(filename=filename, url=item.url, mtime=date_local))
if item.is_video and self.download_videos is True: if item.is_video and self.download_videos is True:
downloaded |= self.download_pic(filename=filename, url=item.video_url, mtime=date_local) filename = self.__prepare_filename(filename_template, lambda: str(item.video_url))
downloaded |= (not _already_downloaded(filename + ".mp4") and
self.download_pic(filename=filename, url=item.video_url, mtime=date_local))
# Save caption if desired # Save caption if desired
metadata_string = _ArbitraryItemFormatter(item).format(self.storyitem_metadata_txt_pattern).strip() metadata_string = _ArbitraryItemFormatter(item).format(self.storyitem_metadata_txt_pattern).strip()
if metadata_string: if metadata_string:
@@ -1129,10 +1259,13 @@ class Instaloader:
self.save_metadata_json(json_filename, profile) self.save_metadata_json(json_filename, profile)
# Catch some errors # Catch some errors
if profile.is_private and (tagged or igtv or highlights or posts): if tagged or igtv or highlights or posts:
if not self.context.is_logged_in: if (not self.context.is_logged_in and
profile.is_private):
raise LoginRequiredException("--login=USERNAME required.") raise LoginRequiredException("--login=USERNAME required.")
if not profile.followed_by_viewer and self.context.username != profile.username: if (self.context.username != profile.username and
profile.is_private and
not profile.followed_by_viewer):
raise PrivateProfileNotFollowedException("Private but not followed.") raise PrivateProfileNotFollowedException("Private but not followed.")
# Download tagged, if requested # Download tagged, if requested
+72 -37
View File
@@ -25,16 +25,15 @@ def copy_session(session: requests.Session, request_timeout: Optional[float] = N
new = requests.Session() new = requests.Session()
new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies)) new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
new.headers = session.headers.copy() new.headers = session.headers.copy()
if request_timeout is not None: # Override default timeout behavior.
# Override default timeout behavior. # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427 new.request = partial(new.request, timeout=request_timeout) # type: ignore
new.request = partial(new.request, timeout=request_timeout) # type: ignore
return new return new
def default_user_agent() -> str: def default_user_agent() -> str:
return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \ return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \
'(KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36' '(KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36'
class InstaloaderContext: class InstaloaderContext:
@@ -53,8 +52,9 @@ class InstaloaderContext:
""" """
def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None, def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
max_connection_attempts: int = 3, request_timeout: Optional[float] = None, max_connection_attempts: int = 3, request_timeout: float = 300.0,
rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None): rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None,
fatal_status_codes: Optional[List[int]] = None):
self.user_agent = user_agent if user_agent is not None else default_user_agent() self.user_agent = user_agent if user_agent is not None else default_user_agent()
self.request_timeout = request_timeout self.request_timeout = request_timeout
@@ -75,6 +75,9 @@ class InstaloaderContext:
# Can be set to True for testing, disables supression of InstaloaderContext._error_catcher # Can be set to True for testing, disables supression of InstaloaderContext._error_catcher
self.raise_all_errors = False self.raise_all_errors = False
# HTTP status codes that should cause an AbortDownloadException
self.fatal_status_codes = fatal_status_codes or []
# Cache profile from id (mapping from id to Profile) # Cache profile from id (mapping from id to Profile)
self.profile_id_cache = dict() # type: Dict[int, Any] self.profile_id_cache = dict() # type: Dict[int, Any]
@@ -161,10 +164,9 @@ class InstaloaderContext:
'ig_vw': '1920', 'csrftoken': '', 'ig_vw': '1920', 'csrftoken': '',
's_network': '', 'ds_user_id': ''}) 's_network': '', 'ds_user_id': ''})
session.headers.update(self._default_http_header(empty_session_only=True)) session.headers.update(self._default_http_header(empty_session_only=True))
if self.request_timeout is not None: # Override default timeout behavior.
# Override default timeout behavior. # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427 session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
return session return session
def save_session_to_file(self, sessionfile): def save_session_to_file(self, sessionfile):
@@ -177,10 +179,9 @@ class InstaloaderContext:
session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile)) session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile))
session.headers.update(self._default_http_header()) session.headers.update(self._default_http_header())
session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']}) session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']})
if self.request_timeout is not None: # Override default timeout behavior.
# Override default timeout behavior. # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427 session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
self._session = session self._session = session
self.username = username self.username = username
@@ -206,10 +207,9 @@ class InstaloaderContext:
'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '', 'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '',
's_network': '', 'ds_user_id': ''}) 's_network': '', 'ds_user_id': ''})
session.headers.update(self._default_http_header()) session.headers.update(self._default_http_header())
if self.request_timeout is not None: # Override default timeout behavior.
# Override default timeout behavior. # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427 session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
session.get('https://www.instagram.com/web/__mid/') session.get('https://www.instagram.com/web/__mid/')
csrf_token = session.cookies.get_dict()['csrftoken'] csrf_token = session.cookies.get_dict()['csrftoken']
session.headers.update({'X-CSRFToken': csrf_token}) session.headers.update({'X-CSRFToken': csrf_token})
@@ -222,8 +222,10 @@ class InstaloaderContext:
data={'enc_password': enc_password, 'username': user}, allow_redirects=True) data={'enc_password': enc_password, 'username': user}, allow_redirects=True)
try: try:
resp_json = login.json() resp_json = login.json()
except json.decoder.JSONDecodeError: except json.decoder.JSONDecodeError as err:
raise ConnectionException("Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason)) raise ConnectionException(
"Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason)
) from err
if resp_json.get('two_factor_required'): if resp_json.get('two_factor_required'):
two_factor_session = copy_session(session, self.request_timeout) two_factor_session = copy_session(session, self.request_timeout)
two_factor_session.headers.update({'X-CSRFToken': csrf_token}) two_factor_session.headers.update({'X-CSRFToken': csrf_token})
@@ -292,7 +294,7 @@ class InstaloaderContext:
def do_sleep(self): def do_sleep(self):
"""Sleep a short time if self.sleep is set. Called before each request to instagram.com.""" """Sleep a short time if self.sleep is set. Called before each request to instagram.com."""
if self.sleep: if self.sleep:
time.sleep(min(random.expovariate(0.7), 5.0)) time.sleep(min(random.expovariate(0.6), 15.0))
def get_json(self, path: str, params: Dict[str, Any], host: str = 'www.instagram.com', def get_json(self, path: str, params: Dict[str, Any], host: str = 'www.instagram.com',
session: Optional[requests.Session] = None, _attempt=1) -> Dict[str, Any]: session: Optional[requests.Session] = None, _attempt=1) -> Dict[str, Any]:
@@ -320,6 +322,11 @@ class InstaloaderContext:
if is_other_query: if is_other_query:
self._rate_controller.wait_before_query('other') self._rate_controller.wait_before_query('other')
resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False) resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False)
if resp.status_code in self.fatal_status_codes:
redirect = " redirect to {}".format(resp.headers['location']) if 'location' in resp.headers else ""
raise AbortDownloadException("Query to https://{}/{} responded with \"{} {}\"{}".format(
host, path, resp.status_code, resp.reason, redirect
))
while resp.is_redirect: while resp.is_redirect:
redirect_url = resp.headers['location'] redirect_url = resp.headers['location']
self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url)) self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url))
@@ -473,7 +480,7 @@ class InstaloaderContext:
.. versionadded:: 4.2.1""" .. versionadded:: 4.2.1"""
with copy_session(self._session, self.request_timeout) as tempsession: with copy_session(self._session, self.request_timeout) as tempsession:
tempsession.headers['User-Agent'] = 'Instagram 123.1.0.26.115 (iPhone12,1; iOS 13_3; en_US; en-US; ' \ tempsession.headers['User-Agent'] = 'Instagram 146.0.0.27.125 (iPhone12,1; iOS 13_3; en_US; en-US; ' \
'scale=2.00; 1656x3584; 190542906)' 'scale=2.00; 1656x3584; 190542906)'
for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With']: for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With']:
tempsession.headers.pop(header, None) tempsession.headers.pop(header, None)
@@ -553,6 +560,7 @@ class RateController:
self._context = context self._context = context
self._query_timestamps = dict() # type: Dict[str, List[float]] self._query_timestamps = dict() # type: Dict[str, List[float]]
self._earliest_next_request_time = 0.0 self._earliest_next_request_time = 0.0
self._iphone_earliest_next_request_time = 0.0
def sleep(self, secs: float): def sleep(self, secs: float):
"""Wait given number of seconds.""" """Wait given number of seconds."""
@@ -563,7 +571,7 @@ class RateController:
def _dump_query_timestamps(self, current_time: float, failed_query_type: str): def _dump_query_timestamps(self, current_time: float, failed_query_type: str):
windows = [10, 11, 20, 22, 30, 60] windows = [10, 11, 20, 22, 30, 60]
self._context.error("Requests within last {} minutes grouped by type:" self._context.error("Number of requests within last {} minutes grouped by type:"
.format('/'.join(str(w) for w in windows)), .format('/'.join(str(w) for w in windows)),
repeat_at_end=False) repeat_at_end=False)
for query_type, times in self._query_timestamps.items(): for query_type, times in self._query_timestamps.items():
@@ -575,11 +583,15 @@ class RateController:
), repeat_at_end=False) ), repeat_at_end=False)
def count_per_sliding_window(self, query_type: str) -> int: def count_per_sliding_window(self, query_type: str) -> int:
"""Return how many requests can be done within the sliding window.""" """Return how many requests of the given type can be done within a sliding window of 11 minutes.
This is called by :meth:`RateController.query_waittime` and allows to simply customize wait times before queries
at query_type granularity. Consider overriding :meth:`RateController.query_waittime` directly if you need more
control."""
# Not static, to allow for the count_per_sliding_window to depend on context-inherent properties, such as # Not static, to allow for the count_per_sliding_window to depend on context-inherent properties, such as
# whether we are logged in. # whether we are logged in.
# pylint:disable=no-self-use # pylint:disable=no-self-use
return 75 if query_type in ['iphone', 'other'] else 200 return 75 if query_type == 'other' else 200
def _reqs_in_sliding_window(self, query_type: Optional[str], current_time: float, window: float) -> List[float]: def _reqs_in_sliding_window(self, query_type: Optional[str], current_time: float, window: float) -> List[float]:
if query_type is not None: if query_type is not None:
@@ -595,6 +607,7 @@ class RateController:
def query_waittime(self, query_type: str, current_time: float, untracked_queries: bool = False) -> float: def query_waittime(self, query_type: str, current_time: float, untracked_queries: bool = False) -> float:
"""Calculate time needed to wait before query can be executed.""" """Calculate time needed to wait before query can be executed."""
per_type_sliding_window = 660 per_type_sliding_window = 660
iphone_sliding_window = 1800
if query_type not in self._query_timestamps: if query_type not in self._query_timestamps:
self._query_timestamps[query_type] = [] self._query_timestamps[query_type] = []
self._query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60, self._query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60,
@@ -620,25 +633,43 @@ class RateController:
def untracked_next_request_time(): def untracked_next_request_time():
if untracked_queries: if untracked_queries:
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, per_type_sliding_window) if query_type == "iphone":
self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6 reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
return self._earliest_next_request_time iphone_sliding_window)
self._iphone_earliest_next_request_time = min(reqs_in_sliding_window) + iphone_sliding_window + 18
else:
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
per_type_sliding_window)
self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6
return max(self._iphone_earliest_next_request_time, self._earliest_next_request_time)
def iphone_next_request():
if query_type == "iphone":
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, iphone_sliding_window)
if len(reqs_in_sliding_window) >= 199:
return min(reqs_in_sliding_window) + iphone_sliding_window + 18
return 0.0
return max(0.0, return max(0.0,
max( max(
per_type_next_request_time(), per_type_next_request_time(),
gql_accumulated_next_request_time(), gql_accumulated_next_request_time(),
untracked_next_request_time(), untracked_next_request_time(),
iphone_next_request(),
) - current_time) ) - current_time)
def wait_before_query(self, query_type: str) -> None: def wait_before_query(self, query_type: str) -> None:
"""This method is called before a query to Instagram. It calls :meth:`RateController.sleep` to wait """This method is called before a query to Instagram.
until the request can be made."""
It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
:meth:`RateController.sleep` to wait until the request can be made."""
waittime = self.query_waittime(query_type, time.monotonic(), False) waittime = self.query_waittime(query_type, time.monotonic(), False)
assert waittime >= 0 assert waittime >= 0
if waittime > 15: if waittime > 15:
self._context.log("\nToo many queries in the last time. Need to wait {} seconds, until {:%H:%M}." formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
.format(round(waittime), datetime.now() + timedelta(seconds=waittime))) "{} minutes".format(round(waittime / 60)))
self._context.log("\nToo many queries in the last time. Need to wait {}, until {:%H:%M}."
.format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)))
if waittime > 0: if waittime > 0:
self.sleep(waittime) self.sleep(waittime)
if query_type not in self._query_timestamps: if query_type not in self._query_timestamps:
@@ -647,8 +678,10 @@ class RateController:
self._query_timestamps[query_type].append(time.monotonic()) self._query_timestamps[query_type].append(time.monotonic())
def handle_429(self, query_type: str) -> None: def handle_429(self, query_type: str) -> None:
"""This method is called to handle a 429 Too Many Requests response. It calls :meth:`RateController.sleep` to """This method is called to handle a 429 Too Many Requests response.
wait until we can repeat the same request."""
It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
:meth:`RateController.sleep` to wait until we can repeat the same request."""
current_time = time.monotonic() current_time = time.monotonic()
waittime = self.query_waittime(query_type, current_time, True) waittime = self.query_waittime(query_type, current_time, True)
assert waittime >= 0 assert waittime >= 0
@@ -658,8 +691,10 @@ class RateController:
"App while Instaloader is running.") "App while Instaloader is running.")
self._context.error(textwrap.fill(text_for_429), repeat_at_end=False) self._context.error(textwrap.fill(text_for_429), repeat_at_end=False)
if waittime > 1.5: if waittime > 1.5:
self._context.error("The request will be retried in {} seconds, at {:%H:%M}." formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
.format(round(waittime), datetime.now() + timedelta(seconds=waittime)), "{} minutes".format(round(waittime / 60)))
self._context.error("The request will be retried in {}, at {:%H:%M}."
.format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)),
repeat_at_end=False) repeat_at_end=False)
if waittime > 0: if waittime > 0:
self.sleep(waittime) self.sleep(waittime)
+10 -10
View File
@@ -5,9 +5,9 @@ import os
from contextlib import contextmanager from contextlib import contextmanager
from datetime import datetime, timedelta from datetime import datetime, timedelta
from lzma import LZMAError from lzma import LZMAError
from typing import Any, Callable, Dict, Iterator, NamedTuple, Optional, Tuple, TypeVar from typing import Any, Callable, Dict, Iterable, Iterator, NamedTuple, Optional, Tuple, TypeVar
from .exceptions import InvalidArgumentException, QueryReturnedBadRequestException from .exceptions import AbortDownloadException, InvalidArgumentException, QueryReturnedBadRequestException
from .instaloadercontext import InstaloaderContext from .instaloadercontext import InstaloaderContext
FrozenNodeIterator = NamedTuple('FrozenNodeIterator', FrozenNodeIterator = NamedTuple('FrozenNodeIterator',
@@ -150,11 +150,7 @@ class NodeIterator(Iterator[T]):
@property @property
def magic(self) -> str: def magic(self) -> str:
"""Magic string for easily identifying a matching iterator file for resuming (hash of some parameters).""" """Magic string for easily identifying a matching iterator file for resuming (hash of some parameters)."""
if 'blake2b' not in hashlib.algorithms_available: magic_hash = hashlib.blake2b(digest_size=6)
magic_hash = hashlib.new('sha224')
else:
# Use blake2b when possible, i.e. on Python >= 3.6.
magic_hash = hashlib.blake2b(digest_size=6) # type:ignore # pylint: disable=no-member
magic_hash.update(json.dumps( magic_hash.update(json.dumps(
[self._query_hash, self._query_variables, self._query_referer, self._context.username] [self._query_hash, self._query_variables, self._query_referer, self._context.username]
).encode()) ).encode())
@@ -204,14 +200,15 @@ class NodeIterator(Iterator[T]):
@contextmanager @contextmanager
def resumable_iteration(context: InstaloaderContext, def resumable_iteration(context: InstaloaderContext,
iterator: Iterator, iterator: Iterable,
load: Callable[[InstaloaderContext, str], Any], load: Callable[[InstaloaderContext, str], Any],
save: Callable[[FrozenNodeIterator, str], None], save: Callable[[FrozenNodeIterator, str], None],
format_path: Callable[[str], str], format_path: Callable[[str], str],
check_bbd: bool = True, check_bbd: bool = True,
enabled: bool = True) -> Iterator[Tuple[bool, int]]: enabled: bool = True) -> Iterator[Tuple[bool, int]]:
""" """
High-level context manager to handle a resumable iteration that can be interrupted with a KeyboardInterrupt. High-level context manager to handle a resumable iteration that can be interrupted
with a :class:`KeyboardInterrupt` or an :class:`AbortDownloadException`.
It can be used as follows to automatically load a previously-saved state into the iterator, save the iterator's It can be used as follows to automatically load a previously-saved state into the iterator, save the iterator's
state when interrupted, and delete the resume file upon completion:: state when interrupted, and delete the resume file upon completion::
@@ -239,6 +236,9 @@ def resumable_iteration(context: InstaloaderContext,
:param format_path: Returns the path to the resume file for the given magic. :param format_path: Returns the path to the resume file for the given magic.
:param check_bbd: Whether to check the best before date and reject an expired FrozenNodeIterator. :param check_bbd: Whether to check the best before date and reject an expired FrozenNodeIterator.
:param enabled: Set to False to disable all functionality and simply execute the inner body. :param enabled: Set to False to disable all functionality and simply execute the inner body.
.. versionchanged:: 4.7
Also interrupt on :class:`AbortDownloadException`.
""" """
if not enabled or not isinstance(iterator, NodeIterator): if not enabled or not isinstance(iterator, NodeIterator):
yield False, 0 yield False, 0
@@ -262,7 +262,7 @@ def resumable_iteration(context: InstaloaderContext,
context.error("Warning: Not resuming from {}: {}".format(resume_file_path, exc)) context.error("Warning: Not resuming from {}: {}".format(resume_file_path, exc))
try: try:
yield is_resuming, start_index yield is_resuming, start_index
except KeyboardInterrupt: except (KeyboardInterrupt, AbortDownloadException):
if os.path.dirname(resume_file_path): if os.path.dirname(resume_file_path):
os.makedirs(os.path.dirname(resume_file_path), exist_ok=True) os.makedirs(os.path.dirname(resume_file_path), exist_ok=True)
save(iterator.freeze(), resume_file_path) save(iterator.freeze(), resume_file_path)
View File
+94 -22
View File
@@ -4,7 +4,7 @@ import re
from base64 import b64decode, b64encode from base64 import b64decode, b64encode
from collections import namedtuple from collections import namedtuple
from datetime import datetime from datetime import datetime
from typing import Any, Dict, Iterator, List, Optional, Union from typing import Any, Dict, Iterable, Iterator, List, Optional, Union
from . import __version__ from . import __version__
from .exceptions import * from .exceptions import *
@@ -128,6 +128,14 @@ class Post:
"""The mediaid is a decimal representation of the media shortcode.""" """The mediaid is a decimal representation of the media shortcode."""
return int(self._node['id']) return int(self._node['id'])
@property
def title(self) -> Optional[str]:
"""Title of post"""
try:
return self._field('title')
except KeyError:
return None
def __repr__(self): def __repr__(self):
return '<Post {}>'.format(self.shortcode) return '<Post {}>'.format(self.shortcode)
@@ -252,26 +260,60 @@ class Post:
"""Type of post, GraphImage, GraphVideo or GraphSidecar""" """Type of post, GraphImage, GraphVideo or GraphSidecar"""
return self._field('__typename') return self._field('__typename')
def get_sidecar_nodes(self) -> Iterator[PostSidecarNode]: @property
"""Sidecar nodes of a Post with typename==GraphSidecar.""" def mediacount(self) -> int:
"""
The number of media in a sidecar Post, or 1 if the Post it not a sidecar.
.. versionadded:: 4.6
"""
if self.typename == 'GraphSidecar': if self.typename == 'GraphSidecar':
edges = self._field('edge_sidecar_to_children', 'edges') edges = self._field('edge_sidecar_to_children', 'edges')
if any(edge['node']['is_video'] for edge in edges): return len(edges)
return 1
def get_is_videos(self) -> List[bool]:
"""
Return a list containing the ``is_video`` property for each media in the post.
.. versionadded:: 4.7
"""
if self.typename == 'GraphSidecar':
edges = self._field('edge_sidecar_to_children', 'edges')
return [edge['node']['is_video'] for edge in edges]
return [self.is_video]
def get_sidecar_nodes(self, start=0, end=-1) -> Iterator[PostSidecarNode]:
"""
Sidecar nodes of a Post with typename==GraphSidecar.
.. versionchanged:: 4.6
Added parameters *start* and *end* to specify a slice of sidecar media.
"""
if self.typename == 'GraphSidecar':
edges = self._field('edge_sidecar_to_children', 'edges')
if end < 0:
end = len(edges)-1
if start < 0:
start = len(edges)-1
if any(edge['node']['is_video'] and 'video_url' not in edge['node'] for edge in edges[start:(end+1)]):
# video_url is only present in full metadata, issue #558. # video_url is only present in full metadata, issue #558.
edges = self._full_metadata['edge_sidecar_to_children']['edges'] edges = self._full_metadata['edge_sidecar_to_children']['edges']
for idx, edge in enumerate(edges): for idx, edge in enumerate(edges):
node = edge['node'] if start <= idx <= end:
is_video = node['is_video'] node = edge['node']
display_url = node['display_url'] is_video = node['is_video']
if not is_video and self._context.is_logged_in: display_url = node['display_url']
try: if not is_video and self._context.is_logged_in:
carousel_media = self._iphone_struct['carousel_media'] try:
orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url'] carousel_media = self._iphone_struct['carousel_media']
display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url) orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
except (InstaloaderException, KeyError, IndexError) as err: display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self)) except (InstaloaderException, KeyError, IndexError) as err:
yield PostSidecarNode(is_video=is_video, display_url=display_url, self._context.error('{} Unable to fetch high quality image version of {}.'.format(
video_url=node['video_url'] if is_video else None) err, self))
yield PostSidecarNode(is_video=is_video, display_url=display_url,
video_url=node['video_url'] if is_video else None)
@property @property
def caption(self) -> Optional[str]: def caption(self) -> Optional[str]:
@@ -330,6 +372,12 @@ class Post:
def video_url(self) -> Optional[str]: def video_url(self) -> Optional[str]:
"""URL of the video, or None.""" """URL of the video, or None."""
if self.is_video: if self.is_video:
if self._context.is_logged_in:
try:
url = self._iphone_struct['video_versions'][0]['url']
return url
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality video version of {}.'.format(err, self))
return self._field('video_url') return self._field('video_url')
return None return None
@@ -378,12 +426,15 @@ class Post:
except KeyError: except KeyError:
return self._field('edge_media_to_comment', 'count') return self._field('edge_media_to_comment', 'count')
def get_comments(self) -> Iterator[PostComment]: def get_comments(self) -> Iterable[PostComment]:
r"""Iterate over all comments of the post. r"""Iterate over all comments of the post.
Each comment is represented by a PostComment namedtuple with fields text (string), created_at (datetime), Each comment is represented by a PostComment namedtuple with fields text (string), created_at (datetime),
id (int), owner (:class:`Profile`) and answers (:class:`~typing.Iterator`\ [:class:`PostCommentAnswer`]) id (int), owner (:class:`Profile`) and answers (:class:`~typing.Iterator`\ [:class:`PostCommentAnswer`])
if available. if available.
.. versionchanged:: 4.7
Change return type to ``Iterable``.
""" """
def _postcommentanswer(node): def _postcommentanswer(node):
return PostCommentAnswer(id=int(node['id']), return PostCommentAnswer(id=int(node['id']),
@@ -418,16 +469,15 @@ class Post:
answers=_postcommentanswers(node)) answers=_postcommentanswers(node))
if self.comments == 0: if self.comments == 0:
# Avoid doing additional requests if there are no comments # Avoid doing additional requests if there are no comments
return return []
comment_edges = self._field('edge_media_to_comment', 'edges') comment_edges = self._field('edge_media_to_comment', 'edges')
answers_count = sum([edge['node'].get('edge_threaded_comments', {}).get('count', 0) for edge in comment_edges]) answers_count = sum([edge['node'].get('edge_threaded_comments', {}).get('count', 0) for edge in comment_edges])
if self.comments == len(comment_edges) + answers_count: if self.comments == len(comment_edges) + answers_count:
# If the Post's metadata already contains all parent comments, don't do GraphQL requests to obtain them # If the Post's metadata already contains all parent comments, don't do GraphQL requests to obtain them
yield from (_postcomment(comment['node']) for comment in comment_edges) return [_postcomment(comment['node']) for comment in comment_edges]
return return NodeIterator(
yield from NodeIterator(
self._context, self._context,
'97b41c52301f77ce508f55e66d17620e', '97b41c52301f77ce508f55e66d17620e',
lambda d: d['data']['shortcode_media']['edge_media_to_parent_comment'], lambda d: d['data']['shortcode_media']['edge_media_to_parent_comment'],
@@ -806,7 +856,7 @@ class Profile:
self._obtain_metadata() self._obtain_metadata()
return NodeIterator( return NodeIterator(
self._context, self._context,
'472f257a40c653c64c666ce877d59d2b', '003056d32c2554def87228bc3fd9668a',
lambda d: d['data']['user']['edge_owner_to_timeline_media'], lambda d: d['data']['user']['edge_owner_to_timeline_media'],
lambda n: Post(self._context, n, self), lambda n: Post(self._context, n, self),
{'id': self.userid}, {'id': self.userid},
@@ -934,11 +984,17 @@ class StoryItem:
self._context = context self._context = context
self._node = node self._node = node
self._owner_profile = owner_profile self._owner_profile = owner_profile
self._iphone_struct_ = None
if 'iphone_struct' in node:
# if loaded from JSON with load_structure_from_file()
self._iphone_struct_ = node['iphone_struct']
def _asdict(self): def _asdict(self):
node = self._node node = self._node
if self._owner_profile: if self._owner_profile:
node['owner'] = self._owner_profile._asdict() node['owner'] = self._owner_profile._asdict()
if self._iphone_struct_:
node['iphone_struct'] = self._iphone_struct_
return node return node
@property @property
@@ -963,6 +1019,15 @@ class StoryItem:
def __hash__(self) -> int: def __hash__(self) -> int:
return hash(self.mediaid) return hash(self.mediaid)
@property
def _iphone_struct(self) -> Dict[str, Any]:
if not self._context.is_logged_in:
raise LoginRequiredException("--login required to access iPhone media info endpoint.")
if not self._iphone_struct_:
data = self._context.get_iphone_json(path='api/v1/media/{}/info/'.format(self.mediaid), params={})
self._iphone_struct_ = data['items'][0]
return self._iphone_struct_
@property @property
def owner_profile(self) -> Profile: def owner_profile(self) -> Profile:
""":class:`Profile` instance of the story item's owner.""" """:class:`Profile` instance of the story item's owner."""
@@ -1014,6 +1079,13 @@ class StoryItem:
@property @property
def url(self) -> str: def url(self) -> str:
"""URL of the picture / video thumbnail of the StoryItem""" """URL of the picture / video thumbnail of the StoryItem"""
if self.typename == "GraphStoryImage" and self._context.is_logged_in:
try:
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
return url
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
return self._node['display_resources'][-1]['src'] return self._node['display_resources'][-1]['src']
@property @property
+6 -9
View File
@@ -3,7 +3,6 @@
import re import re
import sys import sys
import os import os
import platform
from setuptools import setup from setuptools import setup
@@ -19,14 +18,11 @@ def get_version():
raise SystemExit("Could not find version string.") raise SystemExit("Could not find version string.")
if sys.version_info < (3, 5): if sys.version_info < (3, 6):
sys.exit('Instaloader requires Python >= 3.5.') sys.exit('Instaloader requires Python >= 3.6.')
requirements = ['requests>=2.4'] requirements = ['requests>=2.4']
if platform.system() == 'Windows' and sys.version_info < (3, 6):
requirements.append('win_unicode_console')
keywords = (['instagram', 'instagram-scraper', 'instagram-client', 'instagram-feed', 'downloader', 'videos', 'photos', keywords = (['instagram', 'instagram-scraper', 'instagram-client', 'instagram-feed', 'downloader', 'videos', 'photos',
'pictures', 'instagram-user-photos', 'instagram-photos', 'instagram-metadata', 'instagram-downloader', 'pictures', 'instagram-user-photos', 'instagram-photos', 'instagram-metadata', 'instagram-downloader',
'instagram-stories']) 'instagram-stories'])
@@ -38,6 +34,7 @@ setup(
name='instaloader', name='instaloader',
version=get_version(), version=get_version(),
packages=['instaloader'], packages=['instaloader'],
package_data={'instaloader': ['py.typed']},
url='https://instaloader.github.io/', url='https://instaloader.github.io/',
license='MIT', license='MIT',
author='Alexander Graf, André Koch-Kramer', author='Alexander Graf, André Koch-Kramer',
@@ -46,9 +43,9 @@ setup(
'from Instagram.', 'from Instagram.',
long_description=open(os.path.join(SRC, 'README.rst')).read(), long_description=open(os.path.join(SRC, 'README.rst')).read(),
install_requires=requirements, install_requires=requirements,
python_requires='>=3.5', python_requires='>=3.6',
entry_points={'console_scripts': ['instaloader=instaloader.__main__:main']}, entry_points={'console_scripts': ['instaloader=instaloader.__main__:main']},
zip_safe=True, zip_safe=False,
keywords=keywords, keywords=keywords,
classifiers=[ classifiers=[
'Development Status :: 5 - Production/Stable', 'Development Status :: 5 - Production/Stable',
@@ -57,10 +54,10 @@ setup(
'Intended Audience :: Developers', 'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License', 'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent', 'Operating System :: OS Independent',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3 :: Only', 'Programming Language :: Python :: 3 :: Only',
'Topic :: Internet', 'Topic :: Internet',
'Topic :: Multimedia :: Graphics' 'Topic :: Multimedia :: Graphics'