Compare commits

...
Author SHA1 Message Date
Alexander Graf 2bf175a111 Release of Version 4.7.1 2021-03-27 20:47:28 +01:00
Alexander Graf 95d8534aed Fix SRCINFO depends metadata in AUR 2021-03-27 20:46:53 +01:00
Alexander GrafandAndré Koch-Kramer b583cd07e1 Get better-quality version of pictures if logged-in
Fixes #1065 (also see #630).

Co-Authored-By: André Koch-Kramer <koch-kramer@web.de>
2021-03-27 20:42:56 +01:00
Alexander Graf 52f258e89b Release of Version 4.7 2021-03-27 15:19:12 +01:00
Alexander Graf e9888e72fd Second Release Candidate for Version 4.7 2021-03-20 17:44:04 +01:00
Alexander Graf c72b2a8fc2 Default download_comments and download_geotags to False 2021-03-20 17:39:37 +01:00
Alexander Graf e2cec0a0bf Update dependencies; test on py3.9; drop python 3.5 support 2021-03-20 17:30:15 +01:00
Alexander Graf 08eaa7d474 always keep exception chains 2021-03-20 17:17:08 +01:00
Alexander Graf f70355e803 Update default user agent 2021-03-20 17:15:05 +01:00
Alexander Graf bba7d5131c First Release Candidate for Version 4.7 2021-03-13 11:24:53 +01:00
Alexander GrafandAndré Koch-Kramer f6731566cd Support resuming of downloading comments
Co-Authored-By: André Koch-Kramer <koch-kramer@web.de>
2021-03-13 11:17:56 +01:00
Alexander Graf cd13211603 Minor optimization in get_sidecar_nodes() 2021-02-24 16:35:08 +01:00
Alexander Graf 8635a576f0 Fix line-too-long pylint complaint 2021-02-24 16:25:56 +01:00
Alexander Graf 8f594a3a30 Merge branch 'master' into upcoming/v4.7 2021-02-24 16:10:27 +01:00
Alexander Graf 0f11ef1bdd Release of version 4.6.2 2021-02-24 16:01:33 +01:00
Alexander Graf a4bb353421 docs: Enhance description of --resume-prefix 2021-02-24 16:01:22 +01:00
Alexander Graf dfdd744804 Minor fixes to top-level README.md 2021-02-24 16:00:28 +01:00
Thomas 9fb112faa8 Fix issue 892 Save content using original filenames (PR 893) (#1025)
- When a post contained sidecars, the filename for caption, json etc. was changed. It was no longer the original filename like before.
- If the post contains sidecars, a local variable is used to build the filenames for the sidecar media.
2021-02-20 10:17:57 +01:00
Alexander Graf f85e49d521 Spare unnecessary full metadata fetch
This fetch was done for sidecar posts that contain a video when going
through a profile. The fetched information is already present with the
new profile query introduced in the last commit, making this full
metadata fetch query unnecessary. Instaloader now better evaluates if
that fetch must be done or not.

The fetch was also (eventually unnecessarily) made when accessing
get_sidecar_posts() on a Post that has been loaded with
load_structure_from_file().
2021-02-17 16:40:54 +01:00
Thomas 6c4da39372 Update the query_hash for post downloads (#1026)
- It's necessary because for the old query_hash IG isn't returning the full structure any more.
- So the old query_hash is replaced with the current one.
2021-02-17 16:07:18 +01:00
Alexander Graf a2d756b177 Flag --abort-on to abort on given status codes
Closes #920.
2021-02-13 19:23:14 +01:00
Alexander Graf e15d67c065 Change login hint from "Warning" to "Hint" 2021-02-13 19:12:13 +01:00
Alexander Graf 640066dd60 Merge branch 'master' into upcoming/v4.7 2021-01-30 17:07:21 +01:00
Alexander GrafandAndré Koch-Kramer 672b0b1840 Fix efficiency bug regarding re-downloading targets
Remove unnecessary queries when a file in a target does not have to be
re-downloaded.

This significantly speeds up the download in cases where the majority of media
is already present locally.

Co-Authored-By: André Koch-Kramer <koch-kramer@web.de>
2021-01-30 16:54:35 +01:00
Alexander Graf 828fa71ede Release of version 4.6.1 2021-01-25 16:04:30 +01:00
Alexander Graf 4fd57f5598 Adjust user agent string and sleep behavior 2021-01-25 15:53:49 +01:00
Alexander Graf c5b686a213 Update RateController to current observations 2021-01-25 15:46:05 +01:00
43c521afa9 Add method to get title of post (#976)
Co-authored-by: Tejas Chauhan <tejas.chauhan@kevit.io>
Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
2021-01-16 18:50:27 +01:00
20 changed files with 525 additions and 249 deletions
+3 -2
View File
@@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8]
python-version: [3.8, 3.9]
steps:
- name: Checkout Instaloader Repository
uses: actions/checkout@v2
@@ -20,11 +20,12 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install Dependencies
run: |
python -m pip install pipenv==2020.6.2
python -m pip install pipenv==2020.11.15
pipenv sync --dev
- name: PyLint
run: pipenv run pylint instaloader
- name: MyPy
run: pipenv run mypy -m instaloader
- name: Build Documentation
if: matrix.python-version == '3.8'
run: pipenv run make -C docs html SPHINXOPTS="-W -n"
+1 -1
View File
@@ -12,7 +12,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.8
python-version: 3.9
architecture: x64
- name: Get the tagged version
id: get_version
-1
View File
@@ -1,7 +1,6 @@
dist: xenial
language: python
python:
- "3.5"
- "3.6"
- "3.7"
install:
+1
View File
@@ -11,6 +11,7 @@ pyinstaller = "*"
pefile = "*"
pywin32-ctypes = "*"
psutil = "*"
typing-extensions = "*"
[packages]
requests = "*"
Generated
+239 -141
View File
@@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
"sha256": "58caf1be034bed9aa7ebb8c78e75ffa7d7abe820d014b6398c444a5db566b306"
"sha256": "9704331c2f9e76af9e5ee386bc359a46ee66d1e2f93aa880d9d148da7ba0e6f5"
},
"pipfile-spec": 6,
"requires": {},
@@ -16,39 +16,42 @@
"default": {
"certifi": {
"hashes": [
"sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3",
"sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41"
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
],
"version": "==2020.6.20"
"version": "==2020.12.5"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
],
"version": "==3.0.4"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==4.0.0"
},
"idna": {
"hashes": [
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.10"
},
"requests": {
"hashes": [
"sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b",
"sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898"
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
],
"index": "pypi",
"version": "==2.24.0"
"version": "==2.25.1"
},
"urllib3": {
"hashes": [
"sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a",
"sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461"
"sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df",
"sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937"
],
"version": "==1.25.10"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.4"
}
},
"develop": {
@@ -68,43 +71,48 @@
},
"astroid": {
"hashes": [
"sha256:2f4078c2a41bf377eea06d71c9d2ba4eb8f6b1af2135bec27bbbb7d8f12bb703",
"sha256:bc58d83eb610252fd8de6363e39d4f1d0619c894b0ed24603b881c02e64c7386"
"sha256:21d735aab248253531bb0f1e1e6d068f0ee23533e18ae8a6171ff892b98297cf",
"sha256:cfc35498ee64017be059ceffab0a25bedf7548ab76f2bea691c5565896e7128d"
],
"version": "==2.4.2"
"markers": "python_version >= '3.6'",
"version": "==2.5.1"
},
"babel": {
"hashes": [
"sha256:1aac2ae2d0d8ea368fa90906567f5c08463d98ade155c0c4bfedd6a0f7160e38",
"sha256:d670ea0b10f8b723672d3a6abeb87b565b244da220d76b4dba1b66269ec152d4"
"sha256:9d35c22fcc79893c3ecc85ac4a56cde1ecf3f19c540bba0922308a6c06ca6fa5",
"sha256:da031ab54472314f210b0adcff1588ee5d1d1d0ba4dbd07b94dba82bde791e05"
],
"version": "==2.8.0"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.9.0"
},
"certifi": {
"hashes": [
"sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3",
"sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41"
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830"
],
"version": "==2020.6.20"
"version": "==2020.12.5"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5"
],
"version": "==3.0.4"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==4.0.0"
},
"docutils": {
"hashes": [
"sha256:0c5b78adfbf7762415433f5515cd5c9e762339e23369dbe8000d84a4bf4ab3af",
"sha256:c2de3a60e9e7d07be26b7f2b00ca0309c207e06c100f9cc2a94931fc75a478fc"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==0.16"
},
"future": {
"hashes": [
"sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d"
],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.18.2"
},
"idna": {
@@ -112,6 +120,7 @@
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.10"
},
"imagesize": {
@@ -119,47 +128,62 @@
"sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1",
"sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.2.0"
},
"importlib-metadata": {
"hashes": [
"sha256:742add720a20d0467df2f444ae41704000f50e1234f46174b51f9c6031a1bd71",
"sha256:b74159469b464a99cb8cc3e21973e4d96e05d3024d337313fedb618a6e86e6f4"
],
"markers": "python_version < '3.8'",
"version": "==3.7.3"
},
"isort": {
"hashes": [
"sha256:54da7e92468955c4fceacd0c86bd0ec997b0e1ee80d97f67c35a78b719dccab1",
"sha256:6e811fcb295968434526407adb8796944f1988c5b65e8139058f2014cbe100fd"
"sha256:c729845434366216d320e936b8ad6f9d681aab72dc7cbc2d51bedc3582f3ad1e",
"sha256:fff4f0c04e1825522ce6949973e83110a6e907750cd92d128b0d14aaaadbffdc"
],
"version": "==4.3.21"
"markers": "python_version >= '3.6' and python_version < '4.0'",
"version": "==5.7.0"
},
"jinja2": {
"hashes": [
"sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
"sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
"sha256:03e47ad063331dd6a3f04a43eddca8a966a26ba0c5b7207a9a9e4e08f1b29419",
"sha256:a6d58433de0ae800347cab1fa3043cebbabe8baa9d29e668f1c768cb87a333c6"
],
"version": "==2.11.2"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==2.11.3"
},
"lazy-object-proxy": {
"hashes": [
"sha256:0c4b206227a8097f05c4dbdd323c50edf81f15db3b8dc064d08c62d37e1a504d",
"sha256:194d092e6f246b906e8f70884e620e459fc54db3259e60cf69a4d66c3fda3449",
"sha256:1be7e4c9f96948003609aa6c974ae59830a6baecc5376c25c92d7d697e684c08",
"sha256:4677f594e474c91da97f489fea5b7daa17b5517190899cf213697e48d3902f5a",
"sha256:48dab84ebd4831077b150572aec802f303117c8cc5c871e182447281ebf3ac50",
"sha256:5541cada25cd173702dbd99f8e22434105456314462326f06dba3e180f203dfd",
"sha256:59f79fef100b09564bc2df42ea2d8d21a64fdcda64979c0fa3db7bdaabaf6239",
"sha256:8d859b89baf8ef7f8bc6b00aa20316483d67f0b1cbf422f5b4dc56701c8f2ffb",
"sha256:9254f4358b9b541e3441b007a0ea0764b9d056afdeafc1a5569eee1cc6c1b9ea",
"sha256:9651375199045a358eb6741df3e02a651e0330be090b3bc79f6d0de31a80ec3e",
"sha256:97bb5884f6f1cdce0099f86b907aa41c970c3c672ac8b9c8352789e103cf3156",
"sha256:9b15f3f4c0f35727d3a0fba4b770b3c4ebbb1fa907dbcc046a1d2799f3edd142",
"sha256:a2238e9d1bb71a56cd710611a1614d1194dc10a175c1e08d75e1a7bcc250d442",
"sha256:a6ae12d08c0bf9909ce12385803a543bfe99b95fe01e752536a60af2b7797c62",
"sha256:ca0a928a3ddbc5725be2dd1cf895ec0a254798915fb3a36af0964a0a4149e3db",
"sha256:cb2c7c57005a6804ab66f106ceb8482da55f5314b7fcb06551db1edae4ad1531",
"sha256:d74bb8693bf9cf75ac3b47a54d716bbb1a92648d5f781fc799347cfc95952383",
"sha256:d945239a5639b3ff35b70a88c5f2f491913eb94871780ebfabb2568bd58afc5a",
"sha256:eba7011090323c1dadf18b3b689845fd96a61ba0a1dfbd7f24b921398affc357",
"sha256:efa1909120ce98bbb3777e8b6f92237f5d5c8ea6758efea36a473e1d38f7d3e4",
"sha256:f3900e8a5de27447acbf900b4750b0ddfd7ec1ea7fbaf11dfa911141bc522af0"
"sha256:1d33d6f789697f401b75ce08e73b1de567b947740f768376631079290118ad39",
"sha256:2f2de8f8ac0be3e40d17730e0600619d35c78c13a099ea91ef7fb4ad944ce694",
"sha256:3782931963dc89e0e9a0ae4348b44762e868ea280e4f8c233b537852a8996ab9",
"sha256:37d9c34b96cca6787fe014aeb651217944a967a5b165e2cacb6b858d2997ab84",
"sha256:38c3865bd220bd983fcaa9aa11462619e84a71233bafd9c880f7b1cb753ca7fa",
"sha256:429c4d1862f3fc37cd56304d880f2eae5bd0da83bdef889f3bd66458aac49128",
"sha256:522b7c94b524389f4a4094c4bf04c2b02228454ddd17c1a9b2801fac1d754871",
"sha256:57fb5c5504ddd45ed420b5b6461a78f58cbb0c1b0cbd9cd5a43ad30a4a3ee4d0",
"sha256:5944a9b95e97de1980c65f03b79b356f30a43de48682b8bdd90aa5089f0ec1f4",
"sha256:6f4e5e68b7af950ed7fdb594b3f19a0014a3ace0fedb86acb896e140ffb24302",
"sha256:71a1ef23f22fa8437974b2d60fedb947c99a957ad625f83f43fd3de70f77f458",
"sha256:8a44e9901c0555f95ac401377032f6e6af66d8fc1fbfad77a7a8b1a826e0b93c",
"sha256:b6577f15d5516d7d209c1a8cde23062c0f10625f19e8dc9fb59268859778d7d7",
"sha256:c8fe2d6ff0ff583784039d0255ea7da076efd08507f2be6f68583b0da32e3afb",
"sha256:cadfa2c2cf54d35d13dc8d231253b7985b97d629ab9ca6e7d672c35539d38163",
"sha256:cd1bdace1a8762534e9a36c073cd54e97d517a17d69a17985961265be6d22847",
"sha256:ddbdcd10eb999d7ab292677f588b658372aadb9a52790f82484a37127a390108",
"sha256:e7273c64bccfd9310e9601b8f4511d84730239516bada26a0c9846c9697617ef",
"sha256:e7428977763150b4cf83255625a80a23dfdc94d43be7791ce90799d446b4e26f",
"sha256:e960e8be509e8d6d618300a6c189555c24efde63e85acaf0b14b2cd1ac743315",
"sha256:ecb5dd5990cec6e7f5c9c1124a37cb2c710c6d69b0c1a5c4aa4b35eba0ada068",
"sha256:ef3f5e288aa57b73b034ce9c1f1ac753d968f9069cd0742d1d69c698a0167166",
"sha256:fa5b2dee0e231fa4ad117be114251bdfe6afe39213bd629d43deb117b6a6c40a",
"sha256:fa7fb7973c622b9e725bee1db569d2c2ee64d2f9a089201c5e8185d482c7352d"
],
"version": "==1.4.3"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==1.5.2"
},
"markupsafe": {
"hashes": [
@@ -168,8 +192,12 @@
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
"sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
"sha256:195d7d2c4fbb0ee8139a6cf67194f3973a6b3042d742ebe0a9ed36d8b6f0c07f",
"sha256:22c178a091fc6630d0d045bdb5992d2dfe14e3259760e713c490da5323866c39",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
"sha256:2beec1e0de6924ea551859edb9e7679da6e4870d32cb766240ce17e0a0ba2014",
"sha256:3b8a6499709d29c2e2399569d96719a1b21dcd94410a586a18526b143ec8470f",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
@@ -178,25 +206,41 @@
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
"sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
"sha256:6f1e273a344928347c1290119b493a1f0303c52f5a5eae5f16d74f48c15d4a85",
"sha256:6fffc775d90dcc9aed1b89219549b329a9250d918fd0b8fa8d93d154918422e1",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
"sha256:7fed13866cf14bba33e7176717346713881f56d9d2bcebab207f7a036f41b850",
"sha256:84dee80c15f1b560d55bcfe6d47b27d070b4681c699c572af2e3c7cc90a3b8e0",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
"sha256:98bae9582248d6cf62321dcb52aaf5d9adf0bad3b40582925ef7c7f0ed85fceb",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
"sha256:a6a744282b7718a2a62d2ed9d993cad6f5f585605ad352c11de459f4108df0a1",
"sha256:acf08ac40292838b3cbbb06cfe9b2cb9ec78fce8baca31ddb87aaac2e2dc3bc2",
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b1dba4527182c95a0db8b6060cc98ac49b9e2f5e64320e2b56e47cb2831978c7",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:b7d644ddb4dbd407d31ffb699f1d140bc35478da613b441c582aeb7c43838dd8",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:bf5aa3cbcfdf57fa2ee9cd1822c862ef23037f5c832ad09cfea57fa846dec193",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:caabedc8323f1e93231b52fc32bdcde6db817623d33e100708d9a68e1f53b26b",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
"sha256:d53bc011414228441014aa71dbec320c66468c1030aae3a6e29778a3382d96e5",
"sha256:d73a845f227b0bfe8a7455ee623525ee656a9e2e749e4742706d80a6065d5e2c",
"sha256:d9be0ba6c527163cbed5e0857c451fcd092ce83947944d6c14bc95441203f032",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be",
"sha256:feb7b34d6325451ef96bc0e36e1a6c0c1c64bc1fbec4b854f4529e51887b1621"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.1.1"
},
"mccabe": {
@@ -208,23 +252,31 @@
},
"mypy": {
"hashes": [
"sha256:2c6cde8aa3426c1682d35190b59b71f661237d74b053822ea3d748e2c9578a7c",
"sha256:3fdda71c067d3ddfb21da4b80e2686b71e9e5c72cca65fa216d207a358827f86",
"sha256:5dd13ff1f2a97f94540fd37a49e5d255950ebcdf446fb597463a40d0df3fac8b",
"sha256:6731603dfe0ce4352c555c6284c6db0dc935b685e9ce2e4cf220abe1e14386fd",
"sha256:6bb93479caa6619d21d6e7160c552c1193f6952f0668cdda2f851156e85186fc",
"sha256:81c7908b94239c4010e16642c9102bfc958ab14e36048fa77d0be3289dda76ea",
"sha256:9c7a9a7ceb2871ba4bac1cf7217a7dd9ccd44c27c2950edbc6dc08530f32ad4e",
"sha256:a4a2cbcfc4cbf45cd126f531dedda8485671545b43107ded25ce952aac6fb308",
"sha256:b7fbfabdbcc78c4f6fc4712544b9b0d6bf171069c6e0e3cb82440dd10ced3406",
"sha256:c05b9e4fb1d8a41d41dec8786c94f3b95d3c5f528298d769eb8e73d293abc48d",
"sha256:d7df6eddb6054d21ca4d3c6249cae5578cb4602951fd2b6ee2f5510ffb098707",
"sha256:e0b61738ab504e656d1fe4ff0c0601387a5489ca122d55390ade31f9ca0e252d",
"sha256:eff7d4a85e9eea55afa34888dfeaccde99e7520b51f867ac28a48492c0b1130c",
"sha256:f05644db6779387ccdb468cc47a44b4356fc2ffa9287135d05b70a98dc83b89a"
"sha256:0d0a87c0e7e3a9becdfbe936c981d32e5ee0ccda3e0f07e1ef2c3d1a817cf73e",
"sha256:25adde9b862f8f9aac9d2d11971f226bd4c8fbaa89fb76bdadb267ef22d10064",
"sha256:28fb5479c494b1bab244620685e2eb3c3f988d71fd5d64cc753195e8ed53df7c",
"sha256:2f9b3407c58347a452fc0736861593e105139b905cca7d097e413453a1d650b4",
"sha256:33f159443db0829d16f0a8d83d94df3109bb6dd801975fe86bacb9bf71628e97",
"sha256:3f2aca7f68580dc2508289c729bd49ee929a436208d2b2b6aab15745a70a57df",
"sha256:499c798053cdebcaa916eef8cd733e5584b5909f789de856b482cd7d069bdad8",
"sha256:4eec37370483331d13514c3f55f446fc5248d6373e7029a29ecb7b7494851e7a",
"sha256:552a815579aa1e995f39fd05dde6cd378e191b063f031f2acfe73ce9fb7f9e56",
"sha256:5873888fff1c7cf5b71efbe80e0e73153fe9212fafdf8e44adfe4c20ec9f82d7",
"sha256:61a3d5b97955422964be6b3baf05ff2ce7f26f52c85dd88db11d5e03e146a3a6",
"sha256:674e822aa665b9fd75130c6c5f5ed9564a38c6cea6a6432ce47eafb68ee578c5",
"sha256:7ce3175801d0ae5fdfa79b4f0cfed08807af4d075b402b7e294e6aa72af9aa2a",
"sha256:9743c91088d396c1a5a3c9978354b61b0382b4e3c440ce83cf77994a43e8c521",
"sha256:9f94aac67a2045ec719ffe6111df543bac7874cee01f41928f6969756e030564",
"sha256:a26f8ec704e5a7423c8824d425086705e381b4f1dfdef6e3a1edab7ba174ec49",
"sha256:abf7e0c3cf117c44d9285cc6128856106183938c68fd4944763003decdcfeb66",
"sha256:b09669bcda124e83708f34a94606e01b614fa71931d356c1f1a5297ba11f110a",
"sha256:cd07039aa5df222037005b08fbbfd69b3ab0b0bd7a07d7906de75ae52c4e3119",
"sha256:d23e0ea196702d918b60c8288561e722bf437d82cb7ef2edcd98cfa38905d506",
"sha256:d65cc1df038ef55a99e617431f0553cd77763869eebdf9042403e16089fe746c",
"sha256:d7da2e1d5f558c37d6e8c1246f1aec1e7349e4913d8fb3cb289a35de573fe2eb"
],
"index": "pypi",
"version": "==0.782"
"version": "==0.812"
},
"mypy-extensions": {
"hashes": [
@@ -235,10 +287,11 @@
},
"packaging": {
"hashes": [
"sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8",
"sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181"
"sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
"sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
],
"version": "==20.4"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==20.9"
},
"pefile": {
"hashes": [
@@ -249,56 +302,82 @@
},
"psutil": {
"hashes": [
"sha256:0ee3c36428f160d2d8fce3c583a0353e848abb7de9732c50cf3356dd49ad63f8",
"sha256:10512b46c95b02842c225f58fa00385c08fa00c68bac7da2d9a58ebe2c517498",
"sha256:4080869ed93cce662905b029a1770fe89c98787e543fa7347f075ade761b19d6",
"sha256:5e9d0f26d4194479a13d5f4b3798260c20cecf9ac9a461e718eb59ea520a360c",
"sha256:66c18ca7680a31bf16ee22b1d21b6397869dda8059dbdb57d9f27efa6615f195",
"sha256:68d36986ded5dac7c2dcd42f2682af1db80d4bce3faa126a6145c1637e1b559f",
"sha256:90990af1c3c67195c44c9a889184f84f5b2320dce3ee3acbd054e3ba0b4a7beb",
"sha256:a5b120bb3c0c71dfe27551f9da2f3209a8257a178ed6c628a819037a8df487f1",
"sha256:d8a82162f23c53b8525cf5f14a355f5d1eea86fa8edde27287dd3a98399e4fdf",
"sha256:f2018461733b23f308c298653c8903d32aaad7873d25e1d228765e91ae42c3f2",
"sha256:ff1977ba1a5f71f89166d5145c3da1cea89a0fdb044075a12c720ee9123ec818"
"sha256:0066a82f7b1b37d334e68697faba68e5ad5e858279fd6351c8ca6024e8d6ba64",
"sha256:02b8292609b1f7fcb34173b25e48d0da8667bc85f81d7476584d889c6e0f2131",
"sha256:0ae6f386d8d297177fd288be6e8d1afc05966878704dad9847719650e44fc49c",
"sha256:0c9ccb99ab76025f2f0bbecf341d4656e9c1351db8cc8a03ccd62e318ab4b5c6",
"sha256:0dd4465a039d343925cdc29023bb6960ccf4e74a65ad53e768403746a9207023",
"sha256:12d844996d6c2b1d3881cfa6fa201fd635971869a9da945cf6756105af73d2df",
"sha256:1bff0d07e76114ec24ee32e7f7f8d0c4b0514b3fae93e3d2aaafd65d22502394",
"sha256:245b5509968ac0bd179287d91210cd3f37add77dad385ef238b275bad35fa1c4",
"sha256:28ff7c95293ae74bf1ca1a79e8805fcde005c18a122ca983abf676ea3466362b",
"sha256:36b3b6c9e2a34b7d7fbae330a85bf72c30b1c827a4366a07443fc4b6270449e2",
"sha256:52de075468cd394ac98c66f9ca33b2f54ae1d9bff1ef6b67a212ee8f639ec06d",
"sha256:5da29e394bdedd9144c7331192e20c1f79283fb03b06e6abd3a8ae45ffecee65",
"sha256:61f05864b42fedc0771d6d8e49c35f07efd209ade09a5afe6a5059e7bb7bf83d",
"sha256:6223d07a1ae93f86451d0198a0c361032c4c93ebd4bf6d25e2fb3edfad9571ef",
"sha256:6323d5d845c2785efb20aded4726636546b26d3b577aded22492908f7c1bdda7",
"sha256:6ffe81843131ee0ffa02c317186ed1e759a145267d54fdef1bc4ea5f5931ab60",
"sha256:74f2d0be88db96ada78756cb3a3e1b107ce8ab79f65aa885f76d7664e56928f6",
"sha256:74fb2557d1430fff18ff0d72613c5ca30c45cdbfcddd6a5773e9fc1fe9364be8",
"sha256:90d4091c2d30ddd0a03e0b97e6a33a48628469b99585e2ad6bf21f17423b112b",
"sha256:90f31c34d25b1b3ed6c40cdd34ff122b1887a825297c017e4cbd6796dd8b672d",
"sha256:99de3e8739258b3c3e8669cb9757c9a861b2a25ad0955f8e53ac662d66de61ac",
"sha256:c6a5fd10ce6b6344e616cf01cc5b849fa8103fbb5ba507b6b2dee4c11e84c935",
"sha256:ce8b867423291cb65cfc6d9c4955ee9bfc1e21fe03bb50e177f2b957f1c2469d",
"sha256:d225cd8319aa1d3c85bf195c4e07d17d3cd68636b8fc97e6cf198f782f99af28",
"sha256:ea313bb02e5e25224e518e4352af4bf5e062755160f77e4b1767dd5ccb65f876",
"sha256:ea372bcc129394485824ae3e3ddabe67dc0b118d262c568b4d2602a7070afdb0",
"sha256:f4634b033faf0d968bb9220dd1c793b897ab7f1189956e1aa9eae752527127d3",
"sha256:fcc01e900c1d7bee2a37e5d6e4f9194760a93597c97fee89c4ae51701de03563"
],
"index": "pypi",
"version": "==5.7.2"
"version": "==5.8.0"
},
"pygments": {
"hashes": [
"sha256:647344a061c249a3b74e230c739f434d7ea4d8b1d5f3721bc0f3558049b38f44",
"sha256:ff7a40b4860b727ab48fad6360eb351cc1b33cbf9b15a0f689ca5353e9463324"
"sha256:2656e1a6edcdabf4275f9a3640db59fd5de107d88e8663c5d4e9a0fa62f77f94",
"sha256:534ef71d539ae97d4c3a4cf7d6f110f214b0e687e92f9cb9d2a3b0d3101289c8"
],
"version": "==2.6.1"
"markers": "python_version >= '3.5'",
"version": "==2.8.1"
},
"pyinstaller": {
"hashes": [
"sha256:3730fa80d088f8bb7084d32480eb87cbb4ddb64123363763cf8f2a1378c1c4b7"
"sha256:f5c0eeb2aa663cce9a5404292c0195011fa500a6501c873a466b2e8cad3c950c"
],
"index": "pypi",
"version": "==3.6"
"version": "==4.2"
},
"pyinstaller-hooks-contrib": {
"hashes": [
"sha256:27558072021857d89524c42136feaa2ffe4f003f1bdf0278f9b24f6902c1759c",
"sha256:892310e6363655838485ee748bf1c5e5cade7963686d9af8650ee218a3e0b031"
],
"version": "==2021.1"
},
"pylint": {
"hashes": [
"sha256:7dd78437f2d8d019717dbf287772d0b2dbdfd13fc016aa7faa08d67bccc46adc",
"sha256:d0ece7d223fe422088b0e8f13fa0a1e8eb745ebffcb8ed53d3e95394b6101a1c"
"sha256:0e21d3b80b96740909d77206d741aa3ce0b06b41be375d92e1f3244a274c1f8a",
"sha256:d09b0b07ba06bcdff463958f53f23df25e740ecd81895f7d2699ec04bbd8dc3b"
],
"index": "pypi",
"version": "==2.5.3"
"version": "==2.7.2"
},
"pyparsing": {
"hashes": [
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.4.7"
},
"pytz": {
"hashes": [
"sha256:a494d53b6d39c3c6e44c3bec237336e14305e4f29bbf800b599253057fbb79ed",
"sha256:c35965d010ce31b23eeb663ed3cc8c906275d6be1a34393a1d73a41febf4a048"
"sha256:83a4a90894bf38e243cf052c8b58f381bfe9a7a483f6a9cab140bc7f702ac4da",
"sha256:eb10ce3e7736052ed3623d49975ce333bcd712c7bb19a58b9e2089d4057d0798"
],
"version": "==2020.1"
"version": "==2021.1"
},
"pywin32-ctypes": {
"hashes": [
@@ -310,25 +389,18 @@
},
"requests": {
"hashes": [
"sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b",
"sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898"
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e"
],
"index": "pypi",
"version": "==2.24.0"
},
"six": {
"hashes": [
"sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259",
"sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"
],
"version": "==1.15.0"
"version": "==2.25.1"
},
"snowballstemmer": {
"hashes": [
"sha256:209f257d7533fdb3cb73bdbd24f436239ca3b2fa67d56f6ff88e86be08cc5ef0",
"sha256:df3bac3df4c2c01363f3dd2cfa78cce2840a79b9f1c2d2de9ce8d31683992f52"
"sha256:b51b447bea85f9968c13b650126a888aabd4cb4463fca868ec596826325dedc2",
"sha256:e997baa4f2e9139951b6f4c631bad912dfd3c792467e2f03d7239464af90e914"
],
"version": "==2.0.0"
"version": "==2.1.0"
},
"sphinx": {
"hashes": [
@@ -343,6 +415,7 @@
"sha256:806111e5e962be97c29ec4c1e7fe277bfd19e9652fb1a4392105b43e01af885a",
"sha256:a072735ec80e7675e3f432fcae8610ecf509c5f1869d17e2eecff44389cdbc58"
],
"markers": "python_version >= '3.5'",
"version": "==1.0.2"
},
"sphinxcontrib-devhelp": {
@@ -350,6 +423,7 @@
"sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e",
"sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4"
],
"markers": "python_version >= '3.5'",
"version": "==1.0.2"
},
"sphinxcontrib-htmlhelp": {
@@ -357,6 +431,7 @@
"sha256:3c0bc24a2c41e340ac37c85ced6dafc879ab485c095b1d65d2461ac2f7cca86f",
"sha256:e8f5bb7e31b2dbb25b9cc435c8ab7a79787ebf7f906155729338f3156d93659b"
],
"markers": "python_version >= '3.5'",
"version": "==1.0.3"
},
"sphinxcontrib-jsmath": {
@@ -364,6 +439,7 @@
"sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178",
"sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"
],
"markers": "python_version >= '3.5'",
"version": "==1.0.1"
},
"sphinxcontrib-qthelp": {
@@ -371,6 +447,7 @@
"sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72",
"sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6"
],
"markers": "python_version >= '3.5'",
"version": "==1.0.3"
},
"sphinxcontrib-serializinghtml": {
@@ -378,62 +455,83 @@
"sha256:eaa0eccc86e982a9b939b2b82d12cc5d013385ba5eadcc7e4fed23f4405f77bc",
"sha256:f242a81d423f59617a8e5cf16f5d4d74e28ee9a66f9e5b637a18082991db5a9a"
],
"markers": "python_version >= '3.5'",
"version": "==1.1.4"
},
"toml": {
"hashes": [
"sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f",
"sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88"
"sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b",
"sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
],
"version": "==0.10.1"
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.10.2"
},
"typed-ast": {
"hashes": [
"sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355",
"sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919",
"sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa",
"sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652",
"sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75",
"sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01",
"sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d",
"sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1",
"sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907",
"sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c",
"sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3",
"sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b",
"sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614",
"sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb",
"sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b",
"sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41",
"sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6",
"sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34",
"sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe",
"sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4",
"sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7"
"sha256:07d49388d5bf7e863f7fa2f124b1b1d89d8aa0e2f7812faff0a5658c01c59aa1",
"sha256:14bf1522cdee369e8f5581238edac09150c765ec1cb33615855889cf33dcb92d",
"sha256:240296b27397e4e37874abb1df2a608a92df85cf3e2a04d0d4d61055c8305ba6",
"sha256:36d829b31ab67d6fcb30e185ec996e1f72b892255a745d3a82138c97d21ed1cd",
"sha256:37f48d46d733d57cc70fd5f30572d11ab8ed92da6e6b28e024e4a3edfb456e37",
"sha256:4c790331247081ea7c632a76d5b2a265e6d325ecd3179d06e9cf8d46d90dd151",
"sha256:5dcfc2e264bd8a1db8b11a892bd1647154ce03eeba94b461effe68790d8b8e07",
"sha256:7147e2a76c75f0f64c4319886e7639e490fee87c9d25cb1d4faef1d8cf83a440",
"sha256:7703620125e4fb79b64aa52427ec192822e9f45d37d4b6625ab37ef403e1df70",
"sha256:8368f83e93c7156ccd40e49a783a6a6850ca25b556c0fa0240ed0f659d2fe496",
"sha256:84aa6223d71012c68d577c83f4e7db50d11d6b1399a9c779046d75e24bed74ea",
"sha256:85f95aa97a35bdb2f2f7d10ec5bbdac0aeb9dafdaf88e17492da0504de2e6400",
"sha256:8db0e856712f79c45956da0c9a40ca4246abc3485ae0d7ecc86a20f5e4c09abc",
"sha256:9044ef2df88d7f33692ae3f18d3be63dec69c4fb1b5a4a9ac950f9b4ba571606",
"sha256:963c80b583b0661918718b095e02303d8078950b26cc00b5e5ea9ababe0de1fc",
"sha256:987f15737aba2ab5f3928c617ccf1ce412e2e321c77ab16ca5a293e7bbffd581",
"sha256:9ec45db0c766f196ae629e509f059ff05fc3148f9ffd28f3cfe75d4afb485412",
"sha256:9fc0b3cb5d1720e7141d103cf4819aea239f7d136acf9ee4a69b047b7986175a",
"sha256:a2c927c49f2029291fbabd673d51a2180038f8cd5a5b2f290f78c4516be48be2",
"sha256:a38878a223bdd37c9709d07cd357bb79f4c760b29210e14ad0fb395294583787",
"sha256:b4fcdcfa302538f70929eb7b392f536a237cbe2ed9cba88e3bf5027b39f5f77f",
"sha256:c0c74e5579af4b977c8b932f40a5464764b2f86681327410aa028a22d2f54937",
"sha256:c1c876fd795b36126f773db9cbb393f19808edd2637e00fd6caba0e25f2c7b64",
"sha256:c9aadc4924d4b5799112837b226160428524a9a45f830e0d0f184b19e4090487",
"sha256:cc7b98bf58167b7f2db91a4327da24fb93368838eb84a44c472283778fc2446b",
"sha256:cf54cfa843f297991b7388c281cb3855d911137223c6b6d2dd82a47ae5125a41",
"sha256:d003156bb6a59cda9050e983441b7fa2487f7800d76bdc065566b7d728b4581a",
"sha256:d175297e9533d8d37437abc14e8a83cbc68af93cc9c1c59c2c292ec59a0697a3",
"sha256:d746a437cdbca200622385305aedd9aef68e8a645e385cc483bdc5e488f07166",
"sha256:e683e409e5c45d5c9082dc1daf13f6374300806240719f95dc783d1fc942af10"
],
"markers": "implementation_name == 'cpython' and python_version < '3.8'",
"version": "==1.4.1"
"markers": "python_version < '3.8' and implementation_name == 'cpython'",
"version": "==1.4.2"
},
"typing-extensions": {
"hashes": [
"sha256:6e95524d8a547a91e08f404ae485bbb71962de46967e1b71a0cb89af24e761c5",
"sha256:79ee589a3caca649a9bfd2a8de4709837400dfa00b6cc81962a1e6a1815969ae",
"sha256:f8d2bd89d25bc39dabe7d23df520442fa1d8969b82544370e03d88b5a591c392"
"sha256:7cb407020f00f7bfc3cb3e7881628838e69d8f3fcab2f64742a5e76b2f841918",
"sha256:99d4073b617d30288f569d3f13d2bd7548c3a7e4c8de87db09a9d29bb3a4a60c",
"sha256:dafc7639cde7f1b6e1acc0f457842a83e722ccca8eef5270af2d74792619a89f"
],
"version": "==3.7.4.2"
"index": "pypi",
"version": "==3.7.4.3"
},
"urllib3": {
"hashes": [
"sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a",
"sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461"
"sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df",
"sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937"
],
"version": "==1.25.10"
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.4"
},
"wrapt": {
"hashes": [
"sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7"
],
"version": "==1.12.1"
},
"zipp": {
"hashes": [
"sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76",
"sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098"
],
"markers": "python_version >= '3.6'",
"version": "==3.4.1"
}
}
}
+4 -2
View File
@@ -29,7 +29,7 @@
.. |downloads| image:: https://pepy.tech/badge/instaloader/month
:alt: PyPI Download Count
:target: https://pepy.tech/project/instaloader/month
:target: https://pepy.tech/project/instaloader
.. badges-end
@@ -84,7 +84,9 @@ To later **update your local copy** of that profiles, you may run
instaloader --fast-update profile [profile ...]
If ``--fast-update`` is given, Instaloader stops when arriving at the
first already-downloaded picture. When updating profiles, Instaloader
first already-downloaded picture.
When updating profiles, Instaloader
automatically **detects profile name changes** and renames the target directory
accordingly.
+1 -1
View File
@@ -6,7 +6,7 @@ pkgbase = instaloader
arch = any
license = MIT
makedepends = python-setuptools
depends = python>=3.5
depends = python>=3.6
depends = python-requests>=2.4
options = !emptydirs
source = instaloader-{{version}}.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v{{version}}
+1 -1
View File
@@ -8,7 +8,7 @@ arch=('any')
url="https://instaloader.github.io/"
license=('MIT')
groups=()
depends=('python>=3.5' 'python-requests>=2.4')
depends=('python>=3.6' 'python-requests>=2.4')
makedepends=('python-setuptools')
options=('!emptydirs')
source=($pkgname-$pkgver.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v$pkgver)
+1 -1
View File
@@ -44,7 +44,7 @@ with open('__main__.py', 'w+') as f:
f.writelines(lines)
# install dependencies and invoke PyInstaller
commands = ["pip install pipenv==2020.6.2",
commands = ["pip install pipenv==2020.11.15",
"pipenv sync --dev",
"pipenv run pyinstaller --log-level=DEBUG instaloader.spec"]
+23 -4
View File
@@ -220,12 +220,20 @@ How to Download
used to save the information to resume an interrupted download. The default
prefix is ``iterator``.
Resuming an interrupted download is supported for most, but not all targets.
Resuming an interrupted download is supported for the following targets:
- Profile posts,
- Profile IGTV posts (:option:`--igtv`),
- Profile tagged posts (:option:`--tagged`),
- Saved posts (``:saved``).
This feature is enabled by default for targets where it is supported;
:option:`--resume-prefix` only changes the name of the iterator files.
To turn this feature off, use :option:`--no-resume`.
JSON files with resume information are always compressed, regardless of
:option:`--no-compress-json`.
This feature is turned off entirely with :option:`--no-resume`.
.. versionadded:: 4.5
.. option:: --no-resume
@@ -238,7 +246,7 @@ How to Download
.. option:: --user-agent USER_AGENT
User Agent to use for HTTP requests. Per default, Instaloader pretends being
Chrome/51.
Chrome/89 on Linux.
.. option:: --max-connection-attempts N
@@ -255,6 +263,17 @@ How to Download
.. versionchanged:: 4.6
Enabled this option by default with a timeout of 300 seconds.
.. option:: --abort-on STATUS_CODE_LIST
Comma-separated list of HTTP status codes that cause Instaloader to abort,
bypassing all retry logic.
For example, with ``--abort-on=302,400,429``, Instaloader will stop if a
request is responded with a 302 redirect, a Bad Request error, or a Too Many
Requests error.
.. versionadded:: 4.7
Miscellaneous Options
^^^^^^^^^^^^^^^^^^^^^
+1 -1
View File
@@ -8,7 +8,7 @@ Install Instaloader
To **install Instaloader**,
#. Ensure that you have `Python <https://www.python.org/>`__, at least
version 3.5, and `pip <https://pypi.python.org/pypi/pip>`__
version 3.6, and `pip <https://pypi.python.org/pypi/pip>`__
installed.
#. Then, install Instaloader using::
+2
View File
@@ -47,3 +47,5 @@ Exceptions
.. autoexception:: TooManyRequestsException
.. autoexception:: AbortDownloadException
+1 -1
View File
@@ -1,7 +1,7 @@
"""Download pictures (or videos) along with their captions and other metadata from Instagram."""
__version__ = '4.6'
__version__ = '4.7.1'
try:
+21 -7
View File
@@ -5,12 +5,12 @@ import datetime
import os
import re
import sys
from argparse import ArgumentParser, SUPPRESS
from argparse import ArgumentParser, ArgumentTypeError, SUPPRESS
from typing import List, Optional
from . import (Instaloader, InstaloaderException, InvalidArgumentException, Post, Profile, ProfileNotExistsException,
StoryItem, __version__, load_structure_from_file, TwoFactorAuthRequiredException,
BadCredentialsException)
from . import (AbortDownloadException, BadCredentialsException, Instaloader, InstaloaderException,
InvalidArgumentException, Post, Profile, ProfileNotExistsException, StoryItem,
TwoFactorAuthRequiredException, __version__, load_structure_from_file)
from .instaloader import get_default_session_filename
from .instaloadercontext import default_user_agent
@@ -27,6 +27,14 @@ def usage_string():
{0} --help""".format(argv0, len(argv0), '')
def http_status_code_list(code_list_str: str) -> List[int]:
codes = [int(s) for s in code_list_str.split(',')]
for code in codes:
if not 100 <= code <= 599:
raise ArgumentTypeError("Invalid HTTP status code: {}".format(code))
return codes
def filterstr_to_filterfunc(filter_str: str, item_type: type):
"""Takes an --post-filter=... or --storyitem-filter=... filter
specification and makes a filter_func Callable out of it."""
@@ -199,7 +207,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
' '.join([p.username for p in profiles])))
if profiles and (download_profile_pic or download_posts) and not instaloader.context.is_logged_in:
instaloader.context.error("Warning: Use --login to download higher-quality versions of pictures.")
instaloader.context.log("Hint: Use --login to download higher-quality versions of pictures.")
instaloader.download_profiles(profiles,
download_profile_pic, download_posts, download_tagged, download_igtv,
download_highlights, download_stories,
@@ -213,6 +221,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
fast_update=fast_update, post_filter=post_filter)
except KeyboardInterrupt:
print("\nInterrupted by user.", file=sys.stderr)
except AbortDownloadException as exc:
print("\nDownload aborted: {}.".format(exc), file=sys.stderr)
# Save session if it is useful
if instaloader.context.is_logged_in:
instaloader.save_session_to_file(sessionfile)
@@ -369,6 +379,9 @@ def main():
g_how.add_argument('--commit-mode', action='store_true', help=SUPPRESS)
g_how.add_argument('--request-timeout', metavar='N', type=float, default=300.0,
help='Seconds to wait before timing out a connection request. Defaults to 300.')
g_how.add_argument('--abort-on', type=http_status_code_list, metavar="STATUS_CODES",
help='Comma-separated list of HTTP status codes that cause Instaloader to abort, bypassing all '
'retry logic.')
g_misc = parser.add_argument_group('Miscellaneous Options')
g_misc.add_argument('-q', '--quiet', action='store_true',
@@ -427,7 +440,8 @@ def main():
request_timeout=args.request_timeout,
resume_prefix=resume_prefix,
check_resume_bbd=not args.use_aged_resume_files,
slide=args.slide)
slide=args.slide,
fatal_status_codes=args.abort_on)
_main(loader,
args.profile,
username=args.login.lower() if args.login is not None else None,
@@ -445,7 +459,7 @@ def main():
storyitem_filter_str=args.storyitem_filter)
loader.close()
except InstaloaderException as err:
raise SystemExit("Fatal error: %s" % err)
raise SystemExit("Fatal error: %s" % err) from err
if __name__ == "__main__":
+12
View File
@@ -64,3 +64,15 @@ class QueryReturnedNotFoundException(ConnectionException):
class TooManyRequestsException(ConnectionException):
pass
class AbortDownloadException(Exception):
"""
Exception that is not catched in the error catchers inside the download loop and so aborts the
download loop.
This exception is not a subclass of ``InstaloaderException``.
.. versionadded:: 4.7
"""
pass
+108 -36
View File
@@ -161,6 +161,7 @@ class Instaloader:
:param resume_prefix: :option:`--resume-prefix`, or None for :option:`--no-resume`.
:param check_resume_bbd: Whether to check the date of expiry of resume files and reject them if expired.
:param slide: :option:`--slide`
:param fatal_status_codes: :option:`--abort-on`
.. attribute:: context
@@ -176,8 +177,8 @@ class Instaloader:
download_pictures=True,
download_videos: bool = True,
download_video_thumbnails: bool = True,
download_geotags: bool = True,
download_comments: bool = True,
download_geotags: bool = False,
download_comments: bool = False,
save_metadata: bool = True,
compress_json: bool = True,
post_metadata_txt_pattern: str = None,
@@ -187,10 +188,11 @@ class Instaloader:
rate_controller: Optional[Callable[[InstaloaderContext], RateController]] = None,
resume_prefix: Optional[str] = "iterator",
check_resume_bbd: bool = True,
slide: Optional[str] = None):
slide: Optional[str] = None,
fatal_status_codes: Optional[List[int]] = None):
self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts,
request_timeout, rate_controller)
request_timeout, rate_controller, fatal_status_codes)
# configuration parameters
self.dirname_pattern = dirname_pattern or "{target}"
@@ -256,7 +258,8 @@ class Instaloader:
request_timeout=self.context.request_timeout,
resume_prefix=self.resume_prefix,
check_resume_bbd=self.check_resume_bbd,
slide=self.slide)
slide=self.slide,
fatal_status_codes=self.context.fatal_status_codes)
yield new_loader
self.context.error_log.extend(new_loader.context.error_log)
new_loader.context.error_log = [] # avoid double-printing of errors
@@ -332,18 +335,47 @@ class Instaloader:
combined_answers.extend(y['answers'])
unique_comments_list[-1]['answers'] = get_unique_comments(combined_answers)
return unique_comments_list
def get_new_comments(new_comments, start):
for idx, comment in enumerate(new_comments, start=start+1):
if idx % 250 == 0:
self.context.log('{}'.format(idx), end='…', flush=True)
yield comment
def save_comments(extended_comments):
unique_comments = get_unique_comments(extended_comments, combine_answers=True)
answer_ids = set(int(answer['id']) for comment in unique_comments for answer in comment.get('answers', []))
with open(filename, 'w') as file:
file.write(json.dumps(list(filter(lambda t: int(t['id']) not in answer_ids, unique_comments)),
indent=4))
base_filename = filename
filename += '_comments.json'
try:
with open(filename) as fp:
comments = json.load(fp)
except (FileNotFoundError, json.decoder.JSONDecodeError):
comments = list()
comments.extend(_postcomment_asdict(comment) for comment in post.get_comments())
comments_iterator = post.get_comments()
try:
with resumable_iteration(
context=self.context,
iterator=comments_iterator,
load=load_structure_from_file,
save=save_structure_to_file,
format_path=lambda magic: "{}_{}_{}.json.xz".format(base_filename, self.resume_prefix, magic),
check_bbd=self.check_resume_bbd,
enabled=self.resume_prefix is not None
) as (_is_resuming, start_index):
comments.extend(_postcomment_asdict(comment)
for comment in get_new_comments(comments_iterator, start_index))
except (KeyboardInterrupt, AbortDownloadException):
if comments:
save_comments(comments)
raise
if comments:
comments = get_unique_comments(comments, combine_answers=True)
answer_ids = set(int(answer['id']) for comment in comments for answer in comment.get('answers', []))
with open(filename, 'w') as file:
file.write(json.dumps(list(filter(lambda t: int(t['id']) not in answer_ids, comments)), indent=4))
save_comments(comments)
self.context.log('comments', end=' ', flush=True)
def save_caption(self, filename: str, mtime: datetime, caption: str) -> None:
@@ -551,6 +583,27 @@ class Instaloader:
:return: True if something was downloaded, False otherwise, i.e. file was already there
"""
def _already_downloaded(path: str) -> bool:
if not os.path.isfile(path):
return False
else:
self.context.log(path + ' exists', end=' ', flush=True)
return True
def _all_already_downloaded(path_base, is_videos_enumerated) -> bool:
if '{filename}' in self.filename_pattern:
# full URL needed to evaluate actual filename, cannot determine at
# this point if all sidecar nodes were already downloaded.
return False
for idx, is_video in is_videos_enumerated:
if self.download_pictures and (not is_video or self.download_video_thumbnails):
if not _already_downloaded("{0}_{1}.jpg".format(path_base, idx)):
return False
if is_video and self.download_videos:
if not _already_downloaded("{0}_{1}.mp4".format(path_base, idx)):
return False
return True
dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target)
filename_template = os.path.join(dirname, self.format_filename(post, target=target))
filename = self.__prepare_filename(filename_template, lambda: post.url)
@@ -559,37 +612,47 @@ class Instaloader:
downloaded = True
if post.typename == 'GraphSidecar':
if self.download_pictures or self.download_videos:
for edge_number, sidecar_node in enumerate(
post.get_sidecar_nodes(self.slide_start, self.slide_end),
start=post.mediacount if self.slide_start < 0 else self.slide_start + 1
if not _all_already_downloaded(
filename_template, enumerate(
(post.get_is_videos()[i]
for i in range(self.slide_start % post.mediacount, self.slide_end % post.mediacount + 1)),
start=self.slide_start % post.mediacount + 1
)
):
if self.download_pictures and (not sidecar_node.is_video or self.download_video_thumbnails):
suffix = str(edge_number)
for edge_number, sidecar_node in enumerate(
post.get_sidecar_nodes(self.slide_start, self.slide_end),
start=self.slide_start % post.mediacount + 1
):
suffix = str(edge_number) # type: Optional[str]
if '{filename}' in self.filename_pattern:
suffix = ''
# pylint:disable=cell-var-from-loop
filename = self.__prepare_filename(filename_template, lambda: sidecar_node.display_url)
# Download sidecar picture or video thumbnail (--no-pictures implies --no-video-thumbnails)
downloaded &= self.download_pic(filename=filename, url=sidecar_node.display_url,
mtime=post.date_local, filename_suffix=suffix)
if sidecar_node.is_video and self.download_videos:
suffix = str(edge_number)
if '{filename}' in self.filename_pattern:
suffix = ''
# pylint:disable=cell-var-from-loop
filename = self.__prepare_filename(filename_template, lambda: sidecar_node.video_url)
# Download sidecar video if desired
downloaded &= self.download_pic(filename=filename, url=sidecar_node.video_url,
mtime=post.date_local, filename_suffix=suffix)
suffix = None
if self.download_pictures and (not sidecar_node.is_video or self.download_video_thumbnails):
# pylint:disable=cell-var-from-loop
sidecar_filename = self.__prepare_filename(filename_template,
lambda: sidecar_node.display_url)
# Download sidecar picture or video thumbnail (--no-pictures implies --no-video-thumbnails)
downloaded &= self.download_pic(filename=sidecar_filename, url=sidecar_node.display_url,
mtime=post.date_local, filename_suffix=suffix)
if sidecar_node.is_video and self.download_videos:
# pylint:disable=cell-var-from-loop
sidecar_filename = self.__prepare_filename(filename_template,
lambda: sidecar_node.video_url)
# Download sidecar video if desired
downloaded &= self.download_pic(filename=sidecar_filename, url=sidecar_node.video_url,
mtime=post.date_local, filename_suffix=suffix)
else:
downloaded = False
elif post.typename == 'GraphImage':
# Download picture
if self.download_pictures:
downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
downloaded = (not _already_downloaded(filename + ".jpg") and
self.download_pic(filename=filename, url=post.url, mtime=post.date_local))
elif post.typename == 'GraphVideo':
# Download video thumbnail (--no-pictures implies --no-video-thumbnails)
if self.download_pictures and self.download_video_thumbnails:
with self.context.error_catcher("Video thumbnail of {}".format(post)):
downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
downloaded = (not _already_downloaded(filename + ".jpg") and
self.download_pic(filename=filename, url=post.url, mtime=post.date_local))
else:
self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename))
@@ -600,7 +663,8 @@ class Instaloader:
# Download video if desired
if post.is_video and self.download_videos:
downloaded &= self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local)
downloaded &= (not _already_downloaded(filename + ".mp4") and
self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local))
# Download geotags if desired
if self.download_geotags and post.location:
@@ -693,17 +757,25 @@ class Instaloader:
:return: True if something was downloaded, False otherwise, i.e. file was already there
"""
def _already_downloaded(path: str) -> bool:
if not os.path.isfile(path):
return False
else:
self.context.log(path + ' exists', end=' ', flush=True)
return True
date_local = item.date_local
dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target)
filename_template = os.path.join(dirname, self.format_filename(item, target=target))
filename = self.__prepare_filename(filename_template, lambda: item.url)
downloaded = False
if not item.is_video or self.download_video_thumbnails is True:
url = item.url
downloaded = self.download_pic(filename=filename, url=url, mtime=date_local)
downloaded = (not _already_downloaded(filename + ".jpg") and
self.download_pic(filename=filename, url=item.url, mtime=date_local))
if item.is_video and self.download_videos is True:
filename = self.__prepare_filename(filename_template, lambda: str(item.video_url))
downloaded |= self.download_pic(filename=filename, url=item.video_url, mtime=date_local)
downloaded |= (not _already_downloaded(filename + ".mp4") and
self.download_pic(filename=filename, url=item.video_url, mtime=date_local))
# Save caption if desired
metadata_string = _ArbitraryItemFormatter(item).format(self.storyitem_metadata_txt_pattern).strip()
if metadata_string:
+58 -19
View File
@@ -33,7 +33,7 @@ def copy_session(session: requests.Session, request_timeout: Optional[float] = N
def default_user_agent() -> str:
return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \
'(KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36'
'(KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36'
class InstaloaderContext:
@@ -53,7 +53,8 @@ class InstaloaderContext:
def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
max_connection_attempts: int = 3, request_timeout: float = 300.0,
rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None):
rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None,
fatal_status_codes: Optional[List[int]] = None):
self.user_agent = user_agent if user_agent is not None else default_user_agent()
self.request_timeout = request_timeout
@@ -74,6 +75,9 @@ class InstaloaderContext:
# Can be set to True for testing, disables supression of InstaloaderContext._error_catcher
self.raise_all_errors = False
# HTTP status codes that should cause an AbortDownloadException
self.fatal_status_codes = fatal_status_codes or []
# Cache profile from id (mapping from id to Profile)
self.profile_id_cache = dict() # type: Dict[int, Any]
@@ -218,8 +222,10 @@ class InstaloaderContext:
data={'enc_password': enc_password, 'username': user}, allow_redirects=True)
try:
resp_json = login.json()
except json.decoder.JSONDecodeError:
raise ConnectionException("Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason))
except json.decoder.JSONDecodeError as err:
raise ConnectionException(
"Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason)
) from err
if resp_json.get('two_factor_required'):
two_factor_session = copy_session(session, self.request_timeout)
two_factor_session.headers.update({'X-CSRFToken': csrf_token})
@@ -288,7 +294,7 @@ class InstaloaderContext:
def do_sleep(self):
"""Sleep a short time if self.sleep is set. Called before each request to instagram.com."""
if self.sleep:
time.sleep(min(random.expovariate(0.7), 5.0))
time.sleep(min(random.expovariate(0.6), 15.0))
def get_json(self, path: str, params: Dict[str, Any], host: str = 'www.instagram.com',
session: Optional[requests.Session] = None, _attempt=1) -> Dict[str, Any]:
@@ -316,6 +322,11 @@ class InstaloaderContext:
if is_other_query:
self._rate_controller.wait_before_query('other')
resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False)
if resp.status_code in self.fatal_status_codes:
redirect = " redirect to {}".format(resp.headers['location']) if 'location' in resp.headers else ""
raise AbortDownloadException("Query to https://{}/{} responded with \"{} {}\"{}".format(
host, path, resp.status_code, resp.reason, redirect
))
while resp.is_redirect:
redirect_url = resp.headers['location']
self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url))
@@ -549,6 +560,7 @@ class RateController:
self._context = context
self._query_timestamps = dict() # type: Dict[str, List[float]]
self._earliest_next_request_time = 0.0
self._iphone_earliest_next_request_time = 0.0
def sleep(self, secs: float):
"""Wait given number of seconds."""
@@ -559,7 +571,7 @@ class RateController:
def _dump_query_timestamps(self, current_time: float, failed_query_type: str):
windows = [10, 11, 20, 22, 30, 60]
self._context.error("Requests within last {} minutes grouped by type:"
self._context.error("Number of requests within last {} minutes grouped by type:"
.format('/'.join(str(w) for w in windows)),
repeat_at_end=False)
for query_type, times in self._query_timestamps.items():
@@ -571,11 +583,15 @@ class RateController:
), repeat_at_end=False)
def count_per_sliding_window(self, query_type: str) -> int:
"""Return how many requests can be done within the sliding window."""
"""Return how many requests of the given type can be done within a sliding window of 11 minutes.
This is called by :meth:`RateController.query_waittime` and allows to simply customize wait times before queries
at query_type granularity. Consider overriding :meth:`RateController.query_waittime` directly if you need more
control."""
# Not static, to allow for the count_per_sliding_window to depend on context-inherent properties, such as
# whether we are logged in.
# pylint:disable=no-self-use
return 75 if query_type in ['iphone', 'other'] else 200
return 75 if query_type == 'other' else 200
def _reqs_in_sliding_window(self, query_type: Optional[str], current_time: float, window: float) -> List[float]:
if query_type is not None:
@@ -591,6 +607,7 @@ class RateController:
def query_waittime(self, query_type: str, current_time: float, untracked_queries: bool = False) -> float:
"""Calculate time needed to wait before query can be executed."""
per_type_sliding_window = 660
iphone_sliding_window = 1800
if query_type not in self._query_timestamps:
self._query_timestamps[query_type] = []
self._query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60,
@@ -616,25 +633,43 @@ class RateController:
def untracked_next_request_time():
if untracked_queries:
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, per_type_sliding_window)
self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6
return self._earliest_next_request_time
if query_type == "iphone":
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
iphone_sliding_window)
self._iphone_earliest_next_request_time = min(reqs_in_sliding_window) + iphone_sliding_window + 18
else:
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time,
per_type_sliding_window)
self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6
return max(self._iphone_earliest_next_request_time, self._earliest_next_request_time)
def iphone_next_request():
if query_type == "iphone":
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, iphone_sliding_window)
if len(reqs_in_sliding_window) >= 199:
return min(reqs_in_sliding_window) + iphone_sliding_window + 18
return 0.0
return max(0.0,
max(
per_type_next_request_time(),
gql_accumulated_next_request_time(),
untracked_next_request_time(),
iphone_next_request(),
) - current_time)
def wait_before_query(self, query_type: str) -> None:
"""This method is called before a query to Instagram. It calls :meth:`RateController.sleep` to wait
until the request can be made."""
"""This method is called before a query to Instagram.
It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
:meth:`RateController.sleep` to wait until the request can be made."""
waittime = self.query_waittime(query_type, time.monotonic(), False)
assert waittime >= 0
if waittime > 15:
self._context.log("\nToo many queries in the last time. Need to wait {} seconds, until {:%H:%M}."
.format(round(waittime), datetime.now() + timedelta(seconds=waittime)))
formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
"{} minutes".format(round(waittime / 60)))
self._context.log("\nToo many queries in the last time. Need to wait {}, until {:%H:%M}."
.format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)))
if waittime > 0:
self.sleep(waittime)
if query_type not in self._query_timestamps:
@@ -643,8 +678,10 @@ class RateController:
self._query_timestamps[query_type].append(time.monotonic())
def handle_429(self, query_type: str) -> None:
"""This method is called to handle a 429 Too Many Requests response. It calls :meth:`RateController.sleep` to
wait until we can repeat the same request."""
"""This method is called to handle a 429 Too Many Requests response.
It calls :meth:`RateController.query_waittime` to determine the time needed to wait and then calls
:meth:`RateController.sleep` to wait until we can repeat the same request."""
current_time = time.monotonic()
waittime = self.query_waittime(query_type, current_time, True)
assert waittime >= 0
@@ -654,8 +691,10 @@ class RateController:
"App while Instaloader is running.")
self._context.error(textwrap.fill(text_for_429), repeat_at_end=False)
if waittime > 1.5:
self._context.error("The request will be retried in {} seconds, at {:%H:%M}."
.format(round(waittime), datetime.now() + timedelta(seconds=waittime)),
formatted_waittime = ("{} seconds".format(round(waittime)) if waittime <= 666 else
"{} minutes".format(round(waittime / 60)))
self._context.error("The request will be retried in {}, at {:%H:%M}."
.format(formatted_waittime, datetime.now() + timedelta(seconds=waittime)),
repeat_at_end=False)
if waittime > 0:
self.sleep(waittime)
+10 -10
View File
@@ -5,9 +5,9 @@ import os
from contextlib import contextmanager
from datetime import datetime, timedelta
from lzma import LZMAError
from typing import Any, Callable, Dict, Iterator, NamedTuple, Optional, Tuple, TypeVar
from typing import Any, Callable, Dict, Iterable, Iterator, NamedTuple, Optional, Tuple, TypeVar
from .exceptions import InvalidArgumentException, QueryReturnedBadRequestException
from .exceptions import AbortDownloadException, InvalidArgumentException, QueryReturnedBadRequestException
from .instaloadercontext import InstaloaderContext
FrozenNodeIterator = NamedTuple('FrozenNodeIterator',
@@ -150,11 +150,7 @@ class NodeIterator(Iterator[T]):
@property
def magic(self) -> str:
"""Magic string for easily identifying a matching iterator file for resuming (hash of some parameters)."""
if 'blake2b' not in hashlib.algorithms_available:
magic_hash = hashlib.new('sha224')
else:
# Use blake2b when possible, i.e. on Python >= 3.6.
magic_hash = hashlib.blake2b(digest_size=6) # type:ignore # pylint: disable=no-member
magic_hash = hashlib.blake2b(digest_size=6)
magic_hash.update(json.dumps(
[self._query_hash, self._query_variables, self._query_referer, self._context.username]
).encode())
@@ -204,14 +200,15 @@ class NodeIterator(Iterator[T]):
@contextmanager
def resumable_iteration(context: InstaloaderContext,
iterator: Iterator,
iterator: Iterable,
load: Callable[[InstaloaderContext, str], Any],
save: Callable[[FrozenNodeIterator, str], None],
format_path: Callable[[str], str],
check_bbd: bool = True,
enabled: bool = True) -> Iterator[Tuple[bool, int]]:
"""
High-level context manager to handle a resumable iteration that can be interrupted with a KeyboardInterrupt.
High-level context manager to handle a resumable iteration that can be interrupted
with a :class:`KeyboardInterrupt` or an :class:`AbortDownloadException`.
It can be used as follows to automatically load a previously-saved state into the iterator, save the iterator's
state when interrupted, and delete the resume file upon completion::
@@ -239,6 +236,9 @@ def resumable_iteration(context: InstaloaderContext,
:param format_path: Returns the path to the resume file for the given magic.
:param check_bbd: Whether to check the best before date and reject an expired FrozenNodeIterator.
:param enabled: Set to False to disable all functionality and simply execute the inner body.
.. versionchanged:: 4.7
Also interrupt on :class:`AbortDownloadException`.
"""
if not enabled or not isinstance(iterator, NodeIterator):
yield False, 0
@@ -262,7 +262,7 @@ def resumable_iteration(context: InstaloaderContext,
context.error("Warning: Not resuming from {}: {}".format(resume_file_path, exc))
try:
yield is_resuming, start_index
except KeyboardInterrupt:
except (KeyboardInterrupt, AbortDownloadException):
if os.path.dirname(resume_file_path):
os.makedirs(os.path.dirname(resume_file_path), exist_ok=True)
save(iterator.freeze(), resume_file_path)
+34 -13
View File
@@ -4,7 +4,7 @@ import re
from base64 import b64decode, b64encode
from collections import namedtuple
from datetime import datetime
from typing import Any, Dict, Iterator, List, Optional, Union
from typing import Any, Dict, Iterable, Iterator, List, Optional, Union
from . import __version__
from .exceptions import *
@@ -128,6 +128,14 @@ class Post:
"""The mediaid is a decimal representation of the media shortcode."""
return int(self._node['id'])
@property
def title(self) -> Optional[str]:
"""Title of post"""
try:
return self._field('title')
except KeyError:
return None
def __repr__(self):
return '<Post {}>'.format(self.shortcode)
@@ -241,7 +249,7 @@ class Post:
if self.typename == "GraphImage" and self._context.is_logged_in:
try:
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
url = re.sub(r'([?&])se=\d+&?', r'\1', orig_url).rstrip('&')
return url
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
@@ -264,6 +272,17 @@ class Post:
return len(edges)
return 1
def get_is_videos(self) -> List[bool]:
"""
Return a list containing the ``is_video`` property for each media in the post.
.. versionadded:: 4.7
"""
if self.typename == 'GraphSidecar':
edges = self._field('edge_sidecar_to_children', 'edges')
return [edge['node']['is_video'] for edge in edges]
return [self.is_video]
def get_sidecar_nodes(self, start=0, end=-1) -> Iterator[PostSidecarNode]:
"""
Sidecar nodes of a Post with typename==GraphSidecar.
@@ -273,13 +292,13 @@ class Post:
"""
if self.typename == 'GraphSidecar':
edges = self._field('edge_sidecar_to_children', 'edges')
if any(edge['node']['is_video'] for edge in edges):
# video_url is only present in full metadata, issue #558.
edges = self._full_metadata['edge_sidecar_to_children']['edges']
if end < 0:
end = len(edges)-1
if start < 0:
start = len(edges)-1
if any(edge['node']['is_video'] and 'video_url' not in edge['node'] for edge in edges[start:(end+1)]):
# video_url is only present in full metadata, issue #558.
edges = self._full_metadata['edge_sidecar_to_children']['edges']
for idx, edge in enumerate(edges):
if start <= idx <= end:
node = edge['node']
@@ -289,7 +308,7 @@ class Post:
try:
carousel_media = self._iphone_struct['carousel_media']
orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
display_url = re.sub(r'([?&])se=\d+&?', r'\1', orig_url).rstrip('&')
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(
err, self))
@@ -407,12 +426,15 @@ class Post:
except KeyError:
return self._field('edge_media_to_comment', 'count')
def get_comments(self) -> Iterator[PostComment]:
def get_comments(self) -> Iterable[PostComment]:
r"""Iterate over all comments of the post.
Each comment is represented by a PostComment namedtuple with fields text (string), created_at (datetime),
id (int), owner (:class:`Profile`) and answers (:class:`~typing.Iterator`\ [:class:`PostCommentAnswer`])
if available.
.. versionchanged:: 4.7
Change return type to ``Iterable``.
"""
def _postcommentanswer(node):
return PostCommentAnswer(id=int(node['id']),
@@ -447,16 +469,15 @@ class Post:
answers=_postcommentanswers(node))
if self.comments == 0:
# Avoid doing additional requests if there are no comments
return
return []
comment_edges = self._field('edge_media_to_comment', 'edges')
answers_count = sum([edge['node'].get('edge_threaded_comments', {}).get('count', 0) for edge in comment_edges])
if self.comments == len(comment_edges) + answers_count:
# If the Post's metadata already contains all parent comments, don't do GraphQL requests to obtain them
yield from (_postcomment(comment['node']) for comment in comment_edges)
return
yield from NodeIterator(
return [_postcomment(comment['node']) for comment in comment_edges]
return NodeIterator(
self._context,
'97b41c52301f77ce508f55e66d17620e',
lambda d: d['data']['shortcode_media']['edge_media_to_parent_comment'],
@@ -835,7 +856,7 @@ class Profile:
self._obtain_metadata()
return NodeIterator(
self._context,
'472f257a40c653c64c666ce877d59d2b',
'003056d32c2554def87228bc3fd9668a',
lambda d: d['data']['user']['edge_owner_to_timeline_media'],
lambda n: Post(self._context, n, self),
{'id': self.userid},
@@ -1061,7 +1082,7 @@ class StoryItem:
if self.typename == "GraphStoryImage" and self._context.is_logged_in:
try:
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
url = re.sub(r'([?&])se=\d+&?', r'\1', orig_url).rstrip('&')
return url
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
+4 -8
View File
@@ -3,7 +3,6 @@
import re
import sys
import os
import platform
from setuptools import setup
@@ -19,14 +18,11 @@ def get_version():
raise SystemExit("Could not find version string.")
if sys.version_info < (3, 5):
sys.exit('Instaloader requires Python >= 3.5.')
if sys.version_info < (3, 6):
sys.exit('Instaloader requires Python >= 3.6.')
requirements = ['requests>=2.4']
if platform.system() == 'Windows' and sys.version_info < (3, 6):
requirements.append('win_unicode_console')
keywords = (['instagram', 'instagram-scraper', 'instagram-client', 'instagram-feed', 'downloader', 'videos', 'photos',
'pictures', 'instagram-user-photos', 'instagram-photos', 'instagram-metadata', 'instagram-downloader',
'instagram-stories'])
@@ -47,7 +43,7 @@ setup(
'from Instagram.',
long_description=open(os.path.join(SRC, 'README.rst')).read(),
install_requires=requirements,
python_requires='>=3.5',
python_requires='>=3.6',
entry_points={'console_scripts': ['instaloader=instaloader.__main__:main']},
zip_safe=False,
keywords=keywords,
@@ -58,10 +54,10 @@ setup(
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3 :: Only',
'Topic :: Internet',
'Topic :: Multimedia :: Graphics'