Compare commits

...
65 Commits
Author SHA1 Message Date
Alexander Graf 37a93ee59a Second Beta Release for Version 4.9 2022-01-15 17:34:18 +01:00
2d4f3b8537 Make {Post,StoryItem}.date_local timezone aware
This commit is made up of 21365ec668 555c86633c d864ce08ff

Discussed in #1305, #1316, #1372.

Co-Authored-By: Misael <misael.om@gmail.com>
Co-Authored-By: Eduardo Kalinowski <eduardo@kalinowski.com.br>
2022-01-15 17:34:18 +01:00
Alexander Graf 8c752532e1 Merge branch 'master' into upcoming/v4.9 2022-01-15 17:17:19 +01:00
Alexander Graf fdcf2e1367 Release of Version 4.8.4 2022-01-15 17:11:45 +01:00
Alexander Graf ca78fee307 Revert date_utc and date_local being timezone aware
Fixes #1379.

Revert "Make {Post,StoryItem}.date_utc timezone aware"
d864ce08ff.

Revert "Remove unnecessary conversion to timezone aware timestamp (#1372)"
555c86633c.

Revert "Timezone option for datetime style formatting (#1316)"
21365ec668.
2022-01-15 17:08:04 +01:00
Alexander Graf d6e5e31005 Release of Version 4.8.3 2022-01-07 14:56:40 +01:00
Alexander Graf d864ce08ff Make {Post,StoryItem}.date_utc timezone aware
Also simplify {Post,StoryItem}.date_local.

Discussed in #1316.
2022-01-07 14:53:23 +01:00
Eduardo Kalinowski 555c86633c Remove unnecessary conversion to timezone aware timestamp (#1372)
Post.date_local now already returns a timestamp with the appropriate
timezone.
2022-01-07 14:31:41 +01:00
Misael 21365ec668 Timezone option for datetime style formatting (#1316)
Add timezone awareness for datetime objects returned by `date_local` and `date_utc`.
Fixes #1305.
2021-12-31 16:50:16 +01:00
Alexander Graf 6343e9a69d Release of Version 4.8.2 2021-11-27 16:25:12 +01:00
Alexander Graf 06574eb428 Fix KeyError on attempt to get incomplete location
Fixes #1349.
2021-11-27 16:25:12 +01:00
JeremyKj87 815ba83d77 avoid comments endless loop - check if query_response is always the same (#1347) 2021-11-27 15:47:32 +01:00
Alexander Graf 5bff9b5474 First Beta Release for Version 4.9 2021-11-12 21:44:08 +01:00
Alexander Graf a23b832a69 Add Post.accessibility_caption property 2021-11-12 21:41:40 +01:00
Alexander Graf 5d18857695 Fix download of hashtags and locations
Fixes #1080, fixes #1129, closes #1240.
2021-11-12 21:33:48 +01:00
Alexander Graf d6fd4c560c Merge branch 'master' into upcoming/v4.9 2021-11-12 20:28:19 +01:00
fireattack 470fc0d905 Fix KeyError for video_url, and better workflow (#1321)
Fixes #1320.
2021-11-12 15:21:33 +01:00
Alexander Graf 0300d8a4f1 Fix CI for Python 3.10 2021-11-12 14:32:15 +01:00
Alexander Graf f09bd51ed4 Update feature suggestion issue template 2021-09-25 10:32:36 +02:00
Arman Yeghiazaryan 7a32272b55 Fix the issue with folder names starting with dots (#1252) 2021-08-14 16:27:14 +02:00
Alexander Graf 9eb2339028 Update default user agent string 2021-08-10 09:25:00 +02:00
Alexander Graf c45b536be9 Update dependencies 2021-08-10 09:25:00 +02:00
Alexander Graf 2c116683c2 Release of Version 4.8.1 2021-08-10 09:15:11 +02:00
Alexander Graf 054fa47245 Fix rare KeyError: 'video_versions' error
Closes #1251.
2021-08-10 09:13:42 +02:00
Alexander Graf 1f8ebe09b4 Release of Version 4.8 2021-08-04 18:27:45 +02:00
Alexander Graf 87dfdff335 Merge branch 'master' into upcoming/v4.8 2021-08-04 18:24:46 +02:00
Alexander Graf 631503d463 Log "Using latest stamps from ..." message
Instaloader always logs which files are loaded and considered in the
current run (such as session files or resume files). This commit adds a
log message "Using latest stamps from ..." when using --latest-stamps,
to be consistent.
2021-08-04 18:12:12 +02:00
Eduardo Kalinowski 43c9d32b22 Avoid uninitialized variable when called with :stories (#1242) 2021-08-04 18:04:16 +02:00
Alexander Graf 1ad9111753 Release of Version 4.7.6 2021-08-04 17:46:06 +02:00
fireattackandAlexander Graf 327fcfd8e8 Download best-quality video (#1232)
Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
2021-08-04 17:42:14 +02:00
Arman Yeghiazaryan ae39ab9893 Fix Post.location / --geotags (#1244)
Closes #1109.
2021-08-04 17:28:56 +02:00
Alexander Graf 0372f7c8e8 Release of Version 4.7.5 2021-07-24 20:34:14 +02:00
Alexander Graf aaeaaa0e22 Fourth Release Candidate for Version 4.8 2021-07-24 19:29:56 +02:00
Alexander Graf 35bd8357d1 Merge branch 'master' into upcoming/v4.8 2021-07-24 19:28:53 +02:00
Eduardo Kalinowski 9cdf679fc1 Fix interruped downloads with --latest-stamps (#1219)
The most recent post is cached in NodeIterator (and
saved to the disk), and its timestamp is used, instead of the
timestamp instaloader was run.

This way, even in later resuming runs the timestamp stored is the same
that would have been stored if the first run.

Fixes #1206.
2021-07-24 19:27:46 +02:00
Alexander Graf 94d599db61 List all login issues in docs/troubleshooting 2021-07-24 19:18:39 +02:00
fireattack a0e3e8db0d Do not delete referer header (#1237)
since it causes IG's CDN to limit download speed
2021-07-24 16:37:07 +02:00
Alexander Graf 26c9f71132 Third Release Candidate for Version 4.8 2021-06-22 21:52:26 +02:00
Eduardo Kalinowski 210719af56 Avoid uninitialized variable (#1204)
fixes #1203
2021-06-22 21:50:06 +02:00
Alexander Graf 8c1129e556 Add GitHub Sponsors link 2021-06-20 13:50:59 +02:00
Alexander Graf c200031be6 Increase stale period for pull requests and issues 2021-06-20 13:50:59 +02:00
Alexander Graf a0d2ec95b9 Release of Version 4.7.4 2021-06-18 15:19:51 +02:00
Alexander Graf 750dd3f47f Second Release Candidate for Version 4.8 2021-06-18 15:13:01 +02:00
Alexander Graf 839b77eef0 First Release Candidate for Version 4.8 2021-06-16 23:17:04 +02:00
Alexander Graf e346842f7b Merge branch 'master' into upcoming/v4.8 2021-06-16 23:08:18 +02:00
Alexander Graf 9b3f809a9a Release of Version 4.7.3 2021-06-16 23:00:18 +02:00
twitter-79 803440ac50 Fix Login Error (#1192)
taking csrf token from login page as it is empty using the old url

Fixes #1150.
2021-06-16 22:35:58 +02:00
Eduardo Kalinowski 455a757159 Make downloads using --latest-stamps resumable (#1193)
Using itertools.takewhile() on a NodeIterator returns a plain Iterator,
and so it's not resumable.

The strategy has been altered to pass an extra argument to
posts_download_loop, a lambda that is evaluated for each post, and
causes the loop to stop when it returns false.
2021-06-16 22:19:16 +02:00
Alexander Graf 07de73bde1 Fix missing 2FA error message 2021-06-15 21:05:57 +02:00
spykard bf83d406d3 Fix and Update has_public_story Functionality (#1183) 2021-06-15 19:39:06 +02:00
Eduardo Kalinowski ac5d6e312a Report Profile renames even if --quiet is enabled (#1180)
While this isn't really an error, it requires user action, so the message
is printed to stderr even with --quiet, and repeated at the end when that
option is not enabled.

Resolves #1159.
2021-06-10 18:49:05 +02:00
Alexander Graf 3bb4a5b4b1 Release of Version 4.7.2 2021-06-06 11:18:48 +02:00
Kalle Westerling 9ffdf258a9 Fix Post.mentions including email addresses (#1167)
Earlier, if someone had an email address in the caption, those were provided as mentions in the Post.mentions attribute. With this update to the regular expression, email addresses are no longer ingested.

Fixes #1029.
2021-06-06 11:13:05 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> d530a7a37f Bump urllib3 from 1.26.4 to 1.26.5 (#1163)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.4 to 1.26.5.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.4...1.26.5)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-04 15:12:01 +02:00
Eduardo Kalinowski d142fb70b0 --latest-stamps, like --fast-update, but without the need to keep downloaded files (#1131)
Adds the --latest-stamps command line option, pointing to a file where
the latest time each profile was scraped. On the next run, only posts
newer than that time are downloaded.

Fixes #1122.
2021-05-27 18:22:33 +02:00
SAO 8a26943215 update documentation for --count option (#1139) 2021-05-17 18:51:39 +02:00
Eduardo Kalinowski 5345470ebf Add option --title-pattern for profile pics and covers (#1127) 2021-05-15 16:25:36 +02:00
Eduardo Kalinowski a2715cbe03 Show progress of profile scraping (#1114) 2021-04-29 21:14:23 +02:00
anokhovd fa19b95107 Add support for loading structures directly from python json objects (#1092) 2021-04-22 20:12:42 +02:00
A Bored Pervert 3837b642b0 Make iPhone endpoint query optional (#1076) 2021-04-22 20:06:35 +02:00
Alexander Graf e67b9e6c37 Improve docs on how to load/save FrozenNodeIterator 2021-04-12 16:54:44 +02:00
Alexander Graf 2bf175a111 Release of Version 4.7.1 2021-03-27 20:47:28 +01:00
Alexander Graf 95d8534aed Fix SRCINFO depends metadata in AUR 2021-03-27 20:46:53 +01:00
Alexander GrafandAndré Koch-Kramer b583cd07e1 Get better-quality version of pictures if logged-in
Fixes #1065 (also see #630).

Co-Authored-By: André Koch-Kramer <koch-kramer@web.de>
2021-03-27 20:42:56 +01:00
Alexander Graf 52f258e89b Release of Version 4.7 2021-03-27 15:19:12 +01:00
27 changed files with 1194 additions and 509 deletions
+9 -2
View File
@@ -6,13 +6,15 @@ labels: "feature suggestion"
--- ---
**Provide us a use case of the feature** **Provide us a use case of the feature**
How could the user invoke the new function? Which problem would it solve? If new information is obtained, how would it be further processed? How could the user invoke the new function? Which problem would it solve?
If new information is obtained, how would it be further processed?
**Describe the solution you'd like** **Describe the solution you'd like**
A clear and concise description of what you want to happen. A clear and concise description of what you want to happen.
**Describe alternatives you've considered** **Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered. A clear and concise description of any alternative solutions or features you've considered.
How do you achieve the here-described goal without the feature you are requesting?
**If the feature request is accepted, would you be willing to submit a pull request?** **If the feature request is accepted, would you be willing to submit a pull request?**
Yes / No Yes / No
@@ -21,4 +23,9 @@ Yes / No
**Additional context** **Additional context**
Add any other context about the feature request here. Add any other context about the feature request here.
<!-- please also see https://instaloader.github.io/contributing.html for how to suggest a feature --> <!--
Also see https://instaloader.github.io/contributing.html for how to suggest a feature.
Please understand that a feature suggestion issue that lacks the required information cannot be
processed.
-->
+2 -1
View File
@@ -1 +1,2 @@
custom: "https://instaloader.github.io/contributing.html#donations" github: [aandergr]
custom: "https://instaloader.github.io/sponsors.html"
+2 -2
View File
@@ -8,7 +8,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
python-version: [3.8, 3.9] python-version: ["3.8", "3.9", "3.10"]
steps: steps:
- name: Checkout Instaloader Repository - name: Checkout Instaloader Repository
uses: actions/checkout@v2 uses: actions/checkout@v2
@@ -20,7 +20,7 @@ jobs:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
- name: Install Dependencies - name: Install Dependencies
run: | run: |
python -m pip install pipenv==2020.11.15 python -m pip install pipenv==2021.5.29
pipenv sync --dev pipenv sync --dev
- name: PyLint - name: PyLint
run: pipenv run pylint instaloader run: pipenv run pylint instaloader
+1 -1
View File
@@ -30,5 +30,5 @@ jobs:
stale-pr-label: 'stale' stale-pr-label: 'stale'
exempt-issue-label: 'leave open' exempt-issue-label: 'leave open'
exempt-pr-label: 'leave open' exempt-pr-label: 'leave open'
days-before-stale: 90 days-before-stale: 135
days-before-close: 14 days-before-close: 14
+2 -2
View File
@@ -10,9 +10,9 @@ jobs:
- name: Checkout Instaloader repository - name: Checkout Instaloader repository
uses: actions/checkout@v2 uses: actions/checkout@v2
- name: Setup Python - name: Setup Python
uses: actions/setup-python@v1 uses: actions/setup-python@v2
with: with:
python-version: 3.9 python-version: "3.9"
architecture: x64 architecture: x64
- name: Get the tagged version - name: Get the tagged version
id: get_version id: get_version
+2
View File
@@ -12,6 +12,8 @@ pefile = "*"
pywin32-ctypes = "*" pywin32-ctypes = "*"
psutil = "*" psutil = "*"
typing-extensions = "*" typing-extensions = "*"
types-requests = "*"
typed-ast = "*"
[packages] [packages]
requests = "*" requests = "*"
Generated
+206 -248
View File
@@ -1,7 +1,7 @@
{ {
"_meta": { "_meta": {
"hash": { "hash": {
"sha256": "9704331c2f9e76af9e5ee386bc359a46ee66d1e2f93aa880d9d148da7ba0e6f5" "sha256": "4ea11d3dd7ae0070a8745cf21960210a09834ca21f47905fd3fab20ea68c6d3b"
}, },
"pipfile-spec": 6, "pipfile-spec": 6,
"requires": {}, "requires": {},
@@ -16,42 +16,41 @@
"default": { "default": {
"certifi": { "certifi": {
"hashes": [ "hashes": [
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c", "sha256:2bbf76fd432960138b3ef6dda3dde0544f27cbf8546c458e60baf371917ba9ee",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830" "sha256:50b1e4f8446b06f41be7dd6338db18e0990601dce795c2b1686458aa7e8fa7d8"
], ],
"version": "==2020.12.5" "version": "==2021.5.30"
}, },
"chardet": { "charset-normalizer": {
"hashes": [ "hashes": [
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa", "sha256:0c8911edd15d19223366a194a513099a302055a962bca2cec0f54b8b63175d8b",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5" "sha256:f23667ebe1084be45f6ae0538e4a5a865206544097e4e8bbcacf42cd02a348f3"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", "markers": "python_version >= '3'",
"version": "==4.0.0" "version": "==2.0.4"
}, },
"idna": { "idna": {
"hashes": [ "hashes": [
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6", "sha256:14475042e284991034cb48e06f6851428fb14c4dc953acd9be9a5e95c7b6dd7a",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0" "sha256:467fbad99067910785144ce333826c71fb0e63a425657295239737f7ecd125f3"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", "markers": "python_version >= '3'",
"version": "==2.10" "version": "==3.2"
}, },
"requests": { "requests": {
"hashes": [ "hashes": [
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804", "sha256:6c1246513ecd5ecd4528a0906f910e8f0f9c6b8ec72030dc9fd154dc1a6efd24",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e" "sha256:b8aa58f8cf793ffd8782d3d8cb19e66ef36f7aba4353eec859e74678b01b07a7"
], ],
"index": "pypi", "index": "pypi",
"version": "==2.25.1" "version": "==2.26.0"
}, },
"urllib3": { "urllib3": {
"hashes": [ "hashes": [
"sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df", "sha256:39fb8672126159acb139a7718dd10806104dec1e2f0f6c88aab05d17df10c8d4",
"sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937" "sha256:f57b4c16c62fa2760b7e3d97c35b255512fb6b59a259730f36ba32ce9f8e342f"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'", "version": "==1.26.6"
"version": "==1.26.4"
} }
}, },
"develop": { "develop": {
@@ -71,177 +70,140 @@
}, },
"astroid": { "astroid": {
"hashes": [ "hashes": [
"sha256:21d735aab248253531bb0f1e1e6d068f0ee23533e18ae8a6171ff892b98297cf", "sha256:3975a0bd5373bdce166e60c851cfcbaf21ee96de80ec518c1f4cb3e94c3fb334",
"sha256:cfc35498ee64017be059ceffab0a25bedf7548ab76f2bea691c5565896e7128d" "sha256:ab7f36e8a78b8e54a62028ba6beef7561db4cdb6f2a5009ecc44a6f42b5697ef"
], ],
"markers": "python_version >= '3.6'", "version": "==2.6.6"
"version": "==2.5.1"
}, },
"babel": { "babel": {
"hashes": [ "hashes": [
"sha256:9d35c22fcc79893c3ecc85ac4a56cde1ecf3f19c540bba0922308a6c06ca6fa5", "sha256:ab49e12b91d937cd11f0b67cb259a57ab4ad2b59ac7a3b41d6c06c0ac5b0def9",
"sha256:da031ab54472314f210b0adcff1588ee5d1d1d0ba4dbd07b94dba82bde791e05" "sha256:bc0c176f9f6a994582230df350aa6e05ba2ebe4b3ac317eab29d9be5d2768da0"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", "version": "==2.9.1"
"version": "==2.9.0"
}, },
"certifi": { "certifi": {
"hashes": [ "hashes": [
"sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c", "sha256:2bbf76fd432960138b3ef6dda3dde0544f27cbf8546c458e60baf371917ba9ee",
"sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830" "sha256:50b1e4f8446b06f41be7dd6338db18e0990601dce795c2b1686458aa7e8fa7d8"
], ],
"version": "==2020.12.5" "version": "==2021.5.30"
}, },
"chardet": { "charset-normalizer": {
"hashes": [ "hashes": [
"sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa", "sha256:0c8911edd15d19223366a194a513099a302055a962bca2cec0f54b8b63175d8b",
"sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5" "sha256:f23667ebe1084be45f6ae0538e4a5a865206544097e4e8bbcacf42cd02a348f3"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", "markers": "python_version >= '3'",
"version": "==4.0.0" "version": "==2.0.4"
}, },
"docutils": { "docutils": {
"hashes": [ "hashes": [
"sha256:0c5b78adfbf7762415433f5515cd5c9e762339e23369dbe8000d84a4bf4ab3af", "sha256:686577d2e4c32380bb50cbb22f575ed742d58168cee37e99117a854bcd88f125",
"sha256:c2de3a60e9e7d07be26b7f2b00ca0309c207e06c100f9cc2a94931fc75a478fc" "sha256:cf316c8370a737a022b72b56874f6602acf974a37a9fba42ec2876387549fc61"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", "version": "==0.17.1"
"version": "==0.16"
}, },
"future": { "future": {
"hashes": [ "hashes": [
"sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d" "sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d"
], ],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.18.2" "version": "==0.18.2"
}, },
"idna": { "idna": {
"hashes": [ "hashes": [
"sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6", "sha256:14475042e284991034cb48e06f6851428fb14c4dc953acd9be9a5e95c7b6dd7a",
"sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0" "sha256:467fbad99067910785144ce333826c71fb0e63a425657295239737f7ecd125f3"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", "markers": "python_version >= '3'",
"version": "==2.10" "version": "==3.2"
}, },
"imagesize": { "imagesize": {
"hashes": [ "hashes": [
"sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1", "sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1",
"sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1" "sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.2.0" "version": "==1.2.0"
}, },
"importlib-metadata": {
"hashes": [
"sha256:742add720a20d0467df2f444ae41704000f50e1234f46174b51f9c6031a1bd71",
"sha256:b74159469b464a99cb8cc3e21973e4d96e05d3024d337313fedb618a6e86e6f4"
],
"markers": "python_version < '3.8'",
"version": "==3.7.3"
},
"isort": { "isort": {
"hashes": [ "hashes": [
"sha256:c729845434366216d320e936b8ad6f9d681aab72dc7cbc2d51bedc3582f3ad1e", "sha256:9c2ea1e62d871267b78307fe511c0838ba0da28698c5732d54e2790bf3ba9899",
"sha256:fff4f0c04e1825522ce6949973e83110a6e907750cd92d128b0d14aaaadbffdc" "sha256:e17d6e2b81095c9db0a03a8025a957f334d6ea30b26f9ec70805411e5c7c81f2"
], ],
"markers": "python_version >= '3.6' and python_version < '4.0'", "version": "==5.9.3"
"version": "==5.7.0"
}, },
"jinja2": { "jinja2": {
"hashes": [ "hashes": [
"sha256:03e47ad063331dd6a3f04a43eddca8a966a26ba0c5b7207a9a9e4e08f1b29419", "sha256:1f06f2da51e7b56b8f238affdd6b4e2c61e39598a378cc49345bc1bd42a978a4",
"sha256:a6d58433de0ae800347cab1fa3043cebbabe8baa9d29e668f1c768cb87a333c6" "sha256:703f484b47a6af502e743c9122595cc812b0271f661722403114f71a79d0f5a4"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", "version": "==3.0.1"
"version": "==2.11.3"
}, },
"lazy-object-proxy": { "lazy-object-proxy": {
"hashes": [ "hashes": [
"sha256:1d33d6f789697f401b75ce08e73b1de567b947740f768376631079290118ad39", "sha256:17e0967ba374fc24141738c69736da90e94419338fd4c7c7bef01ee26b339653",
"sha256:2f2de8f8ac0be3e40d17730e0600619d35c78c13a099ea91ef7fb4ad944ce694", "sha256:1fee665d2638491f4d6e55bd483e15ef21f6c8c2095f235fef72601021e64f61",
"sha256:3782931963dc89e0e9a0ae4348b44762e868ea280e4f8c233b537852a8996ab9", "sha256:22ddd618cefe54305df49e4c069fa65715be4ad0e78e8d252a33debf00f6ede2",
"sha256:37d9c34b96cca6787fe014aeb651217944a967a5b165e2cacb6b858d2997ab84", "sha256:24a5045889cc2729033b3e604d496c2b6f588c754f7a62027ad4437a7ecc4837",
"sha256:38c3865bd220bd983fcaa9aa11462619e84a71233bafd9c880f7b1cb753ca7fa", "sha256:410283732af311b51b837894fa2f24f2c0039aa7f220135192b38fcc42bd43d3",
"sha256:429c4d1862f3fc37cd56304d880f2eae5bd0da83bdef889f3bd66458aac49128", "sha256:4732c765372bd78a2d6b2150a6e99d00a78ec963375f236979c0626b97ed8e43",
"sha256:522b7c94b524389f4a4094c4bf04c2b02228454ddd17c1a9b2801fac1d754871", "sha256:489000d368377571c6f982fba6497f2aa13c6d1facc40660963da62f5c379726",
"sha256:57fb5c5504ddd45ed420b5b6461a78f58cbb0c1b0cbd9cd5a43ad30a4a3ee4d0", "sha256:4f60460e9f1eb632584c9685bccea152f4ac2130e299784dbaf9fae9f49891b3",
"sha256:5944a9b95e97de1980c65f03b79b356f30a43de48682b8bdd90aa5089f0ec1f4", "sha256:5743a5ab42ae40caa8421b320ebf3a998f89c85cdc8376d6b2e00bd12bd1b587",
"sha256:6f4e5e68b7af950ed7fdb594b3f19a0014a3ace0fedb86acb896e140ffb24302", "sha256:85fb7608121fd5621cc4377a8961d0b32ccf84a7285b4f1d21988b2eae2868e8",
"sha256:71a1ef23f22fa8437974b2d60fedb947c99a957ad625f83f43fd3de70f77f458", "sha256:9698110e36e2df951c7c36b6729e96429c9c32b3331989ef19976592c5f3c77a",
"sha256:8a44e9901c0555f95ac401377032f6e6af66d8fc1fbfad77a7a8b1a826e0b93c", "sha256:9d397bf41caad3f489e10774667310d73cb9c4258e9aed94b9ec734b34b495fd",
"sha256:b6577f15d5516d7d209c1a8cde23062c0f10625f19e8dc9fb59268859778d7d7", "sha256:b579f8acbf2bdd9ea200b1d5dea36abd93cabf56cf626ab9c744a432e15c815f",
"sha256:c8fe2d6ff0ff583784039d0255ea7da076efd08507f2be6f68583b0da32e3afb", "sha256:b865b01a2e7f96db0c5d12cfea590f98d8c5ba64ad222300d93ce6ff9138bcad",
"sha256:cadfa2c2cf54d35d13dc8d231253b7985b97d629ab9ca6e7d672c35539d38163", "sha256:bf34e368e8dd976423396555078def5cfc3039ebc6fc06d1ae2c5a65eebbcde4",
"sha256:cd1bdace1a8762534e9a36c073cd54e97d517a17d69a17985961265be6d22847", "sha256:c6938967f8528b3668622a9ed3b31d145fab161a32f5891ea7b84f6b790be05b",
"sha256:ddbdcd10eb999d7ab292677f588b658372aadb9a52790f82484a37127a390108", "sha256:d1c2676e3d840852a2de7c7d5d76407c772927addff8d742b9808fe0afccebdf",
"sha256:e7273c64bccfd9310e9601b8f4511d84730239516bada26a0c9846c9697617ef", "sha256:d7124f52f3bd259f510651450e18e0fd081ed82f3c08541dffc7b94b883aa981",
"sha256:e7428977763150b4cf83255625a80a23dfdc94d43be7791ce90799d446b4e26f", "sha256:d900d949b707778696fdf01036f58c9876a0d8bfe116e8d220cfd4b15f14e741",
"sha256:e960e8be509e8d6d618300a6c189555c24efde63e85acaf0b14b2cd1ac743315", "sha256:ebfd274dcd5133e0afae738e6d9da4323c3eb021b3e13052d8cbd0e457b1256e",
"sha256:ecb5dd5990cec6e7f5c9c1124a37cb2c710c6d69b0c1a5c4aa4b35eba0ada068", "sha256:ed361bb83436f117f9917d282a456f9e5009ea12fd6de8742d1a4752c3017e93",
"sha256:ef3f5e288aa57b73b034ce9c1f1ac753d968f9069cd0742d1d69c698a0167166", "sha256:f5144c75445ae3ca2057faac03fda5a902eff196702b0a24daf1d6ce0650514b"
"sha256:fa5b2dee0e231fa4ad117be114251bdfe6afe39213bd629d43deb117b6a6c40a",
"sha256:fa7fb7973c622b9e725bee1db569d2c2ee64d2f9a089201c5e8185d482c7352d"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", "version": "==1.6.0"
"version": "==1.5.2"
}, },
"markupsafe": { "markupsafe": {
"hashes": [ "hashes": [
"sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473", "sha256:01a9b8ea66f1658938f65b93a85ebe8bc016e6769611be228d797c9d998dd298",
"sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161", "sha256:023cb26ec21ece8dc3907c0e8320058b2e0cb3c55cf9564da612bc325bed5e64",
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235", "sha256:0446679737af14f45767963a1a9ef7620189912317d095f2d9ffa183a4d25d2b",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5", "sha256:0717a7390a68be14b8c793ba258e075c6f4ca819f15edfc2a3a027c823718567",
"sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42", "sha256:0955295dd5eec6cb6cc2fe1698f4c6d84af2e92de33fbcac4111913cd100a6ff",
"sha256:195d7d2c4fbb0ee8139a6cf67194f3973a6b3042d742ebe0a9ed36d8b6f0c07f", "sha256:10f82115e21dc0dfec9ab5c0223652f7197feb168c940f3ef61563fc2d6beb74",
"sha256:22c178a091fc6630d0d045bdb5992d2dfe14e3259760e713c490da5323866c39", "sha256:1d609f577dc6e1aa17d746f8bd3c31aa4d258f4070d61b2aa5c4166c1539de35",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff", "sha256:2ef54abee730b502252bcdf31b10dacb0a416229b72c18b19e24a4509f273d26",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b", "sha256:3c112550557578c26af18a1ccc9e090bfe03832ae994343cfdacd287db6a6ae7",
"sha256:2beec1e0de6924ea551859edb9e7679da6e4870d32cb766240ce17e0a0ba2014", "sha256:47ab1e7b91c098ab893b828deafa1203de86d0bc6ab587b160f78fe6c4011f75",
"sha256:3b8a6499709d29c2e2399569d96719a1b21dcd94410a586a18526b143ec8470f", "sha256:49e3ceeabbfb9d66c3aef5af3a60cc43b85c33df25ce03d0031a608b0a8b2e3f",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1", "sha256:4efca8f86c54b22348a5467704e3fec767b2db12fc39c6d963168ab1d3fc9135",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e", "sha256:53edb4da6925ad13c07b6d26c2a852bd81e364f95301c66e930ab2aef5b5ddd8",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183", "sha256:594c67807fb16238b30c44bdf74f36c02cdf22d1c8cda91ef8a0ed8dabf5620a",
"sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66", "sha256:611d1ad9a4288cf3e3c16014564df047fe08410e628f89805e475368bd304914",
"sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b", "sha256:6557b31b5e2c9ddf0de32a691f2312a32f77cd7681d8af66c2692efdbef84c18",
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1", "sha256:693ce3f9e70a6cf7d2fb9e6c9d8b204b6b39897a2c4a1aa65728d5ac97dcc1d8",
"sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15", "sha256:6a7fae0dd14cf60ad5ff42baa2e95727c3d81ded453457771d02b7d2b3f9c0c2",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1", "sha256:6c4ca60fa24e85fe25b912b01e62cb969d69a23a5d5867682dd3e80b5b02581d",
"sha256:6f1e273a344928347c1290119b493a1f0303c52f5a5eae5f16d74f48c15d4a85", "sha256:7d91275b0245b1da4d4cfa07e0faedd5b0812efc15b702576d103293e252af1b",
"sha256:6fffc775d90dcc9aed1b89219549b329a9250d918fd0b8fa8d93d154918422e1", "sha256:905fec760bd2fa1388bb5b489ee8ee5f7291d692638ea5f67982d968366bef9f",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e", "sha256:97383d78eb34da7e1fa37dd273c20ad4320929af65d156e35a5e2d89566d9dfb",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b", "sha256:984d76483eb32f1bcb536dc27e4ad56bba4baa70be32fa87152832cdd9db0833",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905", "sha256:a30e67a65b53ea0a5e62fe23682cfe22712e01f453b95233b25502f7c61cb415",
"sha256:7fed13866cf14bba33e7176717346713881f56d9d2bcebab207f7a036f41b850", "sha256:ab3ef638ace319fa26553db0624c4699e31a28bb2a835c5faca8f8acf6a5a902",
"sha256:84dee80c15f1b560d55bcfe6d47b27d070b4681c699c572af2e3c7cc90a3b8e0", "sha256:b2f4bf27480f5e5e8ce285a8c8fd176c0b03e93dcc6646477d4630e83440c6a9",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735", "sha256:b7f2d075102dc8c794cbde1947378051c4e5180d52d276987b8d28a3bd58c17d",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d", "sha256:be98f628055368795d818ebf93da628541e10b75b41c559fdf36d104c5787066",
"sha256:98bae9582248d6cf62321dcb52aaf5d9adf0bad3b40582925ef7c7f0ed85fceb", "sha256:d7f9850398e85aba693bb640262d3611788b1f29a79f0c93c565694658f4071f",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e", "sha256:f5653a225f31e113b152e56f154ccbe59eeb1c7487b39b9d9f9cdb58e6c79dc5",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d", "sha256:f826e31d18b516f653fe296d967d700fddad5901ae07c622bb3705955e1faa94",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c", "sha256:f8ba0e8349a38d3001fae7eadded3f6606f0da5d748ee53cc1dab1d6527b9509",
"sha256:a6a744282b7718a2a62d2ed9d993cad6f5f585605ad352c11de459f4108df0a1", "sha256:f9081981fe268bd86831e5c75f7de206ef275defcb82bc70740ae6dc507aee51",
"sha256:acf08ac40292838b3cbbb06cfe9b2cb9ec78fce8baca31ddb87aaac2e2dc3bc2", "sha256:fa130dd50c57d53368c9d59395cb5526eda596d3ffe36666cd81a44d56e48872"
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b1dba4527182c95a0db8b6060cc98ac49b9e2f5e64320e2b56e47cb2831978c7",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:b7d644ddb4dbd407d31ffb699f1d140bc35478da613b441c582aeb7c43838dd8",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:bf5aa3cbcfdf57fa2ee9cd1822c862ef23037f5c832ad09cfea57fa846dec193",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:caabedc8323f1e93231b52fc32bdcde6db817623d33e100708d9a68e1f53b26b",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
"sha256:d53bc011414228441014aa71dbec320c66468c1030aae3a6e29778a3382d96e5",
"sha256:d73a845f227b0bfe8a7455ee623525ee656a9e2e749e4742706d80a6065d5e2c",
"sha256:d9be0ba6c527163cbed5e0857c451fcd092ce83947944d6c14bc95441203f032",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be",
"sha256:feb7b34d6325451ef96bc0e36e1a6c0c1c64bc1fbec4b854f4529e51887b1621"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", "version": "==2.0.1"
"version": "==1.1.1"
}, },
"mccabe": { "mccabe": {
"hashes": [ "hashes": [
@@ -252,31 +214,32 @@
}, },
"mypy": { "mypy": {
"hashes": [ "hashes": [
"sha256:0d0a87c0e7e3a9becdfbe936c981d32e5ee0ccda3e0f07e1ef2c3d1a817cf73e", "sha256:088cd9c7904b4ad80bec811053272986611b84221835e079be5bcad029e79dd9",
"sha256:25adde9b862f8f9aac9d2d11971f226bd4c8fbaa89fb76bdadb267ef22d10064", "sha256:0aadfb2d3935988ec3815952e44058a3100499f5be5b28c34ac9d79f002a4a9a",
"sha256:28fb5479c494b1bab244620685e2eb3c3f988d71fd5d64cc753195e8ed53df7c", "sha256:119bed3832d961f3a880787bf621634ba042cb8dc850a7429f643508eeac97b9",
"sha256:2f9b3407c58347a452fc0736861593e105139b905cca7d097e413453a1d650b4", "sha256:1a85e280d4d217150ce8cb1a6dddffd14e753a4e0c3cf90baabb32cefa41b59e",
"sha256:33f159443db0829d16f0a8d83d94df3109bb6dd801975fe86bacb9bf71628e97", "sha256:3c4b8ca36877fc75339253721f69603a9c7fdb5d4d5a95a1a1b899d8b86a4de2",
"sha256:3f2aca7f68580dc2508289c729bd49ee929a436208d2b2b6aab15745a70a57df", "sha256:3e382b29f8e0ccf19a2df2b29a167591245df90c0b5a2542249873b5c1d78212",
"sha256:499c798053cdebcaa916eef8cd733e5584b5909f789de856b482cd7d069bdad8", "sha256:42c266ced41b65ed40a282c575705325fa7991af370036d3f134518336636f5b",
"sha256:4eec37370483331d13514c3f55f446fc5248d6373e7029a29ecb7b7494851e7a", "sha256:53fd2eb27a8ee2892614370896956af2ff61254c275aaee4c230ae771cadd885",
"sha256:552a815579aa1e995f39fd05dde6cd378e191b063f031f2acfe73ce9fb7f9e56", "sha256:704098302473cb31a218f1775a873b376b30b4c18229421e9e9dc8916fd16150",
"sha256:5873888fff1c7cf5b71efbe80e0e73153fe9212fafdf8e44adfe4c20ec9f82d7", "sha256:7df1ead20c81371ccd6091fa3e2878559b5c4d4caadaf1a484cf88d93ca06703",
"sha256:61a3d5b97955422964be6b3baf05ff2ce7f26f52c85dd88db11d5e03e146a3a6", "sha256:866c41f28cee548475f146aa4d39a51cf3b6a84246969f3759cb3e9c742fc072",
"sha256:674e822aa665b9fd75130c6c5f5ed9564a38c6cea6a6432ce47eafb68ee578c5", "sha256:a155d80ea6cee511a3694b108c4494a39f42de11ee4e61e72bc424c490e46457",
"sha256:7ce3175801d0ae5fdfa79b4f0cfed08807af4d075b402b7e294e6aa72af9aa2a", "sha256:adaeee09bfde366d2c13fe6093a7df5df83c9a2ba98638c7d76b010694db760e",
"sha256:9743c91088d396c1a5a3c9978354b61b0382b4e3c440ce83cf77994a43e8c521", "sha256:b6fb13123aeef4a3abbcfd7e71773ff3ff1526a7d3dc538f3929a49b42be03f0",
"sha256:9f94aac67a2045ec719ffe6111df543bac7874cee01f41928f6969756e030564", "sha256:b94e4b785e304a04ea0828759172a15add27088520dc7e49ceade7834275bedb",
"sha256:a26f8ec704e5a7423c8824d425086705e381b4f1dfdef6e3a1edab7ba174ec49", "sha256:c0df2d30ed496a08de5daed2a9ea807d07c21ae0ab23acf541ab88c24b26ab97",
"sha256:abf7e0c3cf117c44d9285cc6128856106183938c68fd4944763003decdcfeb66", "sha256:c6c2602dffb74867498f86e6129fd52a2770c48b7cd3ece77ada4fa38f94eba8",
"sha256:b09669bcda124e83708f34a94606e01b614fa71931d356c1f1a5297ba11f110a", "sha256:ceb6e0a6e27fb364fb3853389607cf7eb3a126ad335790fa1e14ed02fba50811",
"sha256:cd07039aa5df222037005b08fbbfd69b3ab0b0bd7a07d7906de75ae52c4e3119", "sha256:d9dd839eb0dc1bbe866a288ba3c1afc33a202015d2ad83b31e875b5905a079b6",
"sha256:d23e0ea196702d918b60c8288561e722bf437d82cb7ef2edcd98cfa38905d506", "sha256:e4dab234478e3bd3ce83bac4193b2ecd9cf94e720ddd95ce69840273bf44f6de",
"sha256:d65cc1df038ef55a99e617431f0553cd77763869eebdf9042403e16089fe746c", "sha256:ec4e0cd079db280b6bdabdc807047ff3e199f334050db5cbb91ba3e959a67504",
"sha256:d7da2e1d5f558c37d6e8c1246f1aec1e7349e4913d8fb3cb289a35de573fe2eb" "sha256:ecd2c3fe726758037234c93df7e98deb257fd15c24c9180dacf1ef829da5f921",
"sha256:ef565033fa5a958e62796867b1df10c40263ea9ded87164d67572834e57a174d"
], ],
"index": "pypi", "index": "pypi",
"version": "==0.812" "version": "==0.910"
}, },
"mypy-extensions": { "mypy-extensions": {
"hashes": [ "hashes": [
@@ -287,18 +250,17 @@
}, },
"packaging": { "packaging": {
"hashes": [ "hashes": [
"sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5", "sha256:7dc96269f53a4ccec5c0670940a4281106dd0bb343f47b7471f779df49c2fbe7",
"sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a" "sha256:c86254f9220d55e31cc94d69bade760f0847da8000def4dfe1c6b872fd14ff14"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", "version": "==21.0"
"version": "==20.9"
}, },
"pefile": { "pefile": {
"hashes": [ "hashes": [
"sha256:a5d6e8305c6b210849b47a6174ddf9c452b2888340b8177874b862ba6c207645" "sha256:ed79b2353daa58421459abf4d685953bde0adf9f6e188944f97ba9795f100246"
], ],
"index": "pypi", "index": "pypi",
"version": "==2019.4.18" "version": "==2021.5.24"
}, },
"psutil": { "psutil": {
"hashes": [ "hashes": [
@@ -336,40 +298,44 @@
}, },
"pygments": { "pygments": {
"hashes": [ "hashes": [
"sha256:2656e1a6edcdabf4275f9a3640db59fd5de107d88e8663c5d4e9a0fa62f77f94", "sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15aef65f0545519f",
"sha256:534ef71d539ae97d4c3a4cf7d6f110f214b0e687e92f9cb9d2a3b0d3101289c8" "sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd34d272ca383b8e"
], ],
"markers": "python_version >= '3.5'", "version": "==2.9.0"
"version": "==2.8.1"
}, },
"pyinstaller": { "pyinstaller": {
"hashes": [ "hashes": [
"sha256:f5c0eeb2aa663cce9a5404292c0195011fa500a6501c873a466b2e8cad3c950c" "sha256:30733baaf8971902286a0ddf77e5499ac5f7bf8e7c39163e83d4f8c696ef265e",
"sha256:4d848cd782ee0893d7ad9fe2bfe535206a79f0b6760cecc5f2add831258b9322",
"sha256:8f747b190e6ad30e2d2fd5da9a64636f61aac8c038c0b7f685efa92c782ea14f",
"sha256:aae456205c68355f9597411090576bb31b614e53976b4c102d072bbe5db8392a",
"sha256:c587da8f521a7ce1b9efb4e3d0117cd63c92dc6cedff24590aeef89372f53012",
"sha256:ecc2baadeeefd2b6fbf39d13c65d4aa603afdda1c6aaaebc4577ba72893fee9e",
"sha256:fed9f5e4802769a416a8f2ca171c6be961d1861cc05a0b71d20dfe05423137e9"
], ],
"index": "pypi", "index": "pypi",
"version": "==4.2" "version": "==4.5.1"
}, },
"pyinstaller-hooks-contrib": { "pyinstaller-hooks-contrib": {
"hashes": [ "hashes": [
"sha256:27558072021857d89524c42136feaa2ffe4f003f1bdf0278f9b24f6902c1759c", "sha256:57964f93eb69255c49159ffdf052aae893feed223b0f69773dfd010ca6c569d9",
"sha256:892310e6363655838485ee748bf1c5e5cade7963686d9af8650ee218a3e0b031" "sha256:7f5d0689b30da3092149fc536a835a94045ac8c9f0e6dfb23ac171890f5ea8f2"
], ],
"version": "==2021.1" "version": "==2021.2"
}, },
"pylint": { "pylint": {
"hashes": [ "hashes": [
"sha256:0e21d3b80b96740909d77206d741aa3ce0b06b41be375d92e1f3244a274c1f8a", "sha256:2e1a0eb2e8ab41d6b5dbada87f066492bb1557b12b76c47c2ee8aa8a11186594",
"sha256:d09b0b07ba06bcdff463958f53f23df25e740ecd81895f7d2699ec04bbd8dc3b" "sha256:8b838c8983ee1904b2de66cce9d0b96649a91901350e956d78f289c3bc87b48e"
], ],
"index": "pypi", "index": "pypi",
"version": "==2.7.2" "version": "==2.9.6"
}, },
"pyparsing": { "pyparsing": {
"hashes": [ "hashes": [
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1", "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b" "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
], ],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==2.4.7" "version": "==2.4.7"
}, },
"pytz": { "pytz": {
@@ -389,11 +355,11 @@
}, },
"requests": { "requests": {
"hashes": [ "hashes": [
"sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804", "sha256:6c1246513ecd5ecd4528a0906f910e8f0f9c6b8ec72030dc9fd154dc1a6efd24",
"sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e" "sha256:b8aa58f8cf793ffd8782d3d8cb19e66ef36f7aba4353eec859e74678b01b07a7"
], ],
"index": "pypi", "index": "pypi",
"version": "==2.25.1" "version": "==2.26.0"
}, },
"snowballstemmer": { "snowballstemmer": {
"hashes": [ "hashes": [
@@ -415,7 +381,6 @@
"sha256:806111e5e962be97c29ec4c1e7fe277bfd19e9652fb1a4392105b43e01af885a", "sha256:806111e5e962be97c29ec4c1e7fe277bfd19e9652fb1a4392105b43e01af885a",
"sha256:a072735ec80e7675e3f432fcae8610ecf509c5f1869d17e2eecff44389cdbc58" "sha256:a072735ec80e7675e3f432fcae8610ecf509c5f1869d17e2eecff44389cdbc58"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.2" "version": "==1.0.2"
}, },
"sphinxcontrib-devhelp": { "sphinxcontrib-devhelp": {
@@ -423,23 +388,20 @@
"sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e", "sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e",
"sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4" "sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.2" "version": "==1.0.2"
}, },
"sphinxcontrib-htmlhelp": { "sphinxcontrib-htmlhelp": {
"hashes": [ "hashes": [
"sha256:3c0bc24a2c41e340ac37c85ced6dafc879ab485c095b1d65d2461ac2f7cca86f", "sha256:d412243dfb797ae3ec2b59eca0e52dac12e75a241bf0e4eb861e450d06c6ed07",
"sha256:e8f5bb7e31b2dbb25b9cc435c8ab7a79787ebf7f906155729338f3156d93659b" "sha256:f5f8bb2d0d629f398bf47d0d69c07bc13b65f75a81ad9e2f71a63d4b7a2f6db2"
], ],
"markers": "python_version >= '3.5'", "version": "==2.0.0"
"version": "==1.0.3"
}, },
"sphinxcontrib-jsmath": { "sphinxcontrib-jsmath": {
"hashes": [ "hashes": [
"sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178",
"sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8" "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.1" "version": "==1.0.1"
}, },
"sphinxcontrib-qthelp": { "sphinxcontrib-qthelp": {
@@ -447,91 +409,87 @@
"sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72", "sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72",
"sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6" "sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6"
], ],
"markers": "python_version >= '3.5'",
"version": "==1.0.3" "version": "==1.0.3"
}, },
"sphinxcontrib-serializinghtml": { "sphinxcontrib-serializinghtml": {
"hashes": [ "hashes": [
"sha256:eaa0eccc86e982a9b939b2b82d12cc5d013385ba5eadcc7e4fed23f4405f77bc", "sha256:352a9a00ae864471d3a7ead8d7d79f5fc0b57e8b3f95e9867eb9eb28999b92fd",
"sha256:f242a81d423f59617a8e5cf16f5d4d74e28ee9a66f9e5b637a18082991db5a9a" "sha256:aa5f6de5dfdf809ef505c4895e51ef5c9eac17d0f287933eb49ec495280b6952"
], ],
"markers": "python_version >= '3.5'", "version": "==1.1.5"
"version": "==1.1.4"
}, },
"toml": { "toml": {
"hashes": [ "hashes": [
"sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b", "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b",
"sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f" "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
], ],
"markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.10.2" "version": "==0.10.2"
}, },
"typed-ast": { "typed-ast": {
"hashes": [ "hashes": [
"sha256:07d49388d5bf7e863f7fa2f124b1b1d89d8aa0e2f7812faff0a5658c01c59aa1", "sha256:01ae5f73431d21eead5015997ab41afa53aa1fbe252f9da060be5dad2c730ace",
"sha256:14bf1522cdee369e8f5581238edac09150c765ec1cb33615855889cf33dcb92d", "sha256:067a74454df670dcaa4e59349a2e5c81e567d8d65458d480a5b3dfecec08c5ff",
"sha256:240296b27397e4e37874abb1df2a608a92df85cf3e2a04d0d4d61055c8305ba6", "sha256:0fb71b8c643187d7492c1f8352f2c15b4c4af3f6338f21681d3681b3dc31a266",
"sha256:36d829b31ab67d6fcb30e185ec996e1f72b892255a745d3a82138c97d21ed1cd", "sha256:1b3ead4a96c9101bef08f9f7d1217c096f31667617b58de957f690c92378b528",
"sha256:37f48d46d733d57cc70fd5f30572d11ab8ed92da6e6b28e024e4a3edfb456e37", "sha256:2068531575a125b87a41802130fa7e29f26c09a2833fea68d9a40cf33902eba6",
"sha256:4c790331247081ea7c632a76d5b2a265e6d325ecd3179d06e9cf8d46d90dd151", "sha256:209596a4ec71d990d71d5e0d312ac935d86930e6eecff6ccc7007fe54d703808",
"sha256:5dcfc2e264bd8a1db8b11a892bd1647154ce03eeba94b461effe68790d8b8e07", "sha256:2c726c276d09fc5c414693a2de063f521052d9ea7c240ce553316f70656c84d4",
"sha256:7147e2a76c75f0f64c4319886e7639e490fee87c9d25cb1d4faef1d8cf83a440", "sha256:398e44cd480f4d2b7ee8d98385ca104e35c81525dd98c519acff1b79bdaac363",
"sha256:7703620125e4fb79b64aa52427ec192822e9f45d37d4b6625ab37ef403e1df70", "sha256:52b1eb8c83f178ab787f3a4283f68258525f8d70f778a2f6dd54d3b5e5fb4341",
"sha256:8368f83e93c7156ccd40e49a783a6a6850ca25b556c0fa0240ed0f659d2fe496", "sha256:5feca99c17af94057417d744607b82dd0a664fd5e4ca98061480fd8b14b18d04",
"sha256:84aa6223d71012c68d577c83f4e7db50d11d6b1399a9c779046d75e24bed74ea", "sha256:7538e495704e2ccda9b234b82423a4038f324f3a10c43bc088a1636180f11a41",
"sha256:85f95aa97a35bdb2f2f7d10ec5bbdac0aeb9dafdaf88e17492da0504de2e6400", "sha256:760ad187b1041a154f0e4d0f6aae3e40fdb51d6de16e5c99aedadd9246450e9e",
"sha256:8db0e856712f79c45956da0c9a40ca4246abc3485ae0d7ecc86a20f5e4c09abc", "sha256:777a26c84bea6cd934422ac2e3b78863a37017618b6e5c08f92ef69853e765d3",
"sha256:9044ef2df88d7f33692ae3f18d3be63dec69c4fb1b5a4a9ac950f9b4ba571606", "sha256:95431a26309a21874005845c21118c83991c63ea800dd44843e42a916aec5899",
"sha256:963c80b583b0661918718b095e02303d8078950b26cc00b5e5ea9ababe0de1fc", "sha256:9ad2c92ec681e02baf81fdfa056fe0d818645efa9af1f1cd5fd6f1bd2bdfd805",
"sha256:987f15737aba2ab5f3928c617ccf1ce412e2e321c77ab16ca5a293e7bbffd581", "sha256:9c6d1a54552b5330bc657b7ef0eae25d00ba7ffe85d9ea8ae6540d2197a3788c",
"sha256:9ec45db0c766f196ae629e509f059ff05fc3148f9ffd28f3cfe75d4afb485412", "sha256:aee0c1256be6c07bd3e1263ff920c325b59849dc95392a05f258bb9b259cf39c",
"sha256:9fc0b3cb5d1720e7141d103cf4819aea239f7d136acf9ee4a69b047b7986175a", "sha256:af3d4a73793725138d6b334d9d247ce7e5f084d96284ed23f22ee626a7b88e39",
"sha256:a2c927c49f2029291fbabd673d51a2180038f8cd5a5b2f290f78c4516be48be2", "sha256:b36b4f3920103a25e1d5d024d155c504080959582b928e91cb608a65c3a49e1a",
"sha256:a38878a223bdd37c9709d07cd357bb79f4c760b29210e14ad0fb395294583787", "sha256:b9574c6f03f685070d859e75c7f9eeca02d6933273b5e69572e5ff9d5e3931c3",
"sha256:b4fcdcfa302538f70929eb7b392f536a237cbe2ed9cba88e3bf5027b39f5f77f", "sha256:bff6ad71c81b3bba8fa35f0f1921fb24ff4476235a6e94a26ada2e54370e6da7",
"sha256:c0c74e5579af4b977c8b932f40a5464764b2f86681327410aa028a22d2f54937", "sha256:c190f0899e9f9f8b6b7863debfb739abcb21a5c054f911ca3596d12b8a4c4c7f",
"sha256:c1c876fd795b36126f773db9cbb393f19808edd2637e00fd6caba0e25f2c7b64", "sha256:c907f561b1e83e93fad565bac5ba9c22d96a54e7ea0267c708bffe863cbe4075",
"sha256:c9aadc4924d4b5799112837b226160428524a9a45f830e0d0f184b19e4090487", "sha256:cae53c389825d3b46fb37538441f75d6aecc4174f615d048321b716df2757fb0",
"sha256:cc7b98bf58167b7f2db91a4327da24fb93368838eb84a44c472283778fc2446b", "sha256:dd4a21253f42b8d2b48410cb31fe501d32f8b9fbeb1f55063ad102fe9c425e40",
"sha256:cf54cfa843f297991b7388c281cb3855d911137223c6b6d2dd82a47ae5125a41", "sha256:dde816ca9dac1d9c01dd504ea5967821606f02e510438120091b84e852367428",
"sha256:d003156bb6a59cda9050e983441b7fa2487f7800d76bdc065566b7d728b4581a", "sha256:f2362f3cb0f3172c42938946dbc5b7843c2a28aec307c49100c8b38764eb6927",
"sha256:d175297e9533d8d37437abc14e8a83cbc68af93cc9c1c59c2c292ec59a0697a3", "sha256:f328adcfebed9f11301eaedfa48e15bdece9b519fb27e6a8c01aa52a17ec31b3",
"sha256:d746a437cdbca200622385305aedd9aef68e8a645e385cc483bdc5e488f07166", "sha256:f8afcf15cc511ada719a88e013cec87c11aff7b91f019295eb4530f96fe5ef2f",
"sha256:e683e409e5c45d5c9082dc1daf13f6374300806240719f95dc783d1fc942af10" "sha256:fb1bbeac803adea29cedd70781399c99138358c26d05fcbd23c13016b7f5ec65"
], ],
"markers": "python_version < '3.8' and implementation_name == 'cpython'", "index": "pypi",
"version": "==1.4.2" "version": "==1.4.3"
},
"types-requests": {
"hashes": [
"sha256:a5a305b43ea57bf64d6731f89816946a405b591eff6de28d4c0fd58422cee779",
"sha256:e21541c0f55c066c491a639309159556dd8c5833e49fcde929c4c47bdb0002ee"
],
"index": "pypi",
"version": "==2.25.6"
}, },
"typing-extensions": { "typing-extensions": {
"hashes": [ "hashes": [
"sha256:7cb407020f00f7bfc3cb3e7881628838e69d8f3fcab2f64742a5e76b2f841918", "sha256:0ac0f89795dd19de6b97debb0c6af1c70987fd80a2d62d1958f7e56fcc31b497",
"sha256:99d4073b617d30288f569d3f13d2bd7548c3a7e4c8de87db09a9d29bb3a4a60c", "sha256:50b6f157849174217d0656f99dc82fe932884fb250826c18350e159ec6cdf342",
"sha256:dafc7639cde7f1b6e1acc0f457842a83e722ccca8eef5270af2d74792619a89f" "sha256:779383f6086d90c99ae41cf0ff39aac8a7937a9283ce0a414e5dd782f4c94a84"
], ],
"index": "pypi", "index": "pypi",
"version": "==3.7.4.3" "version": "==3.10.0.0"
}, },
"urllib3": { "urllib3": {
"hashes": [ "hashes": [
"sha256:2f4da4594db7e1e110a944bb1b551fdf4e6c136ad42e4234131391e21eb5b0df", "sha256:39fb8672126159acb139a7718dd10806104dec1e2f0f6c88aab05d17df10c8d4",
"sha256:e7b021f7241115872f92f43c6508082facffbd1c048e3c6e2bb9c2a157e28937" "sha256:f57b4c16c62fa2760b7e3d97c35b255512fb6b59a259730f36ba32ce9f8e342f"
], ],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'", "version": "==1.26.6"
"version": "==1.26.4"
}, },
"wrapt": { "wrapt": {
"hashes": [ "hashes": [
"sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7" "sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7"
], ],
"version": "==1.12.1" "version": "==1.12.1"
},
"zipp": {
"hashes": [
"sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76",
"sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098"
],
"markers": "python_version >= '3.6'",
"version": "==3.4.1"
} }
} }
} }
+38 -18
View File
@@ -86,6 +86,16 @@ To later **update your local copy** of that profiles, you may run
If ``--fast-update`` is given, Instaloader stops when arriving at the If ``--fast-update`` is given, Instaloader stops when arriving at the
first already-downloaded picture. first already-downloaded picture.
Alternatively, you can use ``--latest-stamps`` to have Instaloader store
the time each profile was last downloaded and only download newer media:
::
instaloader --latest-stamps -- profile [profile ...]
With this option it's possible to move or delete downloaded media and still keep
the archive updated.
When updating profiles, Instaloader When updating profiles, Instaloader
automatically **detects profile name changes** and renames the target directory automatically **detects profile name changes** and renames the target directory
accordingly. accordingly.
@@ -104,6 +114,34 @@ already have a valid session cookie file.
`Instaloader Documentation <https://instaloader.github.io/basic-usage.html>`__ `Instaloader Documentation <https://instaloader.github.io/basic-usage.html>`__
Contributing
------------
As an open source project, Instaloader heavily depends on the contributions from
its community. See
`contributing <https://instaloader.github.io/contributing.html>`__
for how you may help Instaloader to become an even greater tool.
Supporters
----------
.. current-sponsors-start
See `Alex' GitHub Sponsors <https://github.com/sponsors/aandergr>`__ page for
how you can sponsor the development of Instaloader!
.. current-sponsors-end
It is a pleasure for us to share our Instaloader to the world, and we are proud
to have attracted such an active and motivating community, with so many users
who share their suggestions and ideas with us. Buying a community-sponsored beer
or coffee from time to time is very likely to further raise our passion for the
development of Instaloader.
| For Donations, we provide GitHub Sponsors page, a PayPal.Me link and a Bitcoin address.
| GitHub Sponsors: `Sponsor @aandergr on GitHub Sponsors <https://github.com/sponsors/aandergr>`__
| PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
| BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
Disclaimer Disclaimer
---------- ----------
@@ -116,21 +154,3 @@ subsidiaries. This is an independent and unofficial project. Use at your own ris
Instaloader is licensed under an MIT license. Refer to ``LICENSE`` file for more information. Instaloader is licensed under an MIT license. Refer to ``LICENSE`` file for more information.
.. disclaimer-end .. disclaimer-end
Contributing
------------
As an open source project, Instaloader heavily depends on the contributions from
its community. See
`contributing <https://instaloader.github.io/contributing.html>`__
for how you may help Instaloader to become an even greater tool.
It is a pleasure for us to share our Instaloader to the world, and we are proud
to have attracted such an active and motivating community, with so many users
who share their suggestions and ideas with us. Buying a community-sponsored beer
or coffee from time to time is very likely to further raise our passion for the
development of Instaloader.
| For Donations, we provide a PayPal.Me link and a Bitcoin address.
| PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
| BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
+1 -1
View File
@@ -6,7 +6,7 @@ pkgbase = instaloader
arch = any arch = any
license = MIT license = MIT
makedepends = python-setuptools makedepends = python-setuptools
depends = python>=3.5 depends = python>=3.6
depends = python-requests>=2.4 depends = python-requests>=2.4
options = !emptydirs options = !emptydirs
source = instaloader-{{version}}.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v{{version}} source = instaloader-{{version}}.tar.gz::https://codeload.github.com/instaloader/instaloader/tar.gz/v{{version}}
+2 -2
View File
@@ -44,8 +44,8 @@ with open('__main__.py', 'w+') as f:
f.writelines(lines) f.writelines(lines)
# install dependencies and invoke PyInstaller # install dependencies and invoke PyInstaller
commands = ["pip install pipenv==2020.11.15", commands = ["pip install pipenv==2021.5.29",
"pipenv sync --dev", f"pipenv --python {sys.version_info.major}.{sys.version_info.minor} sync --dev",
"pipenv run pyinstaller --log-level=DEBUG instaloader.spec"] "pipenv run pyinstaller --log-level=DEBUG instaloader.spec"]
for command in commands: for command in commands:
+31 -7
View File
@@ -39,6 +39,16 @@ To later **update your local copy** of that profiles, you may run
If :option:`--fast-update` is given, Instaloader stops when arriving at the If :option:`--fast-update` is given, Instaloader stops when arriving at the
first already-downloaded picture. first already-downloaded picture.
Alternatively, you can use :option:`--latest-stamps` to have Instaloader store
the time each profile was last downloaded and only download newer media:
::
instaloader --latest-stamps -- profile [profile ...]
With this option it's possible to move or delete downloaded media and still keep
the archive updated.
When updating profiles, Instaloader When updating profiles, Instaloader
automatically **detects profile name changes** and renames the target directory automatically **detects profile name changes** and renames the target directory
accordingly. accordingly.
@@ -149,27 +159,37 @@ target. The default is ``--dirname-pattern={target}``. In the dirname
pattern, the token ``{target}`` is replaced by the target name, and pattern, the token ``{target}`` is replaced by the target name, and
``{profile}`` is replaced by the owner of the post which is downloaded. ``{profile}`` is replaced by the owner of the post which is downloaded.
:option:`--filename-pattern` configures the path of the post's files relative :option:`--filename-pattern` configures the path of the post and story's files relative
to the target directory that is specified with :option:`--dirname-pattern`. to the target directory that is specified with :option:`--dirname-pattern`.
The default is ``--filename-pattern={date_utc}_UTC``. The default is ``--filename-pattern={date_utc}_UTC``.
The tokens ``{target}`` and ``{profile}`` are replaced like in the The tokens ``{target}`` and ``{profile}`` are replaced like in the
dirname pattern. The following tokens are defined for usage with dirname pattern.
:option:`--filename-pattern`:
:option:`--title-pattern` is similar to :option:`--filename-pattern`, but for profile
pics, hashtag profile pics, and highlight covers. The default is
``{date_utc}_UTC_{typename}`` if :option:`--dirname-pattern` contains ``{target}`` or
``{profile}``, or ``{target}_{date_utc}_UTC_{typename}`` if it does not. Some tokens
are not supported for this option, see below for details.
The following tokens are defined for usage with
:option:`--filename-pattern` and :option:`--title-pattern`:
- ``{target}`` - ``{target}``
Target name (as given in Instaloader command line) Target name (as given in Instaloader command line)
- ``{profile}`` (same as ``{owner_username}``) - ``{profile}`` (same as ``{owner_username}``)
Owner of the Post / StoryItem. Owner of the Post / StoryItem / ProfilePic. For hashtag profile pics and
highlight covers, equivalent to ``{target}``.
- ``{owner_id}`` - ``{owner_id}``
Unique integer ID of owner profile. Unique integer ID of owner profile. For hashtag profile pics, equivalent to
``{target}``.
- ``{shortcode}`` - ``{shortcode}``
Shortcode (identifier string). Shortcode (identifier string). Not available for :option:`--title-pattern`.
- ``{mediaid}`` - ``{mediaid}``
Integer representation of shortcode. Integer representation of shortcode. Not available for :option:`--title-pattern`.
- ``{filename}`` - ``{filename}``
Instagram's internal filename. Instagram's internal filename.
@@ -182,6 +202,10 @@ dirname pattern. The following tokens are defined for usage with
{date_utc:%Y-%m-%d_%H-%M-%S} {date_utc:%Y-%m-%d_%H-%M-%S}
- ``{typename}``
Type of media being saved, such as GraphImage, GraphStoryVideo, profile_pic,
etc.
For example, encode the poster's profile name in the filenames with:: For example, encode the poster's profile name in the filenames with::
instaloader --filename-pattern={date_utc}_UTC_{profile} "#hashtag" instaloader --filename-pattern={date_utc}_UTC_{profile} "#hashtag"
+38 -5
View File
@@ -100,7 +100,6 @@ What to Download of each Post
Do not xz compress JSON files, rather create pretty formatted JSONs. Do not xz compress JSON files, rather create pretty formatted JSONs.
What to Download of each Profile What to Download of each Profile
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -147,6 +146,23 @@ Which Posts to Download
This flag is recommended when you use Instaloader to update your personal This flag is recommended when you use Instaloader to update your personal
Instagram archive. Instagram archive.
.. option:: --latest-stamps [STAMPSFILE]
Works similarly to :option:`--fast-update`, but instead of relying on already
downloaded media, the time each profile was downloaded is stored, and only
media newer than the last download is fetched. This allows updating your
personal Instagram archive while emptying the target directories.
Only works for media associated with a specific profile, and that is returned
in chronological order: profile posts, profile stories, profile IGTV posts
and profile tagged posts.
By default, the information is stored in
``~/.config/instaloader/latest-stamps.ini``, but you can specify an
alternative location.
.. versionadded:: 4.8
.. option:: --post-filter filter, --only-if filter .. option:: --post-filter filter, --only-if filter
Expression that, if given, must evaluate to True for each post to be Expression that, if given, must evaluate to True for each post to be
@@ -164,8 +180,8 @@ Which Posts to Download
.. option:: --count COUNT, -c .. option:: --count COUNT, -c
Do not attempt to download more than COUNT posts. Applies only to Do not attempt to download more than COUNT posts. Applies to
``#hashtag``, ``%location id``, and ``:feed``. ``#hashtag``, ``%location_id``, ``:feed``, and ``:saved``.
Login (Download Private Profiles) Login (Download Private Profiles)
@@ -206,12 +222,23 @@ How to Download
.. option:: --filename-pattern FILENAME_PATTERN .. option:: --filename-pattern FILENAME_PATTERN
Prefix of filenames, relative to the directory given with Prefix of filenames for posts and stories, relative to the directory given with
:option:`--dirname-pattern`. ``{profile}`` is replaced by the profile name, :option:`--dirname-pattern`. ``{profile}`` is replaced by the profile name,
``{target}`` is replaced by the target you specified, i.e. either ``:feed``, ``{target}`` is replaced by the target you specified, i.e. either ``:feed``,
``#hashtag`` or the profile name. Defaults to ``{date_utc}_UTC``. ``#hashtag`` or the profile name. Defaults to ``{date_utc}_UTC``.
See :ref:`filename-specification` for a list of supported tokens. See :ref:`filename-specification` for a list of supported tokens.
.. option:: --title-pattern TITLE_PATTERN
Prefix of filenames for profile pics, hashtag profile pics, and highlight
covers, relative to the directory given with :option:`--dirname-pattern`.
Defaults to ``{date_utc}_UTC_{typename}`` if :option:`--dirname-pattern`
contains ``{target}`` or ``{profile}``, otherwise defaults to
``{target}_{date_utc}_UTC_{typename}``.
See :ref:`filename-specification` for a list of supported tokens.
.. versionadded:: 4.8
.. option:: --resume-prefix prefix .. option:: --resume-prefix prefix
For many targets, Instaloader is capable of resuming a previously-aborted For many targets, Instaloader is capable of resuming a previously-aborted
@@ -246,7 +273,7 @@ How to Download
.. option:: --user-agent USER_AGENT .. option:: --user-agent USER_AGENT
User Agent to use for HTTP requests. Per default, Instaloader pretends being User Agent to use for HTTP requests. Per default, Instaloader pretends being
Chrome/89 on Linux. Chrome/92 on Linux.
.. option:: --max-connection-attempts N .. option:: --max-connection-attempts N
@@ -274,6 +301,12 @@ How to Download
.. versionadded:: 4.7 .. versionadded:: 4.7
.. option:: --no-iphone
Do not attempt to download iPhone version of images and videos.
.. versionadded:: 4.8
Miscellaneous Options Miscellaneous Options
^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^
+5 -17
View File
@@ -132,21 +132,9 @@ details. If you feel the need to do so, please create a "feature suggestion".
be implemented and the underlying problem could be solved. Also **describe be implemented and the underlying problem could be solved. Also **describe
alternatives** that you have considered. alternatives** that you have considered.
Donations Sponsoring
--------- ----------
.. donations-start .. include:: sponsors.rst
:start-after: donations-start
It is a pleasure for us to share our Instaloader to the world, and we are proud :end-before: donations-end
to have attracted such an active and motivating community, with so many users
who share their suggestions and ideas with us. Buying a community-sponsored beer
or coffee from time to time is very likely to further raise our passion for the
development of Instaloader.
| For donations, we provide a PayPal.Me link and a Bitcoin address.
| `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
| BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
.. donations-end
.. (Discussion in :issue:`130`)
+11 -8
View File
@@ -78,13 +78,6 @@ Useful Links
- `Issue Tracker / Bug Tracker <https://github.com/instaloader/instaloader/issues>`__ - `Issue Tracker / Bug Tracker <https://github.com/instaloader/instaloader/issues>`__
- `Version History <https://github.com/instaloader/instaloader/releases>`__ - `Version History <https://github.com/instaloader/instaloader/releases>`__
Disclaimer
----------
.. include:: ../README.rst
:start-after: disclaimer-start
:end-before: disclaimer-end
Contributing Contributing
------------ ------------
@@ -92,10 +85,20 @@ As an open source project, Instaloader heavily depends on the contributions from
its community. See :ref:`contributing` for how you may help Instaloader to its community. See :ref:`contributing` for how you may help Instaloader to
become an even greater tool. become an even greater tool.
.. include:: contributing.rst Supporters
----------
.. include:: sponsors.rst
:start-after: donations-start :start-after: donations-start
:end-before: donations-end :end-before: donations-end
Disclaimer
----------
.. include:: ../README.rst
:start-after: disclaimer-start
:end-before: disclaimer-end
.. ..
* :ref:`genindex` * :ref:`genindex`
* :ref:`modindex` * :ref:`modindex`
+7
View File
@@ -25,6 +25,13 @@ Iterator :class:`NodeIterator` and a context manager
.. autoclass:: FrozenNodeIterator .. autoclass:: FrozenNodeIterator
:no-show-inheritance: :no-show-inheritance:
A serializable representation of a :class:`NodeIterator` instance, saving
its iteration state.
It can be serialized and deserialized with :func:`save_structure_to_file`
and :func:`load_structure_from_file`, as well as with :mod:`json` and
:mod:`pickle` thanks to being a :func:`~collections.namedtuple`.
``resumable_iteration`` ``resumable_iteration``
""""""""""""""""""""""" """""""""""""""""""""""
+10
View File
@@ -77,6 +77,16 @@ Loading and Saving
:class:`Post`, :class:`StoryItem`, :class:`Profile`, :class:`Hashtag` and :class:`Post`, :class:`StoryItem`, :class:`Profile`, :class:`Hashtag` and
:class:`FrozenNodeIterator` can be saved and loaded to/from JSON files. :class:`FrozenNodeIterator` can be saved and loaded to/from JSON files.
.. autofunction:: load_structure
.. autofunction:: load_structure_from_file .. autofunction:: load_structure_from_file
.. autofunction:: get_json_structure
.. autofunction:: save_structure_to_file .. autofunction:: save_structure_to_file
LatestStamps
""""""""""""
.. autoclass:: LatestStamps
:no-show-inheritance:
+23
View File
@@ -0,0 +1,23 @@
.. _sponsors:
Sponsoring the development of Instaloader
=========================================
.. donations-start
.. include:: ../README.rst
:start-after: current-sponsors-start
:end-before: current-sponsors-end
It is a pleasure for us to share our Instaloader to the world, and we are proud
to have attracted such an active and motivating community, with so many users
who share their suggestions and ideas with us. Buying a community-sponsored beer
or coffee from time to time is very likely to further raise our passion for the
development of Instaloader.
| For Donations, we provide GitHub Sponsors page, a PayPal.Me link and a Bitcoin address.
| GitHub Sponsors: `Sponsor @aandergr on GitHub Sponsors <https://github.com/sponsors/aandergr>`__
| PayPal: `PayPal.me/aandergr <https://www.paypal.me/aandergr>`__
| BTC: 1Nst4LoadeYzrKjJ1DX9CpbLXBYE9RKLwY
.. donations-end
+2 -1
View File
@@ -62,7 +62,8 @@ Two-Factor-Authentication. It also supports handling the *checkpoint challenge*,
issued when Instagram suspects authentication activity on your account, by issued when Instagram suspects authentication activity on your account, by
pointing the user to an URL to be opened in a browser. pointing the user to an URL to be opened in a browser.
Nevertheless, in :issue:`92` and :issue:`615` users reported problems with Nevertheless, in :issue:`92`, :issue:`615`, :issue:`1150` and :issue:`1217`,
users reported problems with
logging in. We recommend to always keep the session file which Instaloader logging in. We recommend to always keep the session file which Instaloader
creates when using :option:`--login`. If a session file is present, creates when using :option:`--login`. If a session file is present,
:option:`--login` does not make use of the failure-prone login procedure. :option:`--login` does not make use of the failure-prone login procedure.
+4 -2
View File
@@ -1,7 +1,7 @@
"""Download pictures (or videos) along with their captions and other metadata from Instagram.""" """Download pictures (or videos) along with their captions and other metadata from Instagram."""
__version__ = '4.7rc2' __version__ = '4.9b2'
try: try:
@@ -15,6 +15,8 @@ else:
from .exceptions import * from .exceptions import *
from .instaloader import Instaloader from .instaloader import Instaloader
from .instaloadercontext import InstaloaderContext, RateController from .instaloadercontext import InstaloaderContext, RateController
from .lateststamps import LatestStamps
from .nodeiterator import NodeIterator, FrozenNodeIterator, resumable_iteration from .nodeiterator import NodeIterator, FrozenNodeIterator, resumable_iteration
from .structures import (Hashtag, Highlight, Post, PostSidecarNode, PostComment, PostCommentAnswer, PostLocation, from .structures import (Hashtag, Highlight, Post, PostSidecarNode, PostComment, PostCommentAnswer, PostLocation,
Profile, Story, StoryItem, TopSearchResults, load_structure_from_file, save_structure_to_file) Profile, Story, StoryItem, TopSearchResults, load_structure_from_file, save_structure_to_file,
load_structure, get_json_structure)
+34 -11
View File
@@ -11,8 +11,9 @@ from typing import List, Optional
from . import (AbortDownloadException, BadCredentialsException, Instaloader, InstaloaderException, from . import (AbortDownloadException, BadCredentialsException, Instaloader, InstaloaderException,
InvalidArgumentException, Post, Profile, ProfileNotExistsException, StoryItem, InvalidArgumentException, Post, Profile, ProfileNotExistsException, StoryItem,
TwoFactorAuthRequiredException, __version__, load_structure_from_file) TwoFactorAuthRequiredException, __version__, load_structure_from_file)
from .instaloader import get_default_session_filename from .instaloader import (get_default_session_filename, get_default_stamps_filename)
from .instaloadercontext import default_user_agent from .instaloadercontext import default_user_agent
from .lateststamps import LatestStamps
def usage_string(): def usage_string():
@@ -76,6 +77,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
download_tagged: bool = False, download_tagged: bool = False,
download_igtv: bool = False, download_igtv: bool = False,
fast_update: bool = False, fast_update: bool = False,
latest_stamps_file: Optional[str] = None,
max_count: Optional[int] = None, post_filter_str: Optional[str] = None, max_count: Optional[int] = None, post_filter_str: Optional[str] = None,
storyitem_filter_str: Optional[str] = None) -> None: storyitem_filter_str: Optional[str] = None) -> None:
"""Download set of profiles, hashtags etc. and handle logging in and session files if desired.""" """Download set of profiles, hashtags etc. and handle logging in and session files if desired."""
@@ -88,6 +90,10 @@ def _main(instaloader: Instaloader, targetlist: List[str],
if storyitem_filter_str is not None: if storyitem_filter_str is not None:
storyitem_filter = filterstr_to_filterfunc(storyitem_filter_str, StoryItem) storyitem_filter = filterstr_to_filterfunc(storyitem_filter_str, StoryItem)
instaloader.context.log('Only download storyitems with property "{}".'.format(storyitem_filter_str)) instaloader.context.log('Only download storyitems with property "{}".'.format(storyitem_filter_str))
latest_stamps = None
if latest_stamps_file is not None:
latest_stamps = LatestStamps(latest_stamps_file)
instaloader.context.log(f"Using latest stamps from {latest_stamps_file}.")
# Login, if desired # Login, if desired
if username is not None: if username is not None:
if not re.match(r"^[A-Za-z0-9._]+$", username): if not re.match(r"^[A-Za-z0-9._]+$", username):
@@ -108,7 +114,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
code = input("Enter 2FA verification code: ") code = input("Enter 2FA verification code: ")
instaloader.two_factor_login(code) instaloader.two_factor_login(code)
break break
except BadCredentialsException: except BadCredentialsException as err:
print(err, file=sys.stderr)
pass pass
else: else:
instaloader.interactive_login(username) instaloader.interactive_login(username)
@@ -172,7 +179,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
post_filter=post_filter) post_filter=post_filter)
elif re.match(r"^[A-Za-z0-9._]+$", target): elif re.match(r"^[A-Za-z0-9._]+$", target):
try: try:
profile = instaloader.check_profile_id(target) profile = instaloader.check_profile_id(target, latest_stamps)
if instaloader.context.is_logged_in and profile.has_blocked_viewer: if instaloader.context.is_logged_in and profile.has_blocked_viewer:
if download_profile_pic or ((download_posts or download_tagged or download_igtv) if download_profile_pic or ((download_posts or download_tagged or download_igtv)
and not profile.is_private): and not profile.is_private):
@@ -191,7 +198,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
instaloader.context.log("Trying again anonymously, helps in case you are just blocked.") instaloader.context.log("Trying again anonymously, helps in case you are just blocked.")
with instaloader.anonymous_copy() as anonymous_loader: with instaloader.anonymous_copy() as anonymous_loader:
with instaloader.context.error_catcher(): with instaloader.context.error_catcher():
anonymous_retry_profiles.add(anonymous_loader.check_profile_id(target)) anonymous_retry_profiles.add(anonymous_loader.check_profile_id(target,
latest_stamps))
instaloader.context.error("Warning: {} will be downloaded anonymously (\"{}\")." instaloader.context.error("Warning: {} will be downloaded anonymously (\"{}\")."
.format(target, err)) .format(target, err))
else: else:
@@ -206,19 +214,21 @@ def _main(instaloader: Instaloader, targetlist: List[str],
if len(profiles) > 1: if len(profiles) > 1:
instaloader.context.log("Downloading {} profiles: {}".format(len(profiles), instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
' '.join([p.username for p in profiles]))) ' '.join([p.username for p in profiles])))
if profiles and (download_profile_pic or download_posts) and not instaloader.context.is_logged_in: if instaloader.context.iphone_support and profiles and (download_profile_pic or download_posts) and \
not instaloader.context.is_logged_in:
instaloader.context.log("Hint: Use --login to download higher-quality versions of pictures.") instaloader.context.log("Hint: Use --login to download higher-quality versions of pictures.")
instaloader.download_profiles(profiles, instaloader.download_profiles(profiles,
download_profile_pic, download_posts, download_tagged, download_igtv, download_profile_pic, download_posts, download_tagged, download_igtv,
download_highlights, download_stories, download_highlights, download_stories,
fast_update, post_filter, storyitem_filter) fast_update, post_filter, storyitem_filter, latest_stamps=latest_stamps)
if anonymous_retry_profiles: if anonymous_retry_profiles:
instaloader.context.log("Downloading anonymously: {}" instaloader.context.log("Downloading anonymously: {}"
.format(' '.join([p.username for p in anonymous_retry_profiles]))) .format(' '.join([p.username for p in anonymous_retry_profiles])))
with instaloader.anonymous_copy() as anonymous_loader: with instaloader.anonymous_copy() as anonymous_loader:
anonymous_loader.download_profiles(anonymous_retry_profiles, anonymous_loader.download_profiles(anonymous_retry_profiles,
download_profile_pic, download_posts, download_tagged, download_igtv, download_profile_pic, download_posts, download_tagged, download_igtv,
fast_update=fast_update, post_filter=post_filter) fast_update=fast_update, post_filter=post_filter,
latest_stamps=latest_stamps)
except KeyboardInterrupt: except KeyboardInterrupt:
print("\nInterrupted by user.", file=sys.stderr) print("\nInterrupted by user.", file=sys.stderr)
except AbortDownloadException as exc: except AbortDownloadException as exc:
@@ -323,7 +333,10 @@ def main():
g_cond.add_argument('-F', '--fast-update', action='store_true', g_cond.add_argument('-F', '--fast-update', action='store_true',
help='For each target, stop when encountering the first already-downloaded picture. This ' help='For each target, stop when encountering the first already-downloaded picture. This '
'flag is recommended when you use Instaloader to update your personal Instagram archive.') 'flag is recommended when you use Instaloader to update your personal Instagram archive.')
g_cond.add_argument('--latest-stamps', nargs='?', metavar='STAMPSFILE', const=get_default_stamps_filename(),
help='Store the timestamps of latest media scraped for each profile. This allows updating '
'your personal Instagram archive even if you delete the destination directories. '
'If STAMPSFILE is not provided, defaults to ' + get_default_stamps_filename())
g_cond.add_argument('--post-filter', '--only-if', metavar='filter', g_cond.add_argument('--post-filter', '--only-if', metavar='filter',
help='Expression that, if given, must evaluate to True for each post to be downloaded. Must be ' help='Expression that, if given, must evaluate to True for each post to be downloaded. Must be '
'a syntactically valid python expression. Variables are evaluated to ' 'a syntactically valid python expression. Variables are evaluated to '
@@ -335,7 +348,7 @@ def main():
g_cond.add_argument('-c', '--count', g_cond.add_argument('-c', '--count',
help='Do not attempt to download more than COUNT posts. ' help='Do not attempt to download more than COUNT posts. '
'Applies only to #hashtag and :feed.') 'Applies to #hashtag, %%location_id, :feed, and :saved.')
g_login = parser.add_argument_group('Login (Download Private Profiles)', g_login = parser.add_argument_group('Login (Download Private Profiles)',
'Instaloader can login to Instagram. This allows downloading private profiles. ' 'Instaloader can login to Instagram. This allows downloading private profiles. '
@@ -358,10 +371,15 @@ def main():
'{target} is replaced by the target you specified, i.e. either :feed, #hashtag or the ' '{target} is replaced by the target you specified, i.e. either :feed, #hashtag or the '
'profile name. Defaults to \'{target}\'.') 'profile name. Defaults to \'{target}\'.')
g_how.add_argument('--filename-pattern', g_how.add_argument('--filename-pattern',
help='Prefix of filenames, relative to the directory given with ' help='Prefix of filenames for posts and stories, relative to the directory given with '
'--dirname-pattern. {profile} is replaced by the profile name,' '--dirname-pattern. {profile} is replaced by the profile name,'
'{target} is replaced by the target you specified, i.e. either :feed' '{target} is replaced by the target you specified, i.e. either :feed'
'#hashtag or the profile name. Defaults to \'{date_utc}_UTC\'') '#hashtag or the profile name. Defaults to \'{date_utc}_UTC\'')
g_how.add_argument('--title-pattern',
help='Prefix of filenames for profile pics, hashtag profile pics, and highlight covers. '
'Defaults to \'{date_utc}_UTC_{typename}\' if --dirname-pattern contains \'{target}\' '
'or \'{dirname}\', or if --dirname-pattern is not specified. Otherwise defaults to '
'\'{target}_{date_utc}_UTC_{typename}\'.')
g_how.add_argument('--resume-prefix', metavar='PREFIX', g_how.add_argument('--resume-prefix', metavar='PREFIX',
help='Prefix for filenames that are used to save the information to resume an interrupted ' help='Prefix for filenames that are used to save the information to resume an interrupted '
'download.') 'download.')
@@ -382,6 +400,8 @@ def main():
g_how.add_argument('--abort-on', type=http_status_code_list, metavar="STATUS_CODES", g_how.add_argument('--abort-on', type=http_status_code_list, metavar="STATUS_CODES",
help='Comma-separated list of HTTP status codes that cause Instaloader to abort, bypassing all ' help='Comma-separated list of HTTP status codes that cause Instaloader to abort, bypassing all '
'retry logic.') 'retry logic.')
g_how.add_argument('--no-iphone', action='store_true',
help='Do not attempt to download iPhone version of images and videos.')
g_misc = parser.add_argument_group('Miscellaneous Options') g_misc = parser.add_argument_group('Miscellaneous Options')
g_misc.add_argument('-q', '--quiet', action='store_true', g_misc.add_argument('-q', '--quiet', action='store_true',
@@ -441,7 +461,9 @@ def main():
resume_prefix=resume_prefix, resume_prefix=resume_prefix,
check_resume_bbd=not args.use_aged_resume_files, check_resume_bbd=not args.use_aged_resume_files,
slide=args.slide, slide=args.slide,
fatal_status_codes=args.abort_on) fatal_status_codes=args.abort_on,
iphone_support=not args.no_iphone,
title_pattern=args.title_pattern)
_main(loader, _main(loader,
args.profile, args.profile,
username=args.login.lower() if args.login is not None else None, username=args.login.lower() if args.login is not None else None,
@@ -454,6 +476,7 @@ def main():
download_tagged=args.tagged, download_tagged=args.tagged,
download_igtv=args.igtv, download_igtv=args.igtv,
fast_update=args.fast_update, fast_update=args.fast_update,
latest_stamps_file=args.latest_stamps,
max_count=int(args.count) if args.count is not None else None, max_count=int(args.count) if args.count is not None else None,
post_filter_str=args.post_filter, post_filter_str=args.post_filter,
storyitem_filter_str=args.storyitem_filter) storyitem_filter_str=args.storyitem_filter)
+2
View File
@@ -65,6 +65,8 @@ class QueryReturnedNotFoundException(ConnectionException):
class TooManyRequestsException(ConnectionException): class TooManyRequestsException(ConnectionException):
pass pass
class IPhoneSupportDisabledException(InstaloaderException):
pass
class AbortDownloadException(Exception): class AbortDownloadException(Exception):
""" """
+225 -78
View File
@@ -10,7 +10,6 @@ import tempfile
from contextlib import contextmanager, suppress from contextlib import contextmanager, suppress
from datetime import datetime, timezone from datetime import datetime, timezone
from functools import wraps from functools import wraps
from hashlib import md5
from io import BytesIO from io import BytesIO
from pathlib import Path from pathlib import Path
from typing import Any, Callable, IO, Iterator, List, Optional, Set, Union, cast from typing import Any, Callable, IO, Iterator, List, Optional, Set, Union, cast
@@ -21,23 +20,30 @@ import urllib3 # type: ignore
from .exceptions import * from .exceptions import *
from .instaloadercontext import InstaloaderContext, RateController from .instaloadercontext import InstaloaderContext, RateController
from .lateststamps import LatestStamps
from .nodeiterator import NodeIterator, resumable_iteration from .nodeiterator import NodeIterator, resumable_iteration
from .sectioniterator import SectionIterator
from .structures import (Hashtag, Highlight, JsonExportable, Post, PostLocation, Profile, Story, StoryItem, from .structures import (Hashtag, Highlight, JsonExportable, Post, PostLocation, Profile, Story, StoryItem,
load_structure_from_file, save_structure_to_file, PostSidecarNode) load_structure_from_file, save_structure_to_file, PostSidecarNode, TitlePic)
def _get_config_dir() -> str:
if platform.system() == "Windows":
# on Windows, use %LOCALAPPDATA%\Instaloader
localappdata = os.getenv("LOCALAPPDATA")
if localappdata is not None:
return os.path.join(localappdata, "Instaloader")
# legacy fallback - store in temp dir if %LOCALAPPDATA% is not set
return os.path.join(tempfile.gettempdir(), ".instaloader-" + getpass.getuser())
# on Unix, use ~/.config/instaloader
return os.path.join(os.getenv("XDG_CONFIG_HOME", os.path.expanduser("~/.config")), "instaloader")
def get_default_session_filename(username: str) -> str: def get_default_session_filename(username: str) -> str:
"""Returns default session filename for given username.""" """Returns default session filename for given username."""
configdir = _get_config_dir()
sessionfilename = "session-{}".format(username) sessionfilename = "session-{}".format(username)
if platform.system() == "Windows": return os.path.join(configdir, sessionfilename)
# on Windows, use %LOCALAPPDATA%\Instaloader\session-USERNAME
localappdata = os.getenv("LOCALAPPDATA")
if localappdata is not None:
return os.path.join(localappdata, "Instaloader", sessionfilename)
# legacy fallback - store in temp dir if %LOCALAPPDATA% is not set
return os.path.join(tempfile.gettempdir(), ".instaloader-" + getpass.getuser(), sessionfilename)
# on Unix, use ~/.config/instaloader/session-USERNAME
return os.path.join(os.getenv("XDG_CONFIG_HOME", os.path.expanduser("~/.config")), "instaloader", sessionfilename)
def get_legacy_session_filename(username: str) -> str: def get_legacy_session_filename(username: str) -> str:
@@ -47,6 +53,17 @@ def get_legacy_session_filename(username: str) -> str:
return filename.lower() return filename.lower()
def get_default_stamps_filename() -> str:
"""
Returns default filename for latest stamps database.
.. versionadded:: 4.8
"""
configdir = _get_config_dir()
return os.path.join(configdir, "latest-stamps.ini")
def format_string_contains_key(format_string: str, key: str) -> bool: def format_string_contains_key(format_string: str, key: str) -> bool:
# pylint:disable=unused-variable # pylint:disable=unused-variable
for literal_text, field_name, format_spec, conversion in string.Formatter().parse(format_string): for literal_text, field_name, format_spec, conversion in string.Formatter().parse(format_string):
@@ -102,7 +119,7 @@ class _ArbitraryItemFormatter(string.Formatter):
def get_value(self, key, args, kwargs): def get_value(self, key, args, kwargs):
"""Override to substitute {ATTRIBUTE} by attributes of our _item.""" """Override to substitute {ATTRIBUTE} by attributes of our _item."""
if key == 'filename' and isinstance(self._item, (Post, StoryItem, PostSidecarNode)): if key == 'filename' and isinstance(self._item, (Post, StoryItem, PostSidecarNode, TitlePic)):
return "{filename}" return "{filename}"
if hasattr(self._item, key): if hasattr(self._item, key):
return getattr(self._item, key) return getattr(self._item, key)
@@ -130,6 +147,10 @@ class _PostPathFormatter(_ArbitraryItemFormatter):
def sanitize_path(ret: str) -> str: def sanitize_path(ret: str) -> str:
"""Replaces '/' with similar looking Division Slash and some other illegal filename characters on Windows.""" """Replaces '/' with similar looking Division Slash and some other illegal filename characters on Windows."""
ret = ret.replace('/', '\u2215') ret = ret.replace('/', '\u2215')
if ret.startswith('.'):
ret = ret.replace('.', '\u2024', 1)
if platform.system() == 'Windows': if platform.system() == 'Windows':
ret = ret.replace(':', '\uff1a').replace('<', '\ufe64').replace('>', '\ufe65').replace('\"', '\uff02') ret = ret.replace(':', '\uff1a').replace('<', '\ufe64').replace('>', '\ufe65').replace('\"', '\uff02')
ret = ret.replace('\\', '\ufe68').replace('|', '\uff5c').replace('?', '\ufe16').replace('*', '\uff0a') ret = ret.replace('\\', '\ufe68').replace('|', '\uff5c').replace('?', '\ufe16').replace('*', '\uff0a')
@@ -144,6 +165,9 @@ class Instaloader:
:param user_agent: :option:`--user-agent` :param user_agent: :option:`--user-agent`
:param dirname_pattern: :option:`--dirname-pattern`, default is ``{target}`` :param dirname_pattern: :option:`--dirname-pattern`, default is ``{target}``
:param filename_pattern: :option:`--filename-pattern`, default is ``{date_utc}_UTC`` :param filename_pattern: :option:`--filename-pattern`, default is ``{date_utc}_UTC``
:param title_pattern:
:option:`--title-pattern`, default is ``{date_utc}_UTC_{typename}`` if ``dirname_pattern`` contains
``{target}`` or ``{profile}``, ``{target}_{date_utc}_UTC_{typename}`` otherwise.
:param download_pictures: not :option:`--no-pictures` :param download_pictures: not :option:`--no-pictures`
:param download_videos: not :option:`--no-videos` :param download_videos: not :option:`--no-videos`
:param download_video_thumbnails: not :option:`--no-video-thumbnails` :param download_video_thumbnails: not :option:`--no-video-thumbnails`
@@ -162,6 +186,7 @@ class Instaloader:
:param check_resume_bbd: Whether to check the date of expiry of resume files and reject them if expired. :param check_resume_bbd: Whether to check the date of expiry of resume files and reject them if expired.
:param slide: :option:`--slide` :param slide: :option:`--slide`
:param fatal_status_codes: :option:`--abort-on` :param fatal_status_codes: :option:`--abort-on`
:param iphone_support: not :option:`--no-iphone`
.. attribute:: context .. attribute:: context
@@ -189,14 +214,25 @@ class Instaloader:
resume_prefix: Optional[str] = "iterator", resume_prefix: Optional[str] = "iterator",
check_resume_bbd: bool = True, check_resume_bbd: bool = True,
slide: Optional[str] = None, slide: Optional[str] = None,
fatal_status_codes: Optional[List[int]] = None): fatal_status_codes: Optional[List[int]] = None,
iphone_support: bool = True,
title_pattern: Optional[str] = None):
self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts, self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts,
request_timeout, rate_controller, fatal_status_codes) request_timeout, rate_controller, fatal_status_codes,
iphone_support)
# configuration parameters # configuration parameters
self.dirname_pattern = dirname_pattern or "{target}" self.dirname_pattern = dirname_pattern or "{target}"
self.filename_pattern = filename_pattern or "{date_utc}_UTC" self.filename_pattern = filename_pattern or "{date_utc}_UTC"
if title_pattern is not None:
self.title_pattern = title_pattern
else:
if (format_string_contains_key(self.dirname_pattern, 'profile') or
format_string_contains_key(self.dirname_pattern, 'target')):
self.title_pattern = '{date_utc}_UTC_{typename}'
else:
self.title_pattern = '{target}_{date_utc}_UTC_{typename}'
self.download_pictures = download_pictures self.download_pictures = download_pictures
self.download_videos = download_videos self.download_videos = download_videos
self.download_video_thumbnails = download_video_thumbnails self.download_video_thumbnails = download_video_thumbnails
@@ -259,7 +295,8 @@ class Instaloader:
resume_prefix=self.resume_prefix, resume_prefix=self.resume_prefix,
check_resume_bbd=self.check_resume_bbd, check_resume_bbd=self.check_resume_bbd,
slide=self.slide, slide=self.slide,
fatal_status_codes=self.context.fatal_status_codes) fatal_status_codes=self.context.fatal_status_codes,
iphone_support=self.context.iphone_support)
yield new_loader yield new_loader
self.context.error_log.extend(new_loader.context.error_log) self.context.error_log.extend(new_loader.context.error_log)
new_loader.context.error_log = [] # avoid double-printing of errors new_loader.context.error_log = [] # avoid double-printing of errors
@@ -421,9 +458,12 @@ class Instaloader:
def save_location(self, filename: str, location: PostLocation, mtime: datetime) -> None: def save_location(self, filename: str, location: PostLocation, mtime: datetime) -> None:
"""Save post location name and Google Maps link.""" """Save post location name and Google Maps link."""
filename += '_location.txt' filename += '_location.txt'
location_string = (location.name + "\n" + if location.lat is not None and location.lng is not None:
"https://maps.google.com/maps?q={0},{1}&ll={0},{1}\n".format(location.lat, location_string = (location.name + "\n" +
location.lng)) "https://maps.google.com/maps?q={0},{1}&ll={0},{1}\n".format(location.lat,
location.lng))
else:
location_string = location.name
with open(filename, 'wb') as text_file: with open(filename, 'wb') as text_file:
with BytesIO(location_string.encode()) as bio: with BytesIO(location_string.encode()) as bio:
shutil.copyfileobj(cast(IO, bio), text_file) shutil.copyfileobj(cast(IO, bio), text_file)
@@ -456,31 +496,51 @@ class Instaloader:
.. versionadded:: 4.3""" .. versionadded:: 4.3"""
def _epoch_to_string(epoch: datetime) -> str:
return epoch.strftime('%Y-%m-%d_%H-%M-%S_UTC')
http_response = self.context.get_raw(url) http_response = self.context.get_raw(url)
date_object = None # type: Optional[datetime] date_object = None # type: Optional[datetime]
if 'Last-Modified' in http_response.headers: if 'Last-Modified' in http_response.headers:
date_object = datetime.strptime(http_response.headers["Last-Modified"], '%a, %d %b %Y %H:%M:%S GMT') date_object = datetime.strptime(http_response.headers["Last-Modified"], '%a, %d %b %Y %H:%M:%S GMT')
date_object = date_object.replace(tzinfo=timezone.utc)
pic_bytes = None pic_bytes = None
pic_identifier = _epoch_to_string(date_object)
else: else:
pic_bytes = http_response.content pic_bytes = http_response.content
pic_identifier = md5(pic_bytes).hexdigest()[:16] ig_filename = url.split('/')[-1].split('?')[0]
filename = self.format_filename_within_target_path(target, owner_profile, pic_identifier, name_suffix, 'jpg') pic_data = TitlePic(owner_profile, target, name_suffix, ig_filename, date_object)
dirname = _PostPathFormatter(pic_data).format(self.dirname_pattern, target=target)
filename_template = os.path.join(dirname,
_PostPathFormatter(pic_data).format(self.title_pattern, target=target))
filename = self.__prepare_filename(filename_template, lambda: url) + ".jpg"
content_length = http_response.headers.get('Content-Length', None) content_length = http_response.headers.get('Content-Length', None)
if os.path.isfile(filename) and (not self.context.is_logged_in or if os.path.isfile(filename) and (not self.context.is_logged_in or
(content_length is not None and (content_length is not None and
os.path.getsize(filename) >= int(content_length))): os.path.getsize(filename) >= int(content_length))):
self.context.log(filename + ' already exists') self.context.log(filename + ' already exists')
return None return
os.makedirs(os.path.dirname(filename), exist_ok=True) os.makedirs(os.path.dirname(filename), exist_ok=True)
self.context.write_raw(pic_bytes if pic_bytes else http_response, filename) self.context.write_raw(pic_bytes if pic_bytes else http_response, filename)
if date_object: if date_object:
os.utime(filename, (datetime.now().timestamp(), date_object.timestamp())) os.utime(filename, (datetime.now().timestamp(), date_object.timestamp()))
self.context.log('') # log output of _get_and_write_raw() does not produce \n self.context.log('') # log output of _get_and_write_raw() does not produce \n
def download_profilepic_if_new(self, profile: Profile, latest_stamps: Optional[LatestStamps]) -> None:
"""
Downloads and saves profile pic if it has not been downloaded before.
:param latest_stamps: Database with the last downloaded data. If not present,
the profile pic is downloaded unless it already exists
.. versionadded:: 4.8
"""
if latest_stamps is None:
self.download_profilepic(profile)
return
profile_pic_basename = profile.profile_pic_url.split('/')[-1].split('?')[0]
saved_basename = latest_stamps.get_profile_pic(profile.username)
if saved_basename == profile_pic_basename:
return
self.download_profilepic(profile)
latest_stamps.set_profile_pic(profile.username, profile_pic_basename)
def download_profilepic(self, profile: Profile) -> None: def download_profilepic(self, profile: Profile) -> None:
"""Downloads and saves profile pic.""" """Downloads and saves profile pic."""
self.download_title_pic(profile.profile_pic_url, profile.username.lower(), 'profile_pic', profile) self.download_title_pic(profile.profile_pic_url, profile.username.lower(), 'profile_pic', profile)
@@ -568,7 +628,8 @@ class Instaloader:
os.makedirs(os.path.dirname(filename), exist_ok=True) os.makedirs(os.path.dirname(filename), exist_ok=True)
return filename return filename
def format_filename(self, item: Union[Post, StoryItem, PostSidecarNode], target: Optional[Union[str, Path]] = None): def format_filename(self, item: Union[Post, StoryItem, PostSidecarNode, TitlePic],
target: Optional[Union[str, Path]] = None):
"""Format filename of a :class:`Post` or :class:`StoryItem` according to ``filename-pattern`` parameter. """Format filename of a :class:`Post` or :class:`StoryItem` according to ``filename-pattern`` parameter.
.. versionadded:: 4.1""" .. versionadded:: 4.1"""
@@ -714,7 +775,8 @@ class Instaloader:
userids: Optional[List[Union[int, Profile]]] = None, userids: Optional[List[Union[int, Profile]]] = None,
fast_update: bool = False, fast_update: bool = False,
filename_target: Optional[str] = ':stories', filename_target: Optional[str] = ':stories',
storyitem_filter: Optional[Callable[[StoryItem], bool]] = None) -> None: storyitem_filter: Optional[Callable[[StoryItem], bool]] = None,
latest_stamps: Optional[LatestStamps] = None) -> None:
""" """
Download available stories from user followees or all stories of users whose ID are given. Download available stories from user followees or all stories of users whose ID are given.
Does not mark stories as seen. Does not mark stories as seen.
@@ -725,20 +787,37 @@ class Instaloader:
:param filename_target: Replacement for {target} in dirname_pattern and filename_pattern :param filename_target: Replacement for {target} in dirname_pattern and filename_pattern
or None if profile name should be used instead or None if profile name should be used instead
:param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded :param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded
:param latest_stamps: Database with the last times each user was scraped
:raises LoginRequiredException: If called without being logged in. :raises LoginRequiredException: If called without being logged in.
.. versionchanged:: 4.8
Add `latest_stamps` parameter.
""" """
if not userids: if not userids:
self.context.log("Retrieving all visible stories...") self.context.log("Retrieving all visible stories...")
profile_count = None
else: else:
userids = [p if isinstance(p, int) else p.userid for p in userids] userids = [p if isinstance(p, int) else p.userid for p in userids]
profile_count = len(userids)
for user_story in self.get_stories(userids): for i, user_story in enumerate(self.get_stories(userids), start=1):
name = user_story.owner_username name = user_story.owner_username
self.context.log("Retrieving stories from profile {}.".format(name)) if profile_count is not None:
msg = "[{0:{w}d}/{1:{w}d}] Retrieving stories from profile {2}.".format(i, profile_count, name,
w=len(str(profile_count)))
else:
msg = "[{:3d}] Retrieving stories from profile {}.".format(i, name)
self.context.log(msg)
totalcount = user_story.itemcount totalcount = user_story.itemcount
count = 1 count = 1
if latest_stamps is not None:
# pylint:disable=cell-var-from-loop
last_scraped = latest_stamps.get_last_story_timestamp(name)
scraped_timestamp = datetime.now().astimezone()
for item in user_story.get_items(): for item in user_story.get_items():
if latest_stamps is not None and item.date_local <= last_scraped:
break
if storyitem_filter is not None and not storyitem_filter(item): if storyitem_filter is not None and not storyitem_filter(item):
self.context.log("<{} skipped>".format(item), flush=True) self.context.log("<{} skipped>".format(item), flush=True)
continue continue
@@ -748,6 +827,8 @@ class Instaloader:
downloaded = self.download_storyitem(item, filename_target if filename_target else name) downloaded = self.download_storyitem(item, filename_target if filename_target else name)
if fast_update and not downloaded: if fast_update and not downloaded:
break break
if latest_stamps is not None:
latest_stamps.set_last_story_timestamp(name, scraped_timestamp)
def download_storyitem(self, item: StoryItem, target: Union[str, Path]) -> bool: def download_storyitem(self, item: StoryItem, target: Union[str, Path]) -> bool:
"""Download one user story. """Download one user story.
@@ -858,7 +939,8 @@ class Instaloader:
post_filter: Optional[Callable[[Post], bool]] = None, post_filter: Optional[Callable[[Post], bool]] = None,
max_count: Optional[int] = None, max_count: Optional[int] = None,
total_count: Optional[int] = None, total_count: Optional[int] = None,
owner_profile: Optional[Profile] = None) -> None: owner_profile: Optional[Profile] = None,
takewhile: Optional[Callable[[Post], bool]] = None) -> None:
""" """
Download the Posts returned by given Post Iterator. Download the Posts returned by given Post Iterator.
@@ -867,6 +949,9 @@ class Instaloader:
.. versionchanged:: 4.5 .. versionchanged:: 4.5
Transparently resume an aborted operation if `posts` is a :class:`NodeIterator`. Transparently resume an aborted operation if `posts` is a :class:`NodeIterator`.
.. versionchanged:: 4.8
Add `takewhile` parameter.
:param posts: Post Iterator to loop through. :param posts: Post Iterator to loop through.
:param target: Target name. :param target: Target name.
:param fast_update: :option:`--fast-update`. :param fast_update: :option:`--fast-update`.
@@ -874,12 +959,15 @@ class Instaloader:
:param max_count: Maximum count of Posts to download (:option:`--count`). :param max_count: Maximum count of Posts to download (:option:`--count`).
:param total_count: Total number of posts returned by given iterator. :param total_count: Total number of posts returned by given iterator.
:param owner_profile: Associated profile, if any. :param owner_profile: Associated profile, if any.
:param takewhile: Expression evaluated for each post. Once it returns false, downloading stops.
""" """
displayed_count = (max_count if total_count is None or max_count is not None and max_count < total_count displayed_count = (max_count if total_count is None or max_count is not None and max_count < total_count
else total_count) else total_count)
sanitized_target = target sanitized_target = target
if isinstance(target, str): if isinstance(target, str):
sanitized_target = _PostPathFormatter.sanitize_path(target) sanitized_target = _PostPathFormatter.sanitize_path(target)
if takewhile is None:
takewhile = lambda _: True
with resumable_iteration( with resumable_iteration(
context=self.context, context=self.context,
iterator=posts, iterator=posts,
@@ -892,7 +980,7 @@ class Instaloader:
enabled=self.resume_prefix is not None enabled=self.resume_prefix is not None
) as (is_resuming, start_index): ) as (is_resuming, start_index):
for number, post in enumerate(posts, start=start_index + 1): for number, post in enumerate(posts, start=start_index + 1):
if max_count is not None and number > max_count: if (max_count is not None and number > max_count) or not takewhile(post):
break break
if displayed_count is not None: if displayed_count is not None:
self.context.log("[{0:{w}d}/{1:{w}d}] ".format(number, displayed_count, self.context.log("[{0:{w}d}/{1:{w}d}] ".format(number, displayed_count,
@@ -1004,18 +1092,12 @@ class Instaloader:
.. versionchanged:: 4.2.9 .. versionchanged:: 4.2.9
Require being logged in (as required by Instagram) Require being logged in (as required by Instagram)
""" """
has_next_page = True yield from SectionIterator(
end_cursor = None self.context,
while has_next_page: lambda d: d["native_location_data"]["recent"],
if end_cursor: lambda m: Post.from_iphone_struct(self.context, m),
params = {'__a': 1, 'max_id': end_cursor} f"explore/locations/{location}/",
else: )
params = {'__a': 1}
location_data = self.context.get_json('explore/locations/{0}/'.format(location),
params)['graphql']['location']['edge_location_to_media']
yield from (Post(self.context, edge['node']) for edge in location_data['edges'])
has_next_page = location_data['page_info']['has_next_page']
end_cursor = location_data['page_info']['end_cursor']
@_requires_login @_requires_login
def download_location(self, location: str, def download_location(self, location: str,
@@ -1111,25 +1193,47 @@ class Instaloader:
def download_tagged(self, profile: Profile, fast_update: bool = False, def download_tagged(self, profile: Profile, fast_update: bool = False,
target: Optional[str] = None, target: Optional[str] = None,
post_filter: Optional[Callable[[Post], bool]] = None) -> None: post_filter: Optional[Callable[[Post], bool]] = None,
latest_stamps: Optional[LatestStamps] = None) -> None:
"""Download all posts where a profile is tagged. """Download all posts where a profile is tagged.
.. versionadded:: 4.1""" .. versionadded:: 4.1
.. versionchanged:: 4.8
Add `latest_stamps` parameter."""
self.context.log("Retrieving tagged posts for profile {}.".format(profile.username)) self.context.log("Retrieving tagged posts for profile {}.".format(profile.username))
self.posts_download_loop(profile.get_tagged_posts(), posts_takewhile: Optional[Callable[[Post], bool]] = None
if latest_stamps is not None:
last_scraped = latest_stamps.get_last_tagged_timestamp(profile.username)
posts_takewhile = lambda p: p.date_local > last_scraped
tagged_posts = profile.get_tagged_posts()
self.posts_download_loop(tagged_posts,
target if target target if target
else (Path(_PostPathFormatter.sanitize_path(profile.username)) / else (Path(_PostPathFormatter.sanitize_path(profile.username)) /
_PostPathFormatter.sanitize_path(':tagged')), _PostPathFormatter.sanitize_path(':tagged')),
fast_update, post_filter) fast_update, post_filter, takewhile=posts_takewhile)
if latest_stamps is not None and tagged_posts.first_item is not None:
latest_stamps.set_last_tagged_timestamp(profile.username, tagged_posts.first_item.date_local)
def download_igtv(self, profile: Profile, fast_update: bool = False, def download_igtv(self, profile: Profile, fast_update: bool = False,
post_filter: Optional[Callable[[Post], bool]] = None) -> None: post_filter: Optional[Callable[[Post], bool]] = None,
latest_stamps: Optional[LatestStamps] = None) -> None:
"""Download IGTV videos of a profile. """Download IGTV videos of a profile.
.. versionadded:: 4.3""" .. versionadded:: 4.3
.. versionchanged:: 4.8
Add `latest_stamps` parameter."""
self.context.log("Retrieving IGTV videos for profile {}.".format(profile.username)) self.context.log("Retrieving IGTV videos for profile {}.".format(profile.username))
self.posts_download_loop(profile.get_igtv_posts(), profile.username, fast_update, post_filter, posts_takewhile: Optional[Callable[[Post], bool]] = None
total_count=profile.igtvcount, owner_profile=profile) if latest_stamps is not None:
last_scraped = latest_stamps.get_last_igtv_timestamp(profile.username)
posts_takewhile = lambda p: p.date_local > last_scraped
igtv_posts = profile.get_igtv_posts()
self.posts_download_loop(igtv_posts, profile.username, fast_update, post_filter,
total_count=profile.igtvcount, owner_profile=profile, takewhile=posts_takewhile)
if latest_stamps is not None and igtv_posts.first_item is not None:
latest_stamps.set_last_igtv_timestamp(profile.username, igtv_posts.first_item.date_local)
def _get_id_filename(self, profile_name: str) -> str: def _get_id_filename(self, profile_name: str) -> str:
if ((format_string_contains_key(self.dirname_pattern, 'profile') or if ((format_string_contains_key(self.dirname_pattern, 'profile') or
@@ -1141,9 +1245,22 @@ class Instaloader:
return os.path.join(self.dirname_pattern.format(), return os.path.join(self.dirname_pattern.format(),
'{0}_id'.format(profile_name.lower())) '{0}_id'.format(profile_name.lower()))
def load_profile_id(self, profile_name: str) -> Optional[int]:
"""
Load ID of profile from profile directory.
.. versionadded:: 4.8
"""
id_filename = self._get_id_filename(profile_name)
try:
with open(id_filename, 'rb') as id_file:
return int(id_file.read())
except (FileNotFoundError, ValueError):
return None
def save_profile_id(self, profile: Profile): def save_profile_id(self, profile: Profile):
""" """
Store ID of profile locally. Store ID of profile on profile directory.
.. versionadded:: 4.0.6 .. versionadded:: 4.0.6
""" """
@@ -1153,13 +1270,18 @@ class Instaloader:
text_file.write(str(profile.userid) + "\n") text_file.write(str(profile.userid) + "\n")
self.context.log("Stored ID {0} for profile {1}.".format(profile.userid, profile.username)) self.context.log("Stored ID {0} for profile {1}.".format(profile.userid, profile.username))
def check_profile_id(self, profile_name: str) -> Profile: def check_profile_id(self, profile_name: str, latest_stamps: Optional[LatestStamps] = None) -> Profile:
""" """
Consult locally stored ID of profile with given name, check whether ID matches and whether name Consult locally stored ID of profile with given name, check whether ID matches and whether name
has changed and return current name of the profile, and store ID of profile. has changed and return current name of the profile, and store ID of profile.
:param profile_name: Profile name :param profile_name: Profile name
:param latest_stamps: Database of downloaded data. If present, IDs are retrieved from it,
otherwise from the target directory
:return: Instance of current profile :return: Instance of current profile
.. versionchanged:: 4.8
Add `latest_stamps` parameter.
""" """
profile = None profile = None
profile_name_not_exists_err = None profile_name_not_exists_err = None
@@ -1167,10 +1289,11 @@ class Instaloader:
profile = Profile.from_username(self.context, profile_name) profile = Profile.from_username(self.context, profile_name)
except ProfileNotExistsException as err: except ProfileNotExistsException as err:
profile_name_not_exists_err = err profile_name_not_exists_err = err
id_filename = self._get_id_filename(profile_name) if latest_stamps is None:
try: profile_id = self.load_profile_id(profile_name)
with open(id_filename, 'rb') as id_file: else:
profile_id = int(id_file.read()) profile_id = latest_stamps.get_profile_id(profile_name)
if profile_id is not None:
if (profile is None) or \ if (profile is None) or \
(profile_id != profile.userid): (profile_id != profile.userid):
if profile is not None: if profile is not None:
@@ -1181,23 +1304,27 @@ class Instaloader:
profile_id)) profile_id))
profile_from_id = Profile.from_id(self.context, profile_id) profile_from_id = Profile.from_id(self.context, profile_id)
newname = profile_from_id.username newname = profile_from_id.username
self.context.log("Profile {0} has changed its name to {1}.".format(profile_name, newname)) self.context.error("Profile {0} has changed its name to {1}.".format(profile_name, newname))
if ((format_string_contains_key(self.dirname_pattern, 'profile') or if latest_stamps is None:
format_string_contains_key(self.dirname_pattern, 'target'))): if ((format_string_contains_key(self.dirname_pattern, 'profile') or
os.rename(self.dirname_pattern.format(profile=profile_name.lower(), format_string_contains_key(self.dirname_pattern, 'target'))):
target=profile_name.lower()), os.rename(self.dirname_pattern.format(profile=profile_name.lower(),
self.dirname_pattern.format(profile=newname.lower(), target=profile_name.lower()),
target=newname.lower())) self.dirname_pattern.format(profile=newname.lower(),
target=newname.lower()))
else:
os.rename('{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower()),
'{0}/{1}_id'.format(self.dirname_pattern.format(), newname.lower()))
else: else:
os.rename('{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower()), latest_stamps.rename_profile(profile_name, newname)
'{0}/{1}_id'.format(self.dirname_pattern.format(), newname.lower()))
return profile_from_id return profile_from_id
# profile exists and profile id matches saved id # profile exists and profile id matches saved id
return profile return profile
except (FileNotFoundError, ValueError):
pass
if profile is not None: if profile is not None:
self.save_profile_id(profile) if latest_stamps is None:
self.save_profile_id(profile)
else:
latest_stamps.save_profile_id(profile.username, profile.userid)
return profile return profile
if profile_name_not_exists_err: if profile_name_not_exists_err:
raise profile_name_not_exists_err raise profile_name_not_exists_err
@@ -1212,7 +1339,8 @@ class Instaloader:
fast_update: bool = False, fast_update: bool = False,
post_filter: Optional[Callable[[Post], bool]] = None, post_filter: Optional[Callable[[Post], bool]] = None,
storyitem_filter: Optional[Callable[[Post], bool]] = None, storyitem_filter: Optional[Callable[[Post], bool]] = None,
raise_errors: bool = False): raise_errors: bool = False,
latest_stamps: Optional[LatestStamps] = None):
"""High-level method to download set of profiles. """High-level method to download set of profiles.
:param profiles: Set of profiles to download. :param profiles: Set of profiles to download.
@@ -1228,11 +1356,15 @@ class Instaloader:
:param raise_errors: :param raise_errors:
Whether :exc:`LoginRequiredException` and :exc:`PrivateProfileNotFollowedException` should be raised or Whether :exc:`LoginRequiredException` and :exc:`PrivateProfileNotFollowedException` should be raised or
catched and printed with :meth:`InstaloaderContext.error_catcher`. catched and printed with :meth:`InstaloaderContext.error_catcher`.
:param latest_stamps: :option:`--latest-stamps`.
.. versionadded:: 4.1 .. versionadded:: 4.1
.. versionchanged:: 4.3 .. versionchanged:: 4.3
Add `igtv` parameter. Add `igtv` parameter.
.. versionchanged:: 4.8
Add `latest_stamps` parameter.
""" """
@contextmanager @contextmanager
@@ -1242,14 +1374,16 @@ class Instaloader:
# error_handler type is Callable[[Optional[str]], ContextManager[None]] (not supported with Python 3.5.0..3.5.3) # error_handler type is Callable[[Optional[str]], ContextManager[None]] (not supported with Python 3.5.0..3.5.3)
error_handler = _error_raiser if raise_errors else self.context.error_catcher error_handler = _error_raiser if raise_errors else self.context.error_catcher
for profile in profiles: for i, profile in enumerate(profiles, start=1):
self.context.log("[{0:{w}d}/{1:{w}d}] Downloading profile {2}".format(i, len(profiles), profile.username,
w=len(str(len(profiles)))))
with error_handler(profile.username): # type: ignore # (ignore type for Python 3.5 support) with error_handler(profile.username): # type: ignore # (ignore type for Python 3.5 support)
profile_name = profile.username profile_name = profile.username
# Download profile picture # Download profile picture
if profile_pic: if profile_pic:
with self.context.error_catcher('Download profile picture of {}'.format(profile_name)): with self.context.error_catcher('Download profile picture of {}'.format(profile_name)):
self.download_profilepic(profile) self.download_profilepic_if_new(profile, latest_stamps)
# Save metadata as JSON if desired. # Save metadata as JSON if desired.
if self.save_metadata: if self.save_metadata:
@@ -1271,12 +1405,14 @@ class Instaloader:
# Download tagged, if requested # Download tagged, if requested
if tagged: if tagged:
with self.context.error_catcher('Download tagged of {}'.format(profile_name)): with self.context.error_catcher('Download tagged of {}'.format(profile_name)):
self.download_tagged(profile, fast_update=fast_update, post_filter=post_filter) self.download_tagged(profile, fast_update=fast_update, post_filter=post_filter,
latest_stamps=latest_stamps)
# Download IGTV, if requested # Download IGTV, if requested
if igtv: if igtv:
with self.context.error_catcher('Download IGTV of {}'.format(profile_name)): with self.context.error_catcher('Download IGTV of {}'.format(profile_name)):
self.download_igtv(profile, fast_update=fast_update, post_filter=post_filter) self.download_igtv(profile, fast_update=fast_update, post_filter=post_filter,
latest_stamps=latest_stamps)
# Download highlights, if requested # Download highlights, if requested
if highlights: if highlights:
@@ -1286,14 +1422,24 @@ class Instaloader:
# Iterate over pictures and download them # Iterate over pictures and download them
if posts: if posts:
self.context.log("Retrieving posts from profile {}.".format(profile_name)) self.context.log("Retrieving posts from profile {}.".format(profile_name))
self.posts_download_loop(profile.get_posts(), profile_name, fast_update, post_filter, posts_takewhile: Optional[Callable[[Post], bool]] = None
total_count=profile.mediacount, owner_profile=profile) if latest_stamps is not None:
# pylint:disable=cell-var-from-loop
last_scraped = latest_stamps.get_last_post_timestamp(profile_name)
posts_takewhile = lambda p: p.date_local > last_scraped
posts_to_download = profile.get_posts()
self.posts_download_loop(posts_to_download, profile_name, fast_update, post_filter,
total_count=profile.mediacount, owner_profile=profile,
takewhile=posts_takewhile)
if latest_stamps is not None and posts_to_download.first_item is not None:
latest_stamps.set_last_post_timestamp(profile_name,
posts_to_download.first_item.date_local)
if stories and profiles: if stories and profiles:
with self.context.error_catcher("Download stories"): with self.context.error_catcher("Download stories"):
self.context.log("Downloading stories") self.context.log("Downloading stories")
self.download_stories(userids=list(profiles), fast_update=fast_update, filename_target=None, self.download_stories(userids=list(profiles), fast_update=fast_update, filename_target=None,
storyitem_filter=storyitem_filter) storyitem_filter=storyitem_filter, latest_stamps=latest_stamps)
def download_profile(self, profile_name: Union[str, Profile], def download_profile(self, profile_name: Union[str, Profile],
profile_pic: bool = True, profile_pic_only: bool = False, profile_pic: bool = True, profile_pic_only: bool = False,
@@ -1392,5 +1538,6 @@ class Instaloader:
code = input("Enter 2FA verification code: ") code = input("Enter 2FA verification code: ")
self.two_factor_login(code) self.two_factor_login(code)
break break
except BadCredentialsException: except BadCredentialsException as err:
print(err, file=sys.stderr)
pass pass
+30 -7
View File
@@ -33,7 +33,7 @@ def copy_session(session: requests.Session, request_timeout: Optional[float] = N
def default_user_agent() -> str: def default_user_agent() -> str:
return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \ return 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \
'(KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36' '(KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36'
class InstaloaderContext: class InstaloaderContext:
@@ -54,7 +54,8 @@ class InstaloaderContext:
def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None, def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
max_connection_attempts: int = 3, request_timeout: float = 300.0, max_connection_attempts: int = 3, request_timeout: float = 300.0,
rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None, rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None,
fatal_status_codes: Optional[List[int]] = None): fatal_status_codes: Optional[List[int]] = None,
iphone_support: bool = True):
self.user_agent = user_agent if user_agent is not None else default_user_agent() self.user_agent = user_agent if user_agent is not None else default_user_agent()
self.request_timeout = request_timeout self.request_timeout = request_timeout
@@ -66,6 +67,7 @@ class InstaloaderContext:
self._graphql_page_length = 50 self._graphql_page_length = 50
self._root_rhx_gis = None self._root_rhx_gis = None
self.two_factor_auth_pending = None self.two_factor_auth_pending = None
self.iphone_support = iphone_support
# error log, filled with error() and printed at the end of Instaloader.main() # error log, filled with error() and printed at the end of Instaloader.main()
self.error_log = [] # type: List[str] self.error_log = [] # type: List[str]
@@ -152,7 +154,6 @@ class InstaloaderContext:
if empty_session_only: if empty_session_only:
del header['Host'] del header['Host']
del header['Origin'] del header['Origin']
del header['Referer']
del header['X-Instagram-AJAX'] del header['X-Instagram-AJAX']
del header['X-Requested-With'] del header['X-Requested-With']
return header return header
@@ -210,8 +211,8 @@ class InstaloaderContext:
# Override default timeout behavior. # Override default timeout behavior.
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427 # Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
session.get('https://www.instagram.com/web/__mid/') csrf_json = self.get_json('accounts/login/', {}, session=session)
csrf_token = session.cookies.get_dict()['csrftoken'] csrf_token = csrf_json['config']['csrf_token']
session.headers.update({'X-CSRFToken': csrf_token}) session.headers.update({'X-CSRFToken': csrf_token})
# Not using self.get_json() here, because we need to access csrftoken cookie # Not using self.get_json() here, because we need to access csrftoken cookie
self.do_sleep() self.do_sleep()
@@ -283,9 +284,9 @@ class InstaloaderContext:
resp_json = login.json() resp_json = login.json()
if resp_json['status'] != 'ok': if resp_json['status'] != 'ok':
if 'message' in resp_json: if 'message' in resp_json:
raise BadCredentialsException("Login error: {}".format(resp_json['message'])) raise BadCredentialsException("2FA error: {}".format(resp_json['message']))
else: else:
raise BadCredentialsException("Login error: \"{}\" status.".format(resp_json['status'])) raise BadCredentialsException("2FA error: \"{}\" status.".format(resp_json['status']))
session.headers.update({'X-CSRFToken': login.cookies['csrftoken']}) session.headers.update({'X-CSRFToken': login.cookies['csrftoken']})
self._session = session self._session = session
self.username = user self.username = user
@@ -528,6 +529,28 @@ class InstaloaderContext:
:raises ConnectionException: When download repeatedly failed.""" :raises ConnectionException: When download repeatedly failed."""
self.write_raw(self.get_raw(url), filename) self.write_raw(self.get_raw(url), filename)
def head(self, url: str, allow_redirects: bool = False) -> requests.Response:
"""HEAD a URL anonymously.
:raises QueryReturnedNotFoundException: When the server responds with a 404.
:raises QueryReturnedForbiddenException: When the server responds with a 403.
:raises ConnectionException: When request failed.
.. versionadded:: 4.7.6
"""
with self.get_anonymous_session() as anonymous_session:
resp = anonymous_session.head(url, allow_redirects=allow_redirects)
if resp.status_code == 200:
return resp
else:
if resp.status_code == 403:
# suspected invalid URL signature
raise QueryReturnedForbiddenException("403 when accessing {}.".format(url))
if resp.status_code == 404:
# 404 not worth retrying.
raise QueryReturnedNotFoundException("404 when accessing {}.".format(url))
raise ConnectionException("HTTP error code {}.".format(resp.status_code))
@property @property
def root_rhx_gis(self) -> Optional[str]: def root_rhx_gis(self) -> Optional[str]:
"""rhx_gis string returned in the / query.""" """rhx_gis string returned in the / query."""
+113
View File
@@ -0,0 +1,113 @@
import configparser
from datetime import datetime, timezone
from typing import Optional
class LatestStamps:
"""LatestStamps class.
Convenience class for retrieving and storing data from the :option:`--latest-stamps` file.
:param latest_stamps_file: path to file.
.. versionadded:: 4.8"""
PROFILE_ID = 'profile-id'
PROFILE_PIC = 'profile-pic'
POST_TIMESTAMP = 'post-timestamp'
TAGGED_TIMESTAMP = 'tagged-timestamp'
IGTV_TIMESTAMP = 'igtv-timestamp'
STORY_TIMESTAMP = 'story-timestamp'
ISO_FORMAT = '%Y-%m-%dT%H:%M:%S.%f%z'
def __init__(self, latest_stamps_file):
self.file = latest_stamps_file
self.data = configparser.ConfigParser()
self.data.read(latest_stamps_file)
def _save(self):
with open(self.file, 'w') as f:
self.data.write(f)
def _ensure_section(self, section: str):
if not self.data.has_section(section):
self.data.add_section(section)
def get_profile_id(self, profile_name: str) -> Optional[int]:
"""Returns stored ID of profile."""
try:
return self.data.getint(profile_name, self.PROFILE_ID)
except (configparser.Error, ValueError):
return None
def save_profile_id(self, profile_name: str, profile_id: int):
"""Stores ID of profile."""
self._ensure_section(profile_name)
self.data.set(profile_name, self.PROFILE_ID, str(profile_id))
self._save()
def rename_profile(self, old_profile: str, new_profile: str):
"""Renames a profile."""
self._ensure_section(new_profile)
for option in [self.PROFILE_ID, self.PROFILE_PIC, self.POST_TIMESTAMP,
self.TAGGED_TIMESTAMP, self.IGTV_TIMESTAMP, self.STORY_TIMESTAMP]:
if self.data.has_option(old_profile, option):
value = self.data.get(old_profile, option)
self.data.set(new_profile, option, value)
self.data.remove_section(old_profile)
self._save()
def _get_timestamp(self, section: str, key: str) -> datetime:
try:
return datetime.strptime(self.data.get(section, key), self.ISO_FORMAT)
except (configparser.Error, ValueError):
return datetime.fromtimestamp(0, timezone.utc)
def _set_timestamp(self, section: str, key: str, timestamp: datetime):
self._ensure_section(section)
self.data.set(section, key, timestamp.strftime(self.ISO_FORMAT))
self._save()
def get_last_post_timestamp(self, profile_name: str) -> datetime:
"""Returns timestamp of last download of a profile's posts."""
return self._get_timestamp(profile_name, self.POST_TIMESTAMP)
def set_last_post_timestamp(self, profile_name: str, timestamp: datetime):
"""Sets timestamp of last download of a profile's posts."""
self._set_timestamp(profile_name, self.POST_TIMESTAMP, timestamp)
def get_last_tagged_timestamp(self, profile_name: str) -> datetime:
"""Returns timestamp of last download of a profile's tagged posts."""
return self._get_timestamp(profile_name, self.TAGGED_TIMESTAMP)
def set_last_tagged_timestamp(self, profile_name: str, timestamp: datetime):
"""Sets timestamp of last download of a profile's tagged posts."""
self._set_timestamp(profile_name, self.TAGGED_TIMESTAMP, timestamp)
def get_last_igtv_timestamp(self, profile_name: str) -> datetime:
"""Returns timestamp of last download of a profile's igtv posts."""
return self._get_timestamp(profile_name, self.IGTV_TIMESTAMP)
def set_last_igtv_timestamp(self, profile_name: str, timestamp: datetime):
"""Sets timestamp of last download of a profile's igtv posts."""
self._set_timestamp(profile_name, self.IGTV_TIMESTAMP, timestamp)
def get_last_story_timestamp(self, profile_name: str) -> datetime:
"""Returns timestamp of last download of a profile's stories."""
return self._get_timestamp(profile_name, self.STORY_TIMESTAMP)
def set_last_story_timestamp(self, profile_name: str, timestamp: datetime):
"""Sets timestamp of last download of a profile's stories."""
self._set_timestamp(profile_name, self.STORY_TIMESTAMP, timestamp)
def get_profile_pic(self, profile_name: str) -> str:
"""Returns filename of profile's last downloaded profile pic."""
try:
return self.data.get(profile_name, self.PROFILE_PIC)
except configparser.Error:
return ""
def set_profile_pic(self, profile_name: str, profile_pic: str):
"""Sets filename of profile's last downloaded profile pic."""
self._ensure_section(profile_name)
self.data.set(profile_name, self.PROFILE_PIC, profile_pic)
self._save()
+32 -13
View File
@@ -17,9 +17,8 @@ FrozenNodeIterator = NamedTuple('FrozenNodeIterator',
('context_username', Optional[str]), ('context_username', Optional[str]),
('total_index', int), ('total_index', int),
('best_before', Optional[float]), ('best_before', Optional[float]),
('remaining_data', Optional[Dict])]) ('remaining_data', Optional[Dict]),
FrozenNodeIterator.__doc__ = \ ('first_node', Optional[Dict])])
"""A serializable representation of a :class:`NodeIterator` instance, saving its iteration state."""
FrozenNodeIterator.query_hash.__doc__ = """The GraphQL ``query_hash`` parameter.""" FrozenNodeIterator.query_hash.__doc__ = """The GraphQL ``query_hash`` parameter."""
FrozenNodeIterator.query_variables.__doc__ = """The GraphQL ``query_variables`` parameter.""" FrozenNodeIterator.query_variables.__doc__ = """The GraphQL ``query_variables`` parameter."""
FrozenNodeIterator.query_referer.__doc__ = """The HTTP referer used for the GraphQL query.""" FrozenNodeIterator.query_referer.__doc__ = """The HTTP referer used for the GraphQL query."""
@@ -28,7 +27,7 @@ FrozenNodeIterator.total_index.__doc__ = """Number of items that have already be
FrozenNodeIterator.best_before.__doc__ = """Date when parts of the stored nodes might have expired.""" FrozenNodeIterator.best_before.__doc__ = """Date when parts of the stored nodes might have expired."""
FrozenNodeIterator.remaining_data.__doc__ = \ FrozenNodeIterator.remaining_data.__doc__ = \
"""The already-retrieved, yet-unprocessed ``edges`` and the ``page_info`` at time of freezing.""" """The already-retrieved, yet-unprocessed ``edges`` and the ``page_info`` at time of freezing."""
FrozenNodeIterator.first_node.__doc__ = """Node data of the first item, if an item has been produced."""
T = TypeVar('T') T = TypeVar('T')
@@ -55,6 +54,9 @@ class NodeIterator(Iterator[T]):
post_iterator = profile.get_posts() post_iterator = profile.get_posts()
post_iterator.thaw(load("resume_information.json")) post_iterator.thaw(load("resume_information.json"))
(an appropriate method to load and save the :class:`FrozenNodeIterator` is e.g.
:func:`load_structure_from_file` and :func:`save_structure_to_file`.)
A :class:`FrozenNodeIterator` can only be thawn with a matching NodeIterator, i.e. a NodeIterator instance that has A :class:`FrozenNodeIterator` can only be thawn with a matching NodeIterator, i.e. a NodeIterator instance that has
been constructed with the same parameters as the instance that is represented by the :class:`FrozenNodeIterator` in been constructed with the same parameters as the instance that is represented by the :class:`FrozenNodeIterator` in
question. This is to ensure that an iteration cannot be resumed in a wrong, unmatching loop. As a quick way to question. This is to ensure that an iteration cannot be resumed in a wrong, unmatching loop. As a quick way to
@@ -88,6 +90,7 @@ class NodeIterator(Iterator[T]):
self._best_before = datetime.now() + NodeIterator._shelf_life self._best_before = datetime.now() + NodeIterator._shelf_life
else: else:
self._data = self._query() self._data = self._query()
self._first_node: Optional[Dict] = None
def _query(self, after: Optional[str] = None) -> Dict: def _query(self, after: Optional[str] = None) -> Dict:
pagination_variables = {'first': NodeIterator._graphql_page_length} # type: Dict[str, Any] pagination_variables = {'first': NodeIterator._graphql_page_length} # type: Dict[str, Any]
@@ -124,17 +127,21 @@ class NodeIterator(Iterator[T]):
except KeyboardInterrupt: except KeyboardInterrupt:
self._page_index, self._total_index = page_index, total_index self._page_index, self._total_index = page_index, total_index
raise raise
return self._node_wrapper(node) item = self._node_wrapper(node)
if self._first_node is None:
self._first_node = node
return item
if self._data['page_info']['has_next_page']: if self._data['page_info']['has_next_page']:
query_response = self._query(self._data['page_info']['end_cursor']) query_response = self._query(self._data['page_info']['end_cursor'])
page_index, data = self._page_index, self._data if self._data['edges'] != query_response['edges']:
try: page_index, data = self._page_index, self._data
self._page_index = 0 try:
self._data = query_response self._page_index = 0
except KeyboardInterrupt: self._data = query_response
self._page_index, self._data = page_index, data except KeyboardInterrupt:
raise self._page_index, self._data = page_index, data
return self.__next__() raise
return self.__next__()
raise StopIteration() raise StopIteration()
@property @property
@@ -156,6 +163,15 @@ class NodeIterator(Iterator[T]):
).encode()) ).encode())
return base64.urlsafe_b64encode(magic_hash.digest()).decode() return base64.urlsafe_b64encode(magic_hash.digest()).decode()
@property
def first_item(self) -> Optional[T]:
"""
If this iterator has produced any items, returns the first item produced.
.. versionadded:: 4.8
"""
return self._node_wrapper(self._first_node) if self._first_node is not None else None
def freeze(self) -> FrozenNodeIterator: def freeze(self) -> FrozenNodeIterator:
"""Freeze the iterator for later resuming.""" """Freeze the iterator for later resuming."""
remaining_data = None remaining_data = None
@@ -170,6 +186,7 @@ class NodeIterator(Iterator[T]):
total_index=max(self.total_index - 1, 0), total_index=max(self.total_index - 1, 0),
best_before=self._best_before.timestamp() if self._best_before else None, best_before=self._best_before.timestamp() if self._best_before else None,
remaining_data=remaining_data, remaining_data=remaining_data,
first_node=self._first_node,
) )
def thaw(self, frozen: FrozenNodeIterator) -> None: def thaw(self, frozen: FrozenNodeIterator) -> None:
@@ -196,6 +213,8 @@ class NodeIterator(Iterator[T]):
self._total_index = frozen.total_index self._total_index = frozen.total_index
self._best_before = datetime.fromtimestamp(frozen.best_before) self._best_before = datetime.fromtimestamp(frozen.best_before)
self._data = frozen.remaining_data self._data = frozen.remaining_data
if frozen.first_node is not None:
self._first_node = frozen.first_node
@contextmanager @contextmanager
+46
View File
@@ -0,0 +1,46 @@
from typing import Any, Callable, Dict, Iterator, Optional, TypeVar
from .instaloadercontext import InstaloaderContext
T = TypeVar('T')
class SectionIterator(Iterator[T]):
"""Iterator for the new 'sections'-style responses.
.. versionadded:: 4.9"""
def __init__(self,
context: InstaloaderContext,
sections_extractor: Callable[[Dict[str, Any]], Dict[str, Any]],
media_wrapper: Callable[[Dict], T],
query_path: str,
first_data: Optional[Dict[str, Any]] = None):
self._context = context
self._sections_extractor = sections_extractor
self._media_wrapper = media_wrapper
self._query_path = query_path
self._data = first_data or self._query()
self._page_index = 0
self._section_index = 0
def __iter__(self):
return self
def _query(self, max_id: Optional[str] = None) -> Dict[str, Any]:
pagination_variables = {"max_id": max_id} if max_id is not None else {}
return self._sections_extractor(
self._context.get_json(self._query_path, params={"__a": 1, **pagination_variables})
)
def __next__(self) -> T:
if self._page_index < len(self._data['sections']):
media = self._data['sections'][self._page_index]['layout_content']['medias'][self._section_index]['media']
self._section_index += 1
if self._section_index >= len(self._data['sections'][self._page_index]['layout_content']['medias']):
self._section_index = 0
self._page_index += 1
return self._media_wrapper(media)
if self._data['more_available']:
self._page_index, self._section_index, self._data = 0, 0, self._query(self._data["next_max_id"])
return self.__next__()
raise StopIteration()
+316 -83
View File
@@ -3,13 +3,17 @@ import lzma
import re import re
from base64 import b64decode, b64encode from base64 import b64decode, b64encode
from collections import namedtuple from collections import namedtuple
from contextlib import suppress
from datetime import datetime from datetime import datetime
from typing import Any, Dict, Iterable, Iterator, List, Optional, Union from itertools import islice
from pathlib import Path
from typing import Any, Dict, Iterable, Iterator, List, Optional, Tuple, Union
from . import __version__ from . import __version__
from .exceptions import * from .exceptions import *
from .instaloadercontext import InstaloaderContext from .instaloadercontext import InstaloaderContext
from .nodeiterator import FrozenNodeIterator, NodeIterator from .nodeiterator import FrozenNodeIterator, NodeIterator
from .sectioniterator import SectionIterator
PostSidecarNode = namedtuple('PostSidecarNode', ['is_video', 'display_url', 'video_url']) PostSidecarNode = namedtuple('PostSidecarNode', ['is_video', 'display_url', 'video_url'])
PostSidecarNode.__doc__ = "Item of a Sidecar Post." PostSidecarNode.__doc__ = "Item of a Sidecar Post."
@@ -34,8 +38,8 @@ PostLocation.id.__doc__ = "ID number of location."
PostLocation.name.__doc__ = "Location name." PostLocation.name.__doc__ = "Location name."
PostLocation.slug.__doc__ = "URL friendly variant of location name." PostLocation.slug.__doc__ = "URL friendly variant of location name."
PostLocation.has_public_page.__doc__ = "Whether location has a public page." PostLocation.has_public_page.__doc__ = "Whether location has a public page."
PostLocation.lat.__doc__ = "Latitude (:class:`float`)." PostLocation.lat.__doc__ = "Latitude (:class:`float` or None)."
PostLocation.lng.__doc__ = "Longitude (:class:`float`)." PostLocation.lng.__doc__ = "Longitude (:class:`float` or None)."
class Post: class Post:
@@ -88,6 +92,41 @@ class Post:
"""Create a post object from a given mediaid""" """Create a post object from a given mediaid"""
return cls.from_shortcode(context, Post.mediaid_to_shortcode(mediaid)) return cls.from_shortcode(context, Post.mediaid_to_shortcode(mediaid))
@classmethod
def from_iphone_struct(cls, context: InstaloaderContext, media: Dict[str, Any]):
"""Create a post from a given iphone_struct.
.. versionadded:: 4.9"""
media_types = {
1: "GraphImage",
2: "GraphVideo",
8: "GraphSidecar",
}
fake_node = {
"shortcode": media["code"],
"id": media["pk"],
"__typename": media_types[media["media_type"]],
"is_video": media_types[media["media_type"]] == "GraphVideo",
"date": media["taken_at"],
"caption": media["caption"].get("text") if media.get("caption") is not None else None,
"title": media.get("title"),
"viewer_has_liked": media["has_liked"],
"edge_media_preview_like": {"count": media["like_count"]},
"iphone_struct": media,
}
with suppress(KeyError):
fake_node["display_url"] = media['image_versions2']['candidates'][0]['url']
with suppress(KeyError):
fake_node["video_url"] = media['video_versions'][-1]['url']
fake_node["video_duration"] = media["video_duration"]
fake_node["video_view_count"] = media["view_count"]
with suppress(KeyError):
fake_node["edge_sidecar_to_children"] = {"edges": [{"node": {
"display_url": node['image_versions2']['candidates'][0]['url'],
"is_video": media_types[node["media_type"]] == "GraphVideo",
}} for node in media["carousel_media"]]}
return cls(context, fake_node, Profile.from_iphone_struct(context, media["user"]) if "user" in media else None)
@staticmethod @staticmethod
def shortcode_to_mediaid(code: str) -> int: def shortcode_to_mediaid(code: str) -> int:
if len(code) > 11: if len(code) > 11:
@@ -168,6 +207,8 @@ class Post:
@property @property
def _iphone_struct(self) -> Dict[str, Any]: def _iphone_struct(self) -> Dict[str, Any]:
if not self._context.iphone_support:
raise IPhoneSupportDisabledException("iPhone support is disabled.")
if not self._context.is_logged_in: if not self._context.is_logged_in:
raise LoginRequiredException("--login required to access iPhone media info endpoint.") raise LoginRequiredException("--login required to access iPhone media info endpoint.")
if not self._iphone_struct_: if not self._iphone_struct_:
@@ -221,17 +262,16 @@ class Post:
@property @property
def date_local(self) -> datetime: def date_local(self) -> datetime:
"""Timestamp when the post was created (local time zone).""" """Timestamp when the post was created (local time zone).
return datetime.fromtimestamp(self._node["date"]
if "date" in self._node .. versionchanged:: 4.9
else self._node["taken_at_timestamp"]) Return timezone aware datetime object."""
return datetime.fromtimestamp(self._get_timestamp_date_created()).astimezone()
@property @property
def date_utc(self) -> datetime: def date_utc(self) -> datetime:
"""Timestamp when the post was created (UTC).""" """Timestamp when the post was created (UTC)."""
return datetime.utcfromtimestamp(self._node["date"] return datetime.utcfromtimestamp(self._get_timestamp_date_created())
if "date" in self._node
else self._node["taken_at_timestamp"])
@property @property
def date(self) -> datetime: def date(self) -> datetime:
@@ -246,10 +286,10 @@ class Post:
@property @property
def url(self) -> str: def url(self) -> str:
"""URL of the picture / video thumbnail of the post""" """URL of the picture / video thumbnail of the post"""
if self.typename == "GraphImage" and self._context.is_logged_in: if self.typename == "GraphImage" and self._context.iphone_support and self._context.is_logged_in:
try: try:
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url'] orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url) url = re.sub(r'([?&])se=\d+&?', r'\1', orig_url).rstrip('&')
return url return url
except (InstaloaderException, KeyError, IndexError) as err: except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self)) self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
@@ -272,6 +312,12 @@ class Post:
return len(edges) return len(edges)
return 1 return 1
def _get_timestamp_date_created(self) -> float:
"""Timestamp when the post was created"""
return (self._node["date"]
if "date" in self._node
else self._node["taken_at_timestamp"])
def get_is_videos(self) -> List[bool]: def get_is_videos(self) -> List[bool]:
""" """
Return a list containing the ``is_video`` property for each media in the post. Return a list containing the ``is_video`` property for each media in the post.
@@ -304,11 +350,11 @@ class Post:
node = edge['node'] node = edge['node']
is_video = node['is_video'] is_video = node['is_video']
display_url = node['display_url'] display_url = node['display_url']
if not is_video and self._context.is_logged_in: if not is_video and self._context.iphone_support and self._context.is_logged_in:
try: try:
carousel_media = self._iphone_struct['carousel_media'] carousel_media = self._iphone_struct['carousel_media']
orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url'] orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url) display_url = re.sub(r'([?&])se=\d+&?', r'\1', orig_url).rstrip('&')
except (InstaloaderException, KeyError, IndexError) as err: except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format( self._context.error('{} Unable to fetch high quality image version of {}.'.format(
err, self)) err, self))
@@ -339,9 +385,11 @@ class Post:
"""List of all lowercased profiles that are mentioned in the Post's caption, without preceeding @.""" """List of all lowercased profiles that are mentioned in the Post's caption, without preceeding @."""
if not self.caption: if not self.caption:
return [] return []
# This regular expression is from jStassen, adjusted to use Python's \w to support Unicode # This regular expression is modified from jStassen, adjusted to use Python's \w to
# support Unicode and a word/beginning of string delimiter at the beginning to ensure
# that no email addresses join the list of mentions.
# http://blog.jstassen.com/2016/03/code-regex-for-instagram-username-and-hashtags/ # http://blog.jstassen.com/2016/03/code-regex-for-instagram-username-and-hashtags/
mention_regex = re.compile(r"(?:@)(\w(?:(?:\w|(?:\.(?!\.))){0,28}(?:\w))?)") mention_regex = re.compile(r"(?:^|\W|_)(?:@)(\w(?:(?:\w|(?:\.(?!\.))){0,28}(?:\w))?)")
return re.findall(mention_regex, self.caption.lower()) return re.findall(mention_regex, self.caption.lower())
@property @property
@@ -354,6 +402,16 @@ class Post:
return (pcaption[:30] + u"\u2026") if len(pcaption) > 31 else pcaption return (pcaption[:30] + u"\u2026") if len(pcaption) > 31 else pcaption
return _elliptify(self.caption) if self.caption else '' return _elliptify(self.caption) if self.caption else ''
@property
def accessibility_caption(self) -> Optional[str]:
"""Accessibility caption of the post, if available.
.. versionadded:: 4.9"""
try:
return self._field("accessibility_caption")
except KeyError:
return None
@property @property
def tagged_users(self) -> List[str]: def tagged_users(self) -> List[str]:
"""List of all lowercased users that are tagged in the Post.""" """List of all lowercased users that are tagged in the Post."""
@@ -372,13 +430,35 @@ class Post:
def video_url(self) -> Optional[str]: def video_url(self) -> Optional[str]:
"""URL of the video, or None.""" """URL of the video, or None."""
if self.is_video: if self.is_video:
if self._context.is_logged_in: version_urls = []
try:
version_urls.append(self._field('video_url'))
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error(f"Warning: Unable to fetch video from graphql of {self}: {err}")
if self._context.iphone_support and self._context.is_logged_in:
try: try:
url = self._iphone_struct['video_versions'][0]['url'] version_urls.extend(version['url'] for version in self._iphone_struct['video_versions'])
return url
except (InstaloaderException, KeyError, IndexError) as err: except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality video version of {}.'.format(err, self)) self._context.error(f"Unable to fetch high-quality video version of {self}: {err}")
return self._field('video_url') version_urls = list(dict.fromkeys(version_urls))
if len(version_urls) == 0:
return None
if len(version_urls) == 1:
return version_urls[0]
url_candidates: List[Tuple[int, str]] = []
for idx, version_url in enumerate(version_urls):
try:
url_candidates.append((
int(self._context.head(version_url, allow_redirects=True).headers.get('Content-Length', 0)),
version_url
))
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error(f"Video URL candidate {idx+1}/{len(version_urls)} for {self}: {err}")
if not url_candidates:
# All candidates fail: Fallback to default URL and handle errors later at the actual download attempt
return version_urls[0]
url_candidates.sort()
return url_candidates[-1][1]
return None return None
@property @property
@@ -551,10 +631,10 @@ class Post:
return None return None
location_id = int(loc['id']) location_id = int(loc['id'])
if any(k not in loc for k in ('name', 'slug', 'has_public_page', 'lat', 'lng')): if any(k not in loc for k in ('name', 'slug', 'has_public_page', 'lat', 'lng')):
loc = self._context.get_json("explore/locations/{0}/".format(location_id), loc.update(self._context.get_json("explore/locations/{0}/".format(location_id),
params={'__a': 1})['graphql']['location'] params={'__a': 1})['native_location_data']['location_info'])
self._location = PostLocation(location_id, loc['name'], loc['slug'], loc['has_public_page'], self._location = PostLocation(location_id, loc['name'], loc['slug'], loc['has_public_page'],
loc['lat'], loc['lng']) loc.get('lat'), loc.get('lng'))
return self._location return self._location
@@ -638,6 +718,20 @@ class Profile:
context.profile_id_cache[profile_id] = profile context.profile_id_cache[profile_id] = profile
return profile return profile
@classmethod
def from_iphone_struct(cls, context: InstaloaderContext, media: Dict[str, Any]):
"""Create a profile from a given iphone_struct.
.. versionadded:: 4.9"""
return cls(context, {
"id": media["pk"],
"username": media["username"],
"is_private": media["is_private"],
"full_name": media["full_name"],
"profile_pic_url_hd": media["profile_pic_url"],
"iphone_struct": media,
})
@classmethod @classmethod
def own_profile(cls, context: InstaloaderContext): def own_profile(cls, context: InstaloaderContext):
"""Return own profile if logged-in. """Return own profile if logged-in.
@@ -691,6 +785,8 @@ class Profile:
@property @property
def _iphone_struct(self) -> Dict[str, Any]: def _iphone_struct(self) -> Dict[str, Any]:
if not self._context.iphone_support:
raise IPhoneSupportDisabledException("iPhone support is disabled.")
if not self._context.is_logged_in: if not self._context.is_logged_in:
raise LoginRequiredException("--login required to access iPhone profile info endpoint.") raise LoginRequiredException("--login required to access iPhone profile info endpoint.")
if not self._iphone_struct_: if not self._iphone_struct_:
@@ -791,14 +887,13 @@ class Profile:
def has_public_story(self) -> bool: def has_public_story(self) -> bool:
if not self._has_public_story: if not self._has_public_story:
self._obtain_metadata() self._obtain_metadata()
# query not rate limited if invoked anonymously: # query rate might be limited:
with self._context.anonymous_copy() as anonymous_context: data = self._context.graphql_query('9ca88e465c3f866a76f7adee3871bdd8',
data = anonymous_context.graphql_query('9ca88e465c3f866a76f7adee3871bdd8', {'user_id': self.userid, 'include_chaining': False,
{'user_id': self.userid, 'include_chaining': False, 'include_reel': False, 'include_suggested_users': False,
'include_reel': False, 'include_suggested_users': False, 'include_logged_out_extras': True,
'include_logged_out_extras': True, 'include_highlight_reels': False},
'include_highlight_reels': False}, 'https://www.instagram.com/{}/'.format(self.username))
'https://www.instagram.com/{}/'.format(self.username))
self._has_public_story = data['data']['user']['has_public_story'] self._has_public_story = data['data']['user']['has_public_story']
assert self._has_public_story is not None assert self._has_public_story is not None
return self._has_public_story return self._has_public_story
@@ -834,7 +929,7 @@ class Profile:
.. versionchanged:: 4.2.1 .. versionchanged:: 4.2.1
Require being logged in for HD version (as required by Instagram).""" Require being logged in for HD version (as required by Instagram)."""
if self._context.is_logged_in: if self._context.iphone_support and self._context.is_logged_in:
try: try:
return self._iphone_struct['hd_profile_pic_url_info']['url'] return self._iphone_struct['hd_profile_pic_url_info']['url']
except (InstaloaderException, KeyError) as err: except (InstaloaderException, KeyError) as err:
@@ -1021,6 +1116,8 @@ class StoryItem:
@property @property
def _iphone_struct(self) -> Dict[str, Any]: def _iphone_struct(self) -> Dict[str, Any]:
if not self._context.iphone_support:
raise IPhoneSupportDisabledException("iPhone support is disabled.")
if not self._context.is_logged_in: if not self._context.is_logged_in:
raise LoginRequiredException("--login required to access iPhone media info endpoint.") raise LoginRequiredException("--login required to access iPhone media info endpoint.")
if not self._iphone_struct_: if not self._iphone_struct_:
@@ -1048,8 +1145,11 @@ class StoryItem:
@property @property
def date_local(self) -> datetime: def date_local(self) -> datetime:
"""Timestamp when the StoryItem was created (local time zone).""" """Timestamp when the StoryItem was created (local time zone).
return datetime.fromtimestamp(self._node['taken_at_timestamp'])
.. versionchanged:: 4.9
Return timezone aware datetime object."""
return datetime.fromtimestamp(self._node['taken_at_timestamp']).astimezone()
@property @property
def date_utc(self) -> datetime: def date_utc(self) -> datetime:
@@ -1079,10 +1179,10 @@ class StoryItem:
@property @property
def url(self) -> str: def url(self) -> str:
"""URL of the picture / video thumbnail of the StoryItem""" """URL of the picture / video thumbnail of the StoryItem"""
if self.typename == "GraphStoryImage" and self._context.is_logged_in: if self.typename == "GraphStoryImage" and self._context.iphone_support and self._context.is_logged_in:
try: try:
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url'] orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url) url = re.sub(r'([?&])se=\d+&?', r'\1', orig_url).rstrip('&')
return url return url
except (InstaloaderException, KeyError, IndexError) as err: except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self)) self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
@@ -1102,7 +1202,35 @@ class StoryItem:
def video_url(self) -> Optional[str]: def video_url(self) -> Optional[str]:
"""URL of the video, or None.""" """URL of the video, or None."""
if self.is_video: if self.is_video:
return self._node['video_resources'][-1]['src'] version_urls = []
try:
version_urls.append(self._node['video_resources'][-1]['src'])
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error(f"Warning: Unable to fetch video from graphql of {self}: {err}")
if self._context.iphone_support and self._context.is_logged_in:
try:
version_urls.extend(version['url'] for version in self._iphone_struct['video_versions'])
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error(f"Unable to fetch high-quality video version of {self}: {err}")
version_urls = list(dict.fromkeys(version_urls))
if len(version_urls) == 0:
return None
if len(version_urls) == 1:
return version_urls[0]
url_candidates: List[Tuple[int, str]] = []
for idx, version_url in enumerate(version_urls):
try:
url_candidates.append((
int(self._context.head(version_url, allow_redirects=True).headers.get('Content-Length', 0)),
version_url
))
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error(f"Video URL candidate {idx+1}/{len(version_urls)} for {self}: {err}")
if not url_candidates:
# All candidates fail: Fallback to default URL and handle errors later at the actual download attempt
return version_urls[0]
url_candidates.sort()
return url_candidates[-1][1]
return None return None
@@ -1301,6 +1429,9 @@ class Hashtag:
L.download_post(post, target="#"+hashtag.name) L.download_post(post, target="#"+hashtag.name)
Also, this class implements == and is hashable. Also, this class implements == and is hashable.
.. versionchanged:: 4.9
Removed ``get_related_tags()`` and ``is_top_media_only`` as these features were removed from Instagram.
""" """
def __init__(self, context: InstaloaderContext, node: Dict[str, Any]): def __init__(self, context: InstaloaderContext, node: Dict[str, Any]):
assert "name" in node assert "name" in node
@@ -1329,8 +1460,8 @@ class Hashtag:
return self._node["name"].lower() return self._node["name"].lower()
def _query(self, params): def _query(self, params):
return self._context.get_json("explore/tags/{0}/".format(self.name), json_response = self._context.get_json("explore/tags/{0}/".format(self.name), params)
params)["graphql"]["hashtag"] return json_response["graphql"]["hashtag"] if "graphql" in json_response else json_response["data"]
def _obtain_metadata(self): def _obtain_metadata(self):
if not self._has_full_metadata: if not self._has_full_metadata:
@@ -1341,7 +1472,9 @@ class Hashtag:
json_node = self._node.copy() json_node = self._node.copy()
# remove posts # remove posts
json_node.pop("edge_hashtag_to_top_posts", None) json_node.pop("edge_hashtag_to_top_posts", None)
json_node.pop("top", None)
json_node.pop("edge_hashtag_to_media", None) json_node.pop("edge_hashtag_to_media", None)
json_node.pop("recent", None)
return json_node return json_node
def __repr__(self): def __repr__(self):
@@ -1377,30 +1510,33 @@ class Hashtag:
return self._metadata("profile_pic_url") return self._metadata("profile_pic_url")
@property @property
def description(self) -> str: def description(self) -> Optional[str]:
return self._metadata("description") return self._metadata("description")
@property @property
def allow_following(self) -> bool: def allow_following(self) -> bool:
return self._metadata("allow_following") return bool(self._metadata("allow_following"))
@property @property
def is_following(self) -> bool: def is_following(self) -> bool:
return self._metadata("is_following") try:
return self._metadata("is_following")
@property except KeyError:
def is_top_media_only(self) -> bool: return bool(self._metadata("following"))
return self._metadata("is_top_media_only")
def get_related_tags(self) -> Iterator["Hashtag"]:
"""Yields similar hashtags."""
yield from (Hashtag(self._context, edge["node"])
for edge in self._metadata("edge_hashtag_to_related_tags", "edges"))
def get_top_posts(self) -> Iterator[Post]: def get_top_posts(self) -> Iterator[Post]:
"""Yields the top posts of the hashtag.""" """Yields the top posts of the hashtag."""
yield from (Post(self._context, edge["node"]) try:
for edge in self._metadata("edge_hashtag_to_top_posts", "edges")) yield from (Post(self._context, edge["node"])
for edge in self._metadata("edge_hashtag_to_top_posts", "edges"))
except KeyError:
yield from SectionIterator(
self._context,
lambda d: d["data"]["top"],
lambda m: Post.from_iphone_struct(self._context, m),
f"explore/tags/{self.name}/",
self._metadata("top"),
)
@property @property
def mediacount(self) -> int: def mediacount(self) -> int:
@@ -1410,22 +1546,34 @@ class Hashtag:
The number of posts with a certain hashtag may differ from the number of posts that can actually be accessed, as The number of posts with a certain hashtag may differ from the number of posts that can actually be accessed, as
the hashtag count might include private posts the hashtag count might include private posts
""" """
return self._metadata("edge_hashtag_to_media", "count") try:
return self._metadata("edge_hashtag_to_media", "count")
except KeyError:
return self._metadata("media_count")
def get_posts(self) -> Iterator[Post]: def get_posts(self) -> Iterator[Post]:
"""Yields the posts associated with this hashtag.""" """Yields the recent posts associated with this hashtag."""
self._metadata("edge_hashtag_to_media", "edges") try:
self._metadata("edge_hashtag_to_media", "page_info") self._metadata("edge_hashtag_to_media", "edges")
conn = self._metadata("edge_hashtag_to_media") self._metadata("edge_hashtag_to_media", "page_info")
yield from (Post(self._context, edge["node"]) for edge in conn["edges"]) conn = self._metadata("edge_hashtag_to_media")
while conn["page_info"]["has_next_page"]:
data = self._query({'__a': 1, 'max_id': conn["page_info"]["end_cursor"]})
conn = data["edge_hashtag_to_media"]
yield from (Post(self._context, edge["node"]) for edge in conn["edges"]) yield from (Post(self._context, edge["node"]) for edge in conn["edges"])
while conn["page_info"]["has_next_page"]:
data = self._query({'__a': 1, 'max_id': conn["page_info"]["end_cursor"]})
conn = data["edge_hashtag_to_media"]
yield from (Post(self._context, edge["node"]) for edge in conn["edges"])
except KeyError:
yield from SectionIterator(
self._context,
lambda d: d["data"]["recent"],
lambda m: Post.from_iphone_struct(self._context, m),
f"explore/tags/{self.name}/",
self._metadata("recent"),
)
def get_all_posts(self) -> Iterator[Post]: def get_all_posts(self) -> Iterator[Post]:
"""Yields all posts, i.e. all most recent posts and the top posts, in almost-chronological order.""" """Yields all posts, i.e. all most recent posts and the top posts, in almost-chronological order."""
sorted_top_posts = iter(sorted(self.get_top_posts(), key=lambda p: p.date_utc, reverse=True)) sorted_top_posts = iter(sorted(islice(self.get_top_posts(), 9), key=lambda p: p.date_utc, reverse=True))
other_posts = self.get_posts() other_posts = self.get_posts()
next_top = next(sorted_top_posts, None) next_top = next(sorted_top_posts, None)
next_other = next(other_posts, None) next_other = next(other_posts, None)
@@ -1452,6 +1600,20 @@ class Hashtag:
yield next_other yield next_other
next_other = next(other_posts, None) next_other = next(other_posts, None)
def get_posts_resumable(self) -> NodeIterator[Post]:
"""Get the recent posts of the hashtag in a resumable fashion.
:rtype: NodeIterator[Post]
.. versionadded:: 4.9"""
return NodeIterator(
self._context, "9b498c08113f1e09617a1703c22b2f32",
lambda d: d['data']['hashtag']['edge_hashtag_to_media'],
lambda n: Post(self._context, n),
{'tag_name': self.name},
f"https://www.instagram.com/explore/tags/{self.name}/"
)
class TopSearchResults: class TopSearchResults:
""" """
@@ -1501,7 +1663,7 @@ class TopSearchResults:
place = location.get('place', {}) place = location.get('place', {})
slug = place.get('slug') slug = place.get('slug')
loc = place.get('location', {}) loc = place.get('location', {})
yield PostLocation(int(loc['pk']), loc['name'], slug, None, loc['lat'], loc['lng']) yield PostLocation(int(loc['pk']), loc['name'], slug, None, loc.get('lat'), loc.get('lng'))
def get_hashtag_strings(self) -> Iterator[str]: def get_hashtag_strings(self) -> Iterator[str]:
""" """
@@ -1531,12 +1693,72 @@ class TopSearchResults:
return self._searchstring return self._searchstring
class TitlePic:
def __init__(self, profile: Optional[Profile], target: Union[str, Path], typename: str,
filename: str, date_utc: Optional[datetime]):
self._profile = profile
self._target = target
self._typename = typename
self._filename = filename
self._date_utc = date_utc
@property
def profile(self) -> Union[str, Path]:
return self._profile.username.lower() if self._profile is not None else self._target
@property
def owner_username(self) -> Union[str, Path]:
return self.profile
@property
def owner_id(self) -> Union[str, Path]:
return str(self._profile.userid) if self._profile is not None else self._target
@property
def target(self) -> Union[str, Path]:
return self._target
@property
def typename(self) -> str:
return self._typename
@property
def filename(self) -> str:
return self._filename
@property
def date_utc(self) -> Optional[datetime]:
return self._date_utc
@property
def date(self) -> Optional[datetime]:
return self.date_utc
@property
def date_local(self) -> Optional[datetime]:
return self._date_utc.astimezone() if self._date_utc is not None else None
JsonExportable = Union[Post, Profile, StoryItem, Hashtag, FrozenNodeIterator] JsonExportable = Union[Post, Profile, StoryItem, Hashtag, FrozenNodeIterator]
def get_json_structure(structure: JsonExportable) -> dict:
"""Returns Instaloader JSON structure for a :class:`Post`, :class:`Profile`, :class:`StoryItem`, :class:`Hashtag`
or :class:`FrozenNodeIterator` so that it can be loaded by :func:`load_structure`.
:param structure: :class:`Post`, :class:`Profile`, :class:`StoryItem` or :class:`Hashtag`
.. versionadded:: 4.8
"""
return {
'node': structure._asdict(),
'instaloader': {'version': __version__, 'node_type': structure.__class__.__name__}
}
def save_structure_to_file(structure: JsonExportable, filename: str) -> None: def save_structure_to_file(structure: JsonExportable, filename: str) -> None:
"""Saves a :class:`Post`, :class:`Profile`, :class:`StoryItem` or :class:`Hashtag` to a '.json' or '.json.xz' file """Saves a :class:`Post`, :class:`Profile`, :class:`StoryItem`, :class:`Hashtag` or :class:`FrozenNodeIterator` to a
such that it can later be loaded by :func:`load_structure_from_file`. '.json' or '.json.xz' file such that it can later be loaded by :func:`load_structure_from_file`.
If the specified filename ends in '.xz', the file will be LZMA compressed. Otherwise, a pretty-printed JSON file If the specified filename ends in '.xz', the file will be LZMA compressed. Otherwise, a pretty-printed JSON file
will be created. will be created.
@@ -1544,8 +1766,7 @@ def save_structure_to_file(structure: JsonExportable, filename: str) -> None:
:param structure: :class:`Post`, :class:`Profile`, :class:`StoryItem` or :class:`Hashtag` :param structure: :class:`Post`, :class:`Profile`, :class:`StoryItem` or :class:`Hashtag`
:param filename: Filename, ends in '.json' or '.json.xz' :param filename: Filename, ends in '.json' or '.json.xz'
""" """
json_structure = {'node': structure._asdict(), json_structure = get_json_structure(structure)
'instaloader': {'version': __version__, 'node_type': structure.__class__.__name__}}
compress = filename.endswith('.xz') compress = filename.endswith('.xz')
if compress: if compress:
with lzma.open(filename, 'wt', check=lzma.CHECK_NONE) as fp: with lzma.open(filename, 'wt', check=lzma.CHECK_NONE) as fp:
@@ -1555,20 +1776,15 @@ def save_structure_to_file(structure: JsonExportable, filename: str) -> None:
json.dump(json_structure, fp=fp, indent=4, sort_keys=True) json.dump(json_structure, fp=fp, indent=4, sort_keys=True)
def load_structure_from_file(context: InstaloaderContext, filename: str) -> JsonExportable: def load_structure(context: InstaloaderContext, json_structure: dict) -> JsonExportable:
"""Loads a :class:`Post`, :class:`Profile`, :class:`StoryItem` or :class:`Hashtag` from a '.json' or '.json.xz' file """Loads a :class:`Post`, :class:`Profile`, :class:`StoryItem`, :class:`Hashtag` or :class:`FrozenNodeIterator` from
that has been saved by :func:`save_structure_to_file`. a json structure.
:param context: :attr:`Instaloader.context` linked to the new object, used for additional queries if neccessary. :param context: :attr:`Instaloader.context` linked to the new object, used for additional queries if neccessary.
:param filename: Filename, ends in '.json' or '.json.xz' :param json_structure: Instaloader JSON structure
.. versionadded:: 4.8
""" """
compressed = filename.endswith('.xz')
if compressed:
fp = lzma.open(filename, 'rt')
else:
fp = open(filename, 'rt')
json_structure = json.load(fp)
fp.close()
if 'node' in json_structure and 'instaloader' in json_structure and \ if 'node' in json_structure and 'instaloader' in json_structure and \
'node_type' in json_structure['instaloader']: 'node_type' in json_structure['instaloader']:
node_type = json_structure['instaloader']['node_type'] node_type = json_structure['instaloader']['node_type']
@@ -1581,11 +1797,28 @@ def load_structure_from_file(context: InstaloaderContext, filename: str) -> Json
elif node_type == "Hashtag": elif node_type == "Hashtag":
return Hashtag(context, json_structure['node']) return Hashtag(context, json_structure['node'])
elif node_type == "FrozenNodeIterator": elif node_type == "FrozenNodeIterator":
if not 'first_node' in json_structure['node']:
json_structure['node']['first_node'] = None
return FrozenNodeIterator(**json_structure['node']) return FrozenNodeIterator(**json_structure['node'])
else:
raise InvalidArgumentException("{}: Not an Instaloader JSON.".format(filename))
elif 'shortcode' in json_structure: elif 'shortcode' in json_structure:
# Post JSON created with Instaloader v3 # Post JSON created with Instaloader v3
return Post.from_shortcode(context, json_structure['shortcode']) return Post.from_shortcode(context, json_structure['shortcode'])
raise InvalidArgumentException("Passed json structure is not an Instaloader JSON")
def load_structure_from_file(context: InstaloaderContext, filename: str) -> JsonExportable:
"""Loads a :class:`Post`, :class:`Profile`, :class:`StoryItem`, :class:`Hashtag` or :class:`FrozenNodeIterator` from
a '.json' or '.json.xz' file that has been saved by :func:`save_structure_to_file`.
:param context: :attr:`Instaloader.context` linked to the new object, used for additional queries if neccessary.
:param filename: Filename, ends in '.json' or '.json.xz'
"""
compressed = filename.endswith('.xz')
if compressed:
fp = lzma.open(filename, 'rt')
else: else:
raise InvalidArgumentException("{}: Not an Instaloader JSON.".format(filename)) # pylint:disable=consider-using-with
fp = open(filename, 'rt')
json_structure = json.load(fp)
fp.close()
return load_structure(context, json_structure)