Compare commits

..
43 Commits
Author SHA1 Message Date
Alexander Graf ffbf5b517c Release of Version 4.3 2020-03-16 09:13:25 +01:00
Alexander Graf 0843e10187 First Release Candidate for Version 4.3 2020-03-15 14:28:16 +01:00
Alexander Graf 78a93c945e Revert "Revert "Remove nonworking saythanks.io link""
saythanks.io does not work again and the link is now removed.
This reverts commit b2acbcf0e0.
2020-03-14 17:56:44 +01:00
Alexander Graf bb3e309195 First Beta Release for Version 4.3 2020-03-14 17:21:07 +01:00
Alexander Graf 6cc2ed54ea Download IGTV Videos of a Profile (--igtv) 2020-03-14 16:56:51 +01:00
Alexander Graf 41cf260fdd Fix post filter for downloading tagged posts 2020-03-14 16:55:37 +01:00
Alexander Graf f910705c51 docs: Fix formatting of request_timeout param 2020-03-07 11:14:07 +01:00
Alexander Graf aa3f19dcbe Second Alpha Release for Version 4.3 2020-03-01 17:44:17 +01:00
Alexander Graf fe258f1abb Also Download Highlight Cover Pictures
Instaloader.download_highlights(), as well as the Instaloader CLI with
--highlight option, now also downloads the cover picture of each highlight.

This closes #323, where it was suggested.
2020-03-01 17:41:58 +01:00
Alexander Graf 534497bea5 fix misplacement of --request-timeout in docs 2020-02-28 09:49:29 +01:00
André Koch-Kramer 819426efdb First Alpha Release for Version 4.3 2020-02-17 18:53:33 +01:00
Alexander Graf 0aac91680f docs: versionadded/changed directives for 4.3 2020-02-17 18:47:45 +01:00
André Koch-Kramer dc289024c8 GitHub Actions workflow for Windows EXE 2020-02-17 18:31:36 +01:00
Alexander Graf b2acbcf0e0 Revert "Remove nonworking saythanks.io link"
saythanks.io works again. This reverts commit
f966ea5bb9.
2020-02-04 09:23:02 +01:00
Alexander Graf de585becdb Rate-control non-graphql queries
Non-GraphQL queries seem to be rate limited too, and this commit adds
a safe-seeming rate-controlling for this kind of queries.

This fixes #464.
2020-02-03 17:19:49 +01:00
Alexander Graf f966ea5bb9 Remove nonworking saythanks.io link
saythanks.io stopped working, BlitzKraft/saythanks.io#60.
2020-01-27 21:31:47 +01:00
André Koch-Kramer 302145f54f Print profile suggestions when profile not found
Relates to #496.
2020-01-27 13:06:42 +01:00
André Koch-Kramer 8764947d10 Raise QueryReturnedNotFoundException if expected structure is missing
- QueryReturnedNotFoundException now inherits ConnectionException
  --> Retry on 404 errors + missing "window._sharedData"

Relates to #146 and #496.
2020-01-27 13:06:42 +01:00
André Koch-Kramer d3403bc686 Implementation of TopSearchResults class
To be able to search Instagram for profiles, hashtags and locations.
2020-01-27 13:06:42 +01:00
Alexander Graf b7bae19374 Update mypy and pylint versions for CI 2020-01-27 09:27:47 +01:00
Alexander Graf 4e87a8cf63 Fix wrong request_timeout in anonymous_copy() context
Fixes a tiny mistake introduced with #422.
2020-01-26 22:07:07 +01:00
Alexander Graf fd13d66367 Deploy documentation on tag only
Before, the documentation was pushed to GitHub Pages whenever a new commit has been pushed to master. I.e., when a new feature was added, the documentation advertised the feature although hasn’t been released yet.

Now, we only deploy the newly-build the documentation when a new tag is pushed, and ensure the the versionadded directives are added before releasing.
2020-01-26 18:54:11 +01:00
corpone2 63eb9e71ca support passing timeout to requests (#422)
Adds a request_timeout parameter to Instaloader instantiation.

Closes #330.
2020-01-13 21:59:00 +01:00
André Koch-Kramer d8ba30b91b Speed up accessing property comments of Post
This has been asked for in #436.
2020-01-13 19:56:50 +01:00
André Koch-Kramer ec0e017b10 Release of version 4.2.10 2019-10-28 12:48:27 +01:00
51bfa01c2a Solving KeyError: 'edge_sidecar_to_children' (#411)
Closes #380.
Closes #394.

Co-Authored-By: Dāvis <davispuh@users.noreply.github.com>
Co-Authored-By: Alexander Graf <mail@agraf.me>
Co-Authored-By: Thammus <koch-kramer@web.de>
2019-10-21 19:47:14 +02:00
Alexander Graf e8d4054f84 Create Stale Issue Action 2019-10-19 16:20:25 +02:00
Leonard Bogdonoff cf293ce312 Change Example Hashtag in Codesnippets 2019-10-13 19:47:44 +02:00
Alexander Graf 3226c43a98 Fix Intersphinx Mapping for Requests to fix CI 2019-10-13 19:45:14 +02:00
corpone 20819d2597 handle alternate rate limit exceeded behavior of redirecting to login 2019-08-22 19:53:52 -07:00
Alexander Graf 4e8236b6e9 Improve contributing.rst and issue templates 2019-08-20 22:05:49 +02:00
André Koch-Kramer b542936c09 Release of version 4.2.9 2019-08-18 18:28:35 +02:00
Ahmad af4dbc00d7 Added likes_count to PostCommentAnswer tuple (#379) 2019-08-18 18:18:35 +02:00
Alexander Graf 0aa1ec7c76 Getting locations now requires --login
Closes #376.
2019-08-18 10:48:46 +02:00
Alexander Graf 75a729781d 194_top_x_of_user example: Fix sort order 2019-08-10 16:25:22 +02:00
Alexander Graf a88316224b Remove ',' after URL in "Checkpoint required" message
Closes #364.
2019-08-10 16:19:35 +02:00
Alexander Graf 87dc82d1b1 Travis: Skip test with Python 3.8 beta 2019-07-27 22:04:56 +02:00
Alexander Graf 6a3b05762e Fix missing directory error in some rare cases 2019-07-27 21:08:29 +02:00
Alexander Graf 6d5bb1e181 Release of Version 4.2.8 2019-05-27 18:26:10 +02:00
Alexander Graf 819057eb6f travis: use Pipfile for dependency pinning 2019-05-27 18:15:45 +02:00
Alexander Graf 5235cc0f6a Travis: test against Python 3.8 2019-05-27 18:08:17 +02:00
Alexander Graf fb06b4b7d6 Fix iPhone query rate control
Resolves #306.
2019-05-27 18:05:51 +02:00
Alexander Graf 1a382cd61c Fixes to unittests
- Fix type issue with unittest.SkipTest

- Fix query timestamp preservation
2019-05-17 21:58:51 +02:00
31 changed files with 1094 additions and 138 deletions
+5 -2
View File
@@ -1,6 +1,7 @@
--- ---
name: Bug report name: 🐛 Bug report
about: Create a report to help us improve about: Create a report to help us improve 🤕
labels: bug
--- ---
@@ -22,3 +23,5 @@ If applicable, add error messages and tracebacks to help explain your problem.
**Additional context** **Additional context**
Add any other context about the problem here. Add any other context about the problem here.
(please also see https://instaloader.github.io/contributing.html for how to report a bug)
+9 -2
View File
@@ -1,6 +1,7 @@
--- ---
name: Feature request name: 🚀 Feature Suggestion
about: Suggest an idea for this project about: Suggest an idea for this project 💡
labels: "feature suggestion"
--- ---
@@ -13,5 +14,11 @@ A clear and concise description of what you want to happen.
**Describe alternatives you've considered** **Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered. A clear and concise description of any alternative solutions or features you've considered.
**If the feature request is accepted, would you be willing to submit a PR?**
Yes / No _(Help can be provided if you need assistance submitting a PR)_
**Additional context** **Additional context**
Add any other context about the feature request here. Add any other context about the feature request here.
(please also see https://instaloader.github.io/contributing.html for how to suggest a feature)
+8
View File
@@ -0,0 +1,8 @@
---
name: ❓ Question
about: If you have a question about how to use Instaloader that's not well covered by our documentation
labels: question
---
Your question here...
+24
View File
@@ -0,0 +1,24 @@
name: Mark stale issues and pull requests
on:
schedule:
- cron: "15 16 * * *"
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'There has been no activity on this issue for an extended period of time. This issue will be closed after further 14 days of inactivity.'
stale-pr-message: 'There has been no activity on this Pull Request for an extended period of time. This Pull Request will be closed after further 14 days of inactivity.'
stale-issue-label: 'stale'
stale-pr-label: 'stale'
exempt-issue-label: 'leave open'
exempt-pr-label: 'leave open'
days-before-stale: 90
days-before-close: 14
operations-per-run: 7
+47
View File
@@ -0,0 +1,47 @@
name: Windows EXE build
on: [push, pull_request]
jobs:
build:
runs-on: windows-latest
steps:
- name: Checkout Instaloader repository
uses: actions/checkout@v2
- name: Setup Python
uses: actions/setup-python@v1
with:
python-version: 3.8
architecture: x64
- name: Get the tagged version
id: get_version
env:
GITHUB_REF: ${{ github.ref }}
run: echo ::set-output name=VERSION::${GITHUB_REF/refs\/tags\/v/}
shell: bash
- name: Run custom python script for EXE creation
env:
VERSION_TAG: v${{ steps.get_version.outputs.VERSION }}
run: python deploy/windows/create_exe.py
- name: Create draft release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: v${{ steps.get_version.outputs.VERSION }}
release_name: Version ${{ steps.get_version.outputs.VERSION }}
draft: true
prerelease: contains(steps.get_version.outputs.VERSION, 'a') || contains(steps.get_version.outputs.VERSION, 'b') || contains(steps.get_version.outputs.VERSION, 'c')
- name: Upload EXE as release asset
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
id: upload-release-asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./instaloader-v${{ steps.get_version.outputs.VERSION }}-windows-standalone.zip
asset_name: instaloader-v${{ steps.get_version.outputs.VERSION }}-windows-standalone.zip
asset_content_type: application/zip
+1
View File
@@ -1 +1,2 @@
__pycache__/ __pycache__/
.mypy_cache/
+4 -5
View File
@@ -5,11 +5,10 @@ python:
- "3.6" - "3.6"
- "3.7" - "3.7"
install: install:
- pip install pylint~=2.3.1 requests mypy - pipenv sync --dev
- pip install -r docs/requirements.txt
script: script:
- python3 -m pylint instaloader - pylint instaloader
- python3 -m mypy -m instaloader - mypy -m instaloader
- make -C docs html SPHINXOPTS="-W -n" - make -C docs html SPHINXOPTS="-W -n"
addons: addons:
ssh_known_hosts: aur.archlinux.org ssh_known_hosts: aur.archlinux.org
@@ -28,7 +27,7 @@ deploy:
repo: instaloader/instaloader.github.io repo: instaloader/instaloader.github.io
target-branch: master target-branch: master
on: on:
branch: master tags: true
python: 3.6 python: 3.6
- provider: script - provider: script
script: deploy/arch/deploy.sh $TRAVIS_TAG script: deploy/arch/deploy.sh $TRAVIS_TAG
+16
View File
@@ -0,0 +1,16 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
pylint = "*"
mypy = "*"
sphinx = "*"
pyinstaller = "*"
pefile = "*"
pywin32-ctypes = "*"
psutil = "*"
[packages]
requests = "*"
Generated
+431
View File
@@ -0,0 +1,431 @@
{
"_meta": {
"hash": {
"sha256": "33408a60fda1a56bf680462fad21643ae0159bc13f3b85d8f10ebf3eef7d8725"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"certifi": {
"hashes": [
"sha256:017c25db2a153ce562900032d5bc68e9f191e44e9a0f762f373977de9df1fbb3",
"sha256:25b64c7da4cd7479594d035c08c2d809eb4aab3a26e5a990ea98cc450c320f1f"
],
"version": "==2019.11.28"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
],
"version": "==3.0.4"
},
"idna": {
"hashes": [
"sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407",
"sha256:ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c"
],
"version": "==2.8"
},
"requests": {
"hashes": [
"sha256:11e007a8a2aa0323f5a921e9e6a2d7e4e67d9877e85773fba9ba6419025cbeb4",
"sha256:9cf5292fcd0f598c671cfc1e0d7d1a7f13bb8085e9a590f48c010551dc6c4b31"
],
"index": "pypi",
"version": "==2.22.0"
},
"urllib3": {
"hashes": [
"sha256:2f3db8b19923a873b3e5256dc9c2dedfa883e33d87c690d9c7913e1f40673cdc",
"sha256:87716c2d2a7121198ebcb7ce7cccf6ce5e9ba539041cfbaeecfb641dc0bf6acc"
],
"version": "==1.25.8"
}
},
"develop": {
"alabaster": {
"hashes": [
"sha256:446438bdcca0e05bd45ea2de1668c1d9b032e1a9154c2c259092d77031ddd359",
"sha256:a661d72d58e6ea8a57f7a86e37d86716863ee5e92788398526d58b26a4e4dc02"
],
"version": "==0.7.12"
},
"altgraph": {
"hashes": [
"sha256:1f05a47122542f97028caf78775a095fbe6a2699b5089de8477eb583167d69aa",
"sha256:c623e5f3408ca61d4016f23a681b9adb100802ca3e3da5e718915a9e4052cebe"
],
"version": "==0.17"
},
"astroid": {
"hashes": [
"sha256:71ea07f44df9568a75d0f354c49143a4575d90645e9fead6dfb52c26a85ed13a",
"sha256:840947ebfa8b58f318d42301cf8c0a20fd794a33b61cc4638e28e9e61ba32f42"
],
"version": "==2.3.3"
},
"babel": {
"hashes": [
"sha256:1aac2ae2d0d8ea368fa90906567f5c08463d98ade155c0c4bfedd6a0f7160e38",
"sha256:d670ea0b10f8b723672d3a6abeb87b565b244da220d76b4dba1b66269ec152d4"
],
"version": "==2.8.0"
},
"certifi": {
"hashes": [
"sha256:017c25db2a153ce562900032d5bc68e9f191e44e9a0f762f373977de9df1fbb3",
"sha256:25b64c7da4cd7479594d035c08c2d809eb4aab3a26e5a990ea98cc450c320f1f"
],
"version": "==2019.11.28"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
],
"version": "==3.0.4"
},
"docutils": {
"hashes": [
"sha256:0c5b78adfbf7762415433f5515cd5c9e762339e23369dbe8000d84a4bf4ab3af",
"sha256:c2de3a60e9e7d07be26b7f2b00ca0309c207e06c100f9cc2a94931fc75a478fc"
],
"version": "==0.16"
},
"future": {
"hashes": [
"sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d"
],
"version": "==0.18.2"
},
"idna": {
"hashes": [
"sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407",
"sha256:ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c"
],
"version": "==2.8"
},
"imagesize": {
"hashes": [
"sha256:6965f19a6a2039c7d48bca7dba2473069ff854c36ae6f19d2cde309d998228a1",
"sha256:b1f6b5a4eab1f73479a50fb79fcf729514a900c341d8503d62a62dbc4127a2b1"
],
"version": "==1.2.0"
},
"isort": {
"hashes": [
"sha256:54da7e92468955c4fceacd0c86bd0ec997b0e1ee80d97f67c35a78b719dccab1",
"sha256:6e811fcb295968434526407adb8796944f1988c5b65e8139058f2014cbe100fd"
],
"version": "==4.3.21"
},
"jinja2": {
"hashes": [
"sha256:93187ffbc7808079673ef52771baa950426fd664d3aad1d0fa3e95644360e250",
"sha256:b0eaf100007721b5c16c1fc1eecb87409464edc10469ddc9a22a27a99123be49"
],
"version": "==2.11.1"
},
"lazy-object-proxy": {
"hashes": [
"sha256:0c4b206227a8097f05c4dbdd323c50edf81f15db3b8dc064d08c62d37e1a504d",
"sha256:194d092e6f246b906e8f70884e620e459fc54db3259e60cf69a4d66c3fda3449",
"sha256:1be7e4c9f96948003609aa6c974ae59830a6baecc5376c25c92d7d697e684c08",
"sha256:4677f594e474c91da97f489fea5b7daa17b5517190899cf213697e48d3902f5a",
"sha256:48dab84ebd4831077b150572aec802f303117c8cc5c871e182447281ebf3ac50",
"sha256:5541cada25cd173702dbd99f8e22434105456314462326f06dba3e180f203dfd",
"sha256:59f79fef100b09564bc2df42ea2d8d21a64fdcda64979c0fa3db7bdaabaf6239",
"sha256:8d859b89baf8ef7f8bc6b00aa20316483d67f0b1cbf422f5b4dc56701c8f2ffb",
"sha256:9254f4358b9b541e3441b007a0ea0764b9d056afdeafc1a5569eee1cc6c1b9ea",
"sha256:9651375199045a358eb6741df3e02a651e0330be090b3bc79f6d0de31a80ec3e",
"sha256:97bb5884f6f1cdce0099f86b907aa41c970c3c672ac8b9c8352789e103cf3156",
"sha256:9b15f3f4c0f35727d3a0fba4b770b3c4ebbb1fa907dbcc046a1d2799f3edd142",
"sha256:a2238e9d1bb71a56cd710611a1614d1194dc10a175c1e08d75e1a7bcc250d442",
"sha256:a6ae12d08c0bf9909ce12385803a543bfe99b95fe01e752536a60af2b7797c62",
"sha256:ca0a928a3ddbc5725be2dd1cf895ec0a254798915fb3a36af0964a0a4149e3db",
"sha256:cb2c7c57005a6804ab66f106ceb8482da55f5314b7fcb06551db1edae4ad1531",
"sha256:d74bb8693bf9cf75ac3b47a54d716bbb1a92648d5f781fc799347cfc95952383",
"sha256:d945239a5639b3ff35b70a88c5f2f491913eb94871780ebfabb2568bd58afc5a",
"sha256:eba7011090323c1dadf18b3b689845fd96a61ba0a1dfbd7f24b921398affc357",
"sha256:efa1909120ce98bbb3777e8b6f92237f5d5c8ea6758efea36a473e1d38f7d3e4",
"sha256:f3900e8a5de27447acbf900b4750b0ddfd7ec1ea7fbaf11dfa911141bc522af0"
],
"version": "==1.4.3"
},
"markupsafe": {
"hashes": [
"sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
"sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
"sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
"sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
"sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
"sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
],
"version": "==1.1.1"
},
"mccabe": {
"hashes": [
"sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42",
"sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"
],
"version": "==0.6.1"
},
"mypy": {
"hashes": [
"sha256:0a9a45157e532da06fe56adcfef8a74629566b607fa2c1ac0122d1ff995c748a",
"sha256:2c35cae79ceb20d47facfad51f952df16c2ae9f45db6cb38405a3da1cf8fc0a7",
"sha256:4b9365ade157794cef9685791032521233729cb00ce76b0ddc78749abea463d2",
"sha256:53ea810ae3f83f9c9b452582261ea859828a9ed666f2e1ca840300b69322c474",
"sha256:634aef60b4ff0f650d3e59d4374626ca6153fcaff96ec075b215b568e6ee3cb0",
"sha256:7e396ce53cacd5596ff6d191b47ab0ea18f8e0ec04e15d69728d530e86d4c217",
"sha256:7eadc91af8270455e0d73565b8964da1642fe226665dd5c9560067cd64d56749",
"sha256:7f672d02fffcbace4db2b05369142e0506cdcde20cea0e07c7c2171c4fd11dd6",
"sha256:85baab8d74ec601e86134afe2bcccd87820f79d2f8d5798c889507d1088287bf",
"sha256:87c556fb85d709dacd4b4cb6167eecc5bbb4f0a9864b69136a0d4640fdc76a36",
"sha256:a6bd44efee4dc8c3324c13785a9dc3519b3ee3a92cada42d2b57762b7053b49b",
"sha256:c6d27bd20c3ba60d5b02f20bd28e20091d6286a699174dfad515636cb09b5a72",
"sha256:e2bb577d10d09a2d8822a042a23b8d62bc3b269667c9eb8e60a6edfa000211b1",
"sha256:f97a605d7c8bc2c6d1172c2f0d5a65b24142e11a58de689046e62c2d632ca8c1"
],
"index": "pypi",
"version": "==0.761"
},
"mypy-extensions": {
"hashes": [
"sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d",
"sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"
],
"version": "==0.4.3"
},
"packaging": {
"hashes": [
"sha256:170748228214b70b672c581a3dd610ee51f733018650740e98c7df862a583f73",
"sha256:e665345f9eef0c621aa0bf2f8d78cf6d21904eef16a93f020240b704a57f1334"
],
"version": "==20.1"
},
"pefile": {
"hashes": [
"sha256:a5d6e8305c6b210849b47a6174ddf9c452b2888340b8177874b862ba6c207645"
],
"index": "pypi",
"version": "==2019.4.18"
},
"psutil": {
"hashes": [
"sha256:094f899ac3ef72422b7e00411b4ed174e3c5a2e04c267db6643937ddba67a05b",
"sha256:10b7f75cc8bd676cfc6fa40cd7d5c25b3f45a0e06d43becd7c2d2871cbb5e806",
"sha256:1b1575240ca9a90b437e5a40db662acd87bbf181f6aa02f0204978737b913c6b",
"sha256:21231ef1c1a89728e29b98a885b8e0a8e00d09018f6da5cdc1f43f988471a995",
"sha256:28f771129bfee9fc6b63d83a15d857663bbdcae3828e1cb926e91320a9b5b5cd",
"sha256:70387772f84fa5c3bb6a106915a2445e20ac8f9821c5914d7cbde148f4d7ff73",
"sha256:b560f5cd86cf8df7bcd258a851ca1ad98f0d5b8b98748e877a0aec4e9032b465",
"sha256:b74b43fecce384a57094a83d2778cdfc2e2d9a6afaadd1ebecb2e75e0d34e10d",
"sha256:e85f727ffb21539849e6012f47b12f6dd4c44965e56591d8dec6e8bc9ab96f4a",
"sha256:fd2e09bb593ad9bdd7429e779699d2d47c1268cbde4dda95fcd1bd17544a0217",
"sha256:ffad8eb2ac614518bbe3c0b8eb9dffdb3a8d2e3a7d5da51c5b974fb723a5c5aa"
],
"index": "pypi",
"version": "==5.6.7"
},
"pygments": {
"hashes": [
"sha256:2a3fe295e54a20164a9df49c75fa58526d3be48e14aceba6d6b1e8ac0bfd6f1b",
"sha256:98c8aa5a9f778fcd1026a17361ddaf7330d1b7c62ae97c3bb0ae73e0b9b6b0fe"
],
"version": "==2.5.2"
},
"pyinstaller": {
"hashes": [
"sha256:3730fa80d088f8bb7084d32480eb87cbb4ddb64123363763cf8f2a1378c1c4b7"
],
"index": "pypi",
"version": "==3.6"
},
"pylint": {
"hashes": [
"sha256:3db5468ad013380e987410a8d6956226963aed94ecb5f9d3a28acca6d9ac36cd",
"sha256:886e6afc935ea2590b462664b161ca9a5e40168ea99e5300935f6591ad467df4"
],
"index": "pypi",
"version": "==2.4.4"
},
"pyparsing": {
"hashes": [
"sha256:4c830582a84fb022400b85429791bc551f1f4871c33f23e44f353119e92f969f",
"sha256:c342dccb5250c08d45fd6f8b4a559613ca603b57498511740e65cd11a2e7dcec"
],
"version": "==2.4.6"
},
"pytz": {
"hashes": [
"sha256:1c557d7d0e871de1f5ccd5833f60fb2550652da6be2693c1e02300743d21500d",
"sha256:b02c06db6cf09c12dd25137e563b31700d3b80fcc4ad23abb7a315f2789819be"
],
"version": "==2019.3"
},
"pywin32-ctypes": {
"hashes": [
"sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942",
"sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98"
],
"index": "pypi",
"version": "==0.2.0"
},
"requests": {
"hashes": [
"sha256:11e007a8a2aa0323f5a921e9e6a2d7e4e67d9877e85773fba9ba6419025cbeb4",
"sha256:9cf5292fcd0f598c671cfc1e0d7d1a7f13bb8085e9a590f48c010551dc6c4b31"
],
"index": "pypi",
"version": "==2.22.0"
},
"six": {
"hashes": [
"sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
"sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
],
"version": "==1.14.0"
},
"snowballstemmer": {
"hashes": [
"sha256:209f257d7533fdb3cb73bdbd24f436239ca3b2fa67d56f6ff88e86be08cc5ef0",
"sha256:df3bac3df4c2c01363f3dd2cfa78cce2840a79b9f1c2d2de9ce8d31683992f52"
],
"version": "==2.0.0"
},
"sphinx": {
"hashes": [
"sha256:298537cb3234578b2d954ff18c5608468229e116a9757af3b831c2b2b4819159",
"sha256:e6e766b74f85f37a5f3e0773a1e1be8db3fcb799deb58ca6d18b70b0b44542a5"
],
"index": "pypi",
"version": "==2.3.1"
},
"sphinxcontrib-applehelp": {
"hashes": [
"sha256:edaa0ab2b2bc74403149cb0209d6775c96de797dfd5b5e2a71981309efab3897",
"sha256:fb8dee85af95e5c30c91f10e7eb3c8967308518e0f7488a2828ef7bc191d0d5d"
],
"version": "==1.0.1"
},
"sphinxcontrib-devhelp": {
"hashes": [
"sha256:6c64b077937330a9128a4da74586e8c2130262f014689b4b89e2d08ee7294a34",
"sha256:9512ecb00a2b0821a146736b39f7aeb90759834b07e81e8cc23a9c70bacb9981"
],
"version": "==1.0.1"
},
"sphinxcontrib-htmlhelp": {
"hashes": [
"sha256:4670f99f8951bd78cd4ad2ab962f798f5618b17675c35c5ac3b2132a14ea8422",
"sha256:d4fd39a65a625c9df86d7fa8a2d9f3cd8299a3a4b15db63b50aac9e161d8eff7"
],
"version": "==1.0.2"
},
"sphinxcontrib-jsmath": {
"hashes": [
"sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178",
"sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"
],
"version": "==1.0.1"
},
"sphinxcontrib-qthelp": {
"hashes": [
"sha256:513049b93031beb1f57d4daea74068a4feb77aa5630f856fcff2e50de14e9a20",
"sha256:79465ce11ae5694ff165becda529a600c754f4bc459778778c7017374d4d406f"
],
"version": "==1.0.2"
},
"sphinxcontrib-serializinghtml": {
"hashes": [
"sha256:c0efb33f8052c04fd7a26c0a07f1678e8512e0faec19f4aa8f2473a8b81d5227",
"sha256:db6615af393650bf1151a6cd39120c29abaf93cc60db8c48eb2dddbfdc3a9768"
],
"version": "==1.1.3"
},
"typed-ast": {
"hashes": [
"sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355",
"sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919",
"sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa",
"sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652",
"sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75",
"sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01",
"sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d",
"sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1",
"sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907",
"sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c",
"sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3",
"sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b",
"sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614",
"sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb",
"sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b",
"sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41",
"sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6",
"sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34",
"sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe",
"sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4",
"sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7"
],
"version": "==1.4.1"
},
"typing-extensions": {
"hashes": [
"sha256:091ecc894d5e908ac75209f10d5b4f118fbdb2eb1ede6a63544054bb1edb41f2",
"sha256:910f4656f54de5993ad9304959ce9bb903f90aadc7c67a0bef07e678014e892d",
"sha256:cf8b63fedea4d89bab840ecbb93e75578af28f76f66c35889bd7065f5af88575"
],
"version": "==3.7.4.1"
},
"urllib3": {
"hashes": [
"sha256:2f3db8b19923a873b3e5256dc9c2dedfa883e33d87c690d9c7913e1f40673cdc",
"sha256:87716c2d2a7121198ebcb7ce7cccf6ce5e9ba539041cfbaeecfb641dc0bf6acc"
],
"version": "==1.25.8"
},
"wrapt": {
"hashes": [
"sha256:565a021fd19419476b9362b05eeaa094178de64f8361e44468f9e9d7843901e1"
],
"version": "==1.11.2"
}
}
}
+3 -6
View File
@@ -2,7 +2,7 @@
.. badges-start .. badges-start
|travis| |pypi| |pyversion| |license| |aur| |contributors| |downloads| |saythanks| |travis| |pypi| |pyversion| |license| |aur| |contributors| |downloads|
.. |travis| image:: https://img.shields.io/travis/instaloader/instaloader/master.svg .. |travis| image:: https://img.shields.io/travis/instaloader/instaloader/master.svg
:alt: Travis-CI Build Status :alt: Travis-CI Build Status
@@ -23,10 +23,6 @@
:alt: Contributor Count :alt: Contributor Count
:target: https://github.com/instaloader/instaloader/graphs/contributors :target: https://github.com/instaloader/instaloader/graphs/contributors
.. |saythanks| image:: https://img.shields.io/badge/Say%20Thanks-!-ff7efd.svg
:alt: Say Thanks!
:target: https://saythanks.io/to/aandergr
.. |aur| image:: https://img.shields.io/aur/version/instaloader.svg .. |aur| image:: https://img.shields.io/aur/version/instaloader.svg
:alt: Arch User Repository Package :alt: Arch User Repository Package
:target: https://aur.archlinux.org/packages/instaloader/ :target: https://aur.archlinux.org/packages/instaloader/
@@ -57,7 +53,8 @@
:: ::
instaloader [--comments] [--geotags] [--stories] [--highlights] [--tagged] instaloader [--comments] [--geotags]
[--stories] [--highlights] [--tagged] [--igtv]
[--login YOUR-USERNAME] [--fast-update] [--login YOUR-USERNAME] [--fast-update]
profile | "#hashtag" | :stories | :feed | :saved profile | "#hashtag" | :stories | :feed | :saved
+86
View File
@@ -0,0 +1,86 @@
import hashlib
import os
import re
import shutil
import subprocess
import sys
# copy the required files into repo root
shutil.copy('docs/favicon.ico', '.')
shutil.copy('deploy/windows/instaloader.spec', '.')
shutil.unpack_archive('deploy/windows/ps', '.', 'xztar')
code = """
import psutil
import subprocess
def __main():
grandparentpid = psutil.Process(os.getppid()).ppid()
grandparentpidsearchstring = ' ' + str(grandparentpid) + ' '
if hasattr(sys, "_MEIPASS"):
ps = os.path.join(sys._MEIPASS, 'tasklist.exe')
else:
ps = 'tasklist'
popen = subprocess.Popen(ps, stdout=subprocess.PIPE, universal_newlines=True)
for examine in iter(popen.stdout.readline, ""):
if grandparentpidsearchstring in examine:
pname = examine
break
popen.stdout.close()
return_code = popen.wait()
if return_code:
raise subprocess.CalledProcessError(return_code, ps)
if pname[0:12] == 'explorer.exe':
subprocess.Popen("cmd /K \\\"{0}\\\"".format(os.path.splitext(os.path.basename(sys.argv[0]))[0]))
else:
main()
if __name__ == "__main__":
__main()
"""
with open('instaloader/__main__.py', 'r') as f:
# adjust imports for changed file structure
regex = re.compile(r'from (?:(\.[^ ]+ )|\.( ))import')
lines = [regex.sub(r'from instaloader\1\2import', line) for line in f.readlines()]
# insert code for magic exe behavior
index = lines.index('if __name__ == "__main__":\n')
code_lines = [cl + '\n' for cl in code.splitlines()]
for i, code_line in enumerate(code_lines):
if i + index < len(lines):
lines[i + index] = code_line
else:
lines.extend(code_lines[i:])
break
with open('__main__.py', 'w+') as f:
f.writelines(lines)
# install dependencies and invoke PyInstaller
commands = ["pip install pipenv==2018.11.26",
"pipenv sync --dev",
"pipenv run pyinstaller --log-level=DEBUG instaloader.spec"]
for command in commands:
print()
print('#' * (len(command) + 6))
print('## {} ##'.format(command))
print('#' * (len(command) + 6))
print(flush=True)
err = subprocess.Popen(command).wait()
if err != 0:
sys.exit(err)
# calculate and store MD5 hash for created executable
hash_md5 = hashlib.md5()
with open('dist/instaloader.exe', 'rb') as f:
for chunk in iter(lambda: f.read(4096), b''):
hash_md5.update(chunk)
with open('dist/instaloader.exe.md5', 'w+') as f:
f.write('{} *instaloader.exe\n'.format(hash_md5.hexdigest()))
# Create ZIP file
shutil.make_archive('instaloader-{}-windows-standalone'.format(os.getenv('VERSION_TAG')), 'zip', 'dist')
+28
View File
@@ -0,0 +1,28 @@
# -*- mode: python -*-
block_cipher = None
a = Analysis(['__main__.py'],
binaries=[],
datas=[('tasklist.exe', '.'), ('framedyn.dll', '.')],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='instaloader',
debug=False,
strip=False,
upx=False,
runtime_tmpdir=None,
console=True , icon='favicon.ico')
BIN
View File
Binary file not shown.
-3
View File
@@ -51,9 +51,6 @@
<li class="nav-item"> <li class="nav-item">
<a target="_blank" class="nav-link" href="https://github.com/instaloader/instaloader">GitHub</a> <a target="_blank" class="nav-link" href="https://github.com/instaloader/instaloader">GitHub</a>
</li> </li>
<li class="nav-item d-sm-none d-lg-block">
<a target="_blank" class="nav-link" href="https://saythanks.io/to/aandergr">Say Thanks!</a>
</li>
</ul> </ul>
<form class="form-inline" action="search.html" method="get"> <form class="form-inline" action="search.html" method="get">
<div class="input-group my-2 my-sm-0 ml-2"> <div class="input-group my-2 my-sm-0 ml-2">
+15 -2
View File
@@ -155,6 +155,14 @@ Profiles
.. autoclass:: Profile .. autoclass:: Profile
:no-show-inheritance: :no-show-inheritance:
TopSearchResults
""""""""""""""""
.. autoclass:: TopSearchResults
:no-show-inheritance:
.. versionadded:: 4.3
Loading and Saving Loading and Saving
"""""""""""""""""" """"""""""""""""""
@@ -179,8 +187,6 @@ Exceptions
.. autoexception:: QueryReturnedBadRequestException .. autoexception:: QueryReturnedBadRequestException
.. autoexception:: QueryReturnedNotFoundException
.. autoexception:: QueryReturnedForbiddenException .. autoexception:: QueryReturnedForbiddenException
.. autoexception:: ProfileNotExistsException .. autoexception:: ProfileNotExistsException
@@ -203,6 +209,13 @@ Exceptions
.. autoexception:: PostChangedException .. autoexception:: PostChangedException
.. autoexception:: QueryReturnedNotFoundException
.. versionchanged:: 4.3
QueryReturnedNotFoundException now inherits ConnectionException
to retry on 404 errors.
.. autoexception:: TooManyRequestsException .. autoexception:: TooManyRequestsException
``InstaloaderContext`` (Low-level functions) ``InstaloaderContext`` (Low-level functions)
+7 -3
View File
@@ -77,10 +77,13 @@ Instaloader supports the following targets:
instructs Instaloader to also **download the user's stories**, instructs Instaloader to also **download the user's stories**,
- :option:`--highlights` - :option:`--highlights`
to **download highlights of each profile that is downloaded**, and to **download highlights of each profile that is downloaded**,
- :option:`--tagged` - :option:`--tagged`
to **download posts where the user is tagged**. to **download posts where the user is tagged**, and
- :option:`--igtv`
to **download IGTV videos**.
- ``"#hashtag"`` - ``"#hashtag"``
Posts with a certain **hashtag** (the quotes are usually necessary), Posts with a certain **hashtag** (the quotes are usually necessary),
@@ -89,6 +92,7 @@ Instaloader supports the following targets:
Posts tagged with a given location; the location ID is the numerical ID Posts tagged with a given location; the location ID is the numerical ID
Instagram labels a location with (e.g. Instagram labels a location with (e.g.
\https://www.instagram.com/explore/locations/**362629379**/plymouth-naval-memorial/). \https://www.instagram.com/explore/locations/**362629379**/plymouth-naval-memorial/).
Requires :option:`--login`.
.. versionadded:: 4.2 .. versionadded:: 4.2
@@ -122,7 +126,7 @@ downloads the pictures and videos and their captions. You can specify
- :option:`--geotags` - :option:`--geotags`
**download geotags** of each post and save them as **download geotags** of each post and save them as
Google Maps link, Google Maps link (requires :option:`--login`),
For a reference of all supported command line options, see For a reference of all supported command line options, see
:ref:`command-line-options`. :ref:`command-line-options`.
+13 -2
View File
@@ -61,8 +61,7 @@ What to Download of each Post
**Download geotags** when available. Geotags are stored as a text file with **Download geotags** when available. Geotags are stored as a text file with
the location's name and a Google Maps link. This requires an additional the location's name and a Google Maps link. This requires an additional
request to the Instagram server for each picture, which is why it is disabled request to the Instagram server for each picture. Requires :option:`--login`.
by default.
.. option:: --comments, -C .. option:: --comments, -C
@@ -130,6 +129,12 @@ What to Download of each Profile
.. versionadded:: 4.1 .. versionadded:: 4.1
.. option:: --igtv
Also download IGTV videos.
.. versionadded:: 4.3
.. option:: --stories-only .. option:: --stories-only
.. deprecated:: 4.1 .. deprecated:: 4.1
@@ -238,6 +243,12 @@ How to Download
.. versionadded:: 4.2 .. versionadded:: 4.2
.. option:: --request-timeout N
Seconds to wait before timing out a connection request.
.. versionadded:: 4.3
Miscellaneous Options Miscellaneous Options
^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^
+2 -2
View File
@@ -2,13 +2,13 @@ import instaloader
L = instaloader.Instaloader() L = instaloader.Instaloader()
posts = L.get_hashtag_posts('milfgarden') posts = L.get_hashtag_posts('urbanphotography')
users = set() users = set()
for post in posts: for post in posts:
if not post.owner_profile in users: if not post.owner_profile in users:
L.download_post(post, '#milfgarden') L.download_post(post, '#urbanphotography')
users.add(post.owner_profile) users.add(post.owner_profile)
else: else:
print("{} from {} skipped.".format(post, post.owner_profile)) print("{} from {} skipped.".format(post, post.owner_profile))
+2 -2
View File
@@ -5,7 +5,7 @@ import instaloader
L = instaloader.Instaloader() L = instaloader.Instaloader()
posts = L.get_hashtag_posts('milfgarden') posts = L.get_hashtag_posts('urbanphotography')
# or # or
# posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts() # posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts()
@@ -14,4 +14,4 @@ UNTIL = datetime(2015, 3, 1)
for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)): for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
print(post.date) print(post.date)
L.download_post(post, '#milfgarden') L.download_post(post, '#urbanphotography')
+3 -1
View File
@@ -9,7 +9,9 @@ X_percentage = 10 # percentage of posts that should be downloaded
L = Instaloader() L = Instaloader()
profile = Profile.from_username(L.context, PROFILE) profile = Profile.from_username(L.context, PROFILE)
posts_sorted_by_likes = sorted(profile.get_posts(), key = lambda p: p.likes + p.comments) posts_sorted_by_likes = sorted(profile.get_posts(),
key=lambda p: p.likes + p.comments,
reverse=True)
for post in islice(posts_sorted_by_likes, ceil(profile.mediacount * X_percentage / 100)): for post in islice(posts_sorted_by_likes, ceil(profile.mediacount * X_percentage / 100)):
L.download_post(post, PROFILE) L.download_post(post, PROFILE)
+1 -1
View File
@@ -47,7 +47,7 @@ autodoc_default_flags = ['show-inheritance', 'members', 'undoc-members']
autodoc_member_order = 'bysource' autodoc_member_order = 'bysource'
intersphinx_mapping = {'python': ('https://docs.python.org/3', None), intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
'requests': ('http://docs.python-requests.org/en/master', None)} 'requests': ('https://requests.kennethreitz.org/en/master/', None)}
extlinks = {'issue': ('https://github.com/instaloader/instaloader/issues/%s', 'issue #')} extlinks = {'issue': ('https://github.com/instaloader/instaloader/issues/%s', 'issue #')}
+15 -4
View File
@@ -15,10 +15,8 @@ Reporting Bugs
-------------- --------------
If you encounter a bug, do not hesitate to report it in our If you encounter a bug, do not hesitate to report it in our
`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. Since `Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. When
Instaloader is actively developed, the majority of bugs is fixed within only reporting a problem, please keep the following in mind:
**4 days** after being reported. When reporting a problem, please keep the
following in mind:
- Ensure you **use the latest version** of Instaloader. The currently-installed - Ensure you **use the latest version** of Instaloader. The currently-installed
version can be found out with ``instaloader --version``. version can be found out with ``instaloader --version``.
@@ -66,6 +64,16 @@ few things to consider:
Suggesting Features Suggesting Features
------------------- -------------------
.. goal-start
Instaloader's goal is to mimick the browser's behavior to access the data that
is available through the Instagram web interface, pack this data into complete
and easily-(re)usable python objects, and provide a user interface for the most
common downloading and metadata collection tasks, without changing any of the
online data.
.. goal-end
If you have an idea of how Instaloader should be enhanced, but do not want to If you have an idea of how Instaloader should be enhanced, but do not want to
implement the feature yourself, feel free to open a ticket in our implement the feature yourself, feel free to open a ticket in our
`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__. `Issue Tracker <https://github.com/instaloader/instaloader/issues>`__.
@@ -82,6 +90,9 @@ Please consider the following:
invoke the new function? Which problem would it solve? If new information is invoke the new function? Which problem would it solve? If new information is
obtained, how would it be further processed? obtained, how would it be further processed?
- If not obvious, briefly motivate how your suggested feature **conforms with
Instaloader's project goal**.
- **Be patient**. Naturally, bugs and pull requests have a higher priority than - **Be patient**. Naturally, bugs and pull requests have a higher priority than
feature suggestions. Keep in mind that this is a free software project, and feature suggestions. Keep in mind that this is a free software project, and
unfortunately we only have limited time to work on it. unfortunately we only have limited time to work on it.
+2 -1
View File
@@ -45,7 +45,8 @@ See :ref:`install` for more options on how to install Instaloader.
:: ::
instaloader [--comments] [--geotags] [--stories] [--highlights] [--tagged] instaloader [--comments] [--geotags]
[--stories] [--highlights] [--tagged] [--igtv]
[--login YOUR-USERNAME] [--fast-update] [--login YOUR-USERNAME] [--fast-update]
profile | "#hashtag" | %location_id | profile | "#hashtag" | %location_id |
:stories | :feed | :saved :stories | :feed | :saved
+2 -2
View File
@@ -1,7 +1,7 @@
"""Download pictures (or videos) along with their captions and other metadata from Instagram.""" """Download pictures (or videos) along with their captions and other metadata from Instagram."""
__version__ = '4.2.7' __version__ = '4.3'
try: try:
@@ -16,4 +16,4 @@ from .exceptions import *
from .instaloader import Instaloader from .instaloader import Instaloader
from .instaloadercontext import InstaloaderContext from .instaloadercontext import InstaloaderContext
from .structures import (Highlight, Post, PostSidecarNode, PostComment, PostCommentAnswer, PostLocation, Profile, Story, from .structures import (Highlight, Post, PostSidecarNode, PostComment, PostCommentAnswer, PostLocation, Profile, Story,
StoryItem, load_structure_from_file, save_structure_to_file) StoryItem, TopSearchResults, load_structure_from_file, save_structure_to_file)
+25 -9
View File
@@ -19,7 +19,8 @@ def usage_string():
argv0 = os.path.basename(sys.argv[0]) argv0 = os.path.basename(sys.argv[0])
argv0 = "instaloader" if argv0 == "__main__.py" else argv0 argv0 = "instaloader" if argv0 == "__main__.py" else argv0
return """ return """
{0} [--comments] [--geotags] [--stories] [--highlights] [--tagged] {0} [--comments] [--geotags]
{2:{1}} [--stories] [--highlights] [--tagged] [--igtv]
{2:{1}} [--login YOUR-USERNAME] [--fast-update] {2:{1}} [--login YOUR-USERNAME] [--fast-update]
{2:{1}} profile | "#hashtag" | %%location_id | :stories | :feed | :saved {2:{1}} profile | "#hashtag" | %%location_id | :stories | :feed | :saved
{0} --help""".format(argv0, len(argv0), '') {0} --help""".format(argv0, len(argv0), '')
@@ -61,7 +62,10 @@ def _main(instaloader: Instaloader, targetlist: List[str],
username: Optional[str] = None, password: Optional[str] = None, username: Optional[str] = None, password: Optional[str] = None,
sessionfile: Optional[str] = None, sessionfile: Optional[str] = None,
download_profile_pic: bool = True, download_posts=True, download_profile_pic: bool = True, download_posts=True,
download_stories: bool = False, download_highlights: bool = False, download_tagged: bool = False, download_stories: bool = False,
download_highlights: bool = False,
download_tagged: bool = False,
download_igtv: bool = False,
fast_update: bool = False, fast_update: bool = False,
max_count: Optional[int] = None, post_filter_str: Optional[str] = None, max_count: Optional[int] = None, post_filter_str: Optional[str] = None,
storyitem_filter_str: Optional[str] = None) -> None: storyitem_filter_str: Optional[str] = None) -> None:
@@ -98,6 +102,9 @@ def _main(instaloader: Instaloader, targetlist: List[str],
else: else:
instaloader.interactive_login(username) instaloader.interactive_login(username)
instaloader.context.log("Logged in as %s." % username) instaloader.context.log("Logged in as %s." % username)
# since 4.2.9 login is required for geotags
if instaloader.download_geotags and not instaloader.context.is_logged_in:
instaloader.context.error("Warning: Use --login to download geotags of posts.")
# Try block for KeyboardInterrupt (save session on ^C) # Try block for KeyboardInterrupt (save session on ^C)
profiles = set() profiles = set()
anonymous_retry_profiles = set() anonymous_retry_profiles = set()
@@ -155,7 +162,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
try: try:
profile = instaloader.check_profile_id(target) profile = instaloader.check_profile_id(target)
if instaloader.context.is_logged_in and profile.has_blocked_viewer: if instaloader.context.is_logged_in and profile.has_blocked_viewer:
if download_profile_pic or ((download_posts or download_tagged) and not profile.is_private): if download_profile_pic or ((download_posts or download_tagged or download_igtv)
and not profile.is_private):
raise ProfileNotExistsException("{} blocked you; But we download her anonymously." raise ProfileNotExistsException("{} blocked you; But we download her anonymously."
.format(target)) .format(target))
else: else:
@@ -166,7 +174,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
# Not only our profile.has_blocked_viewer condition raises ProfileNotExistsException, # Not only our profile.has_blocked_viewer condition raises ProfileNotExistsException,
# check_profile_id() also does, since access to blocked profile may be responded with 404. # check_profile_id() also does, since access to blocked profile may be responded with 404.
if instaloader.context.is_logged_in and (download_profile_pic or download_posts or if instaloader.context.is_logged_in and (download_profile_pic or download_posts or
download_tagged): download_tagged or download_igtv):
instaloader.context.log(err) instaloader.context.log(err)
instaloader.context.log("Trying again anonymously, helps in case you are just blocked.") instaloader.context.log("Trying again anonymously, helps in case you are just blocked.")
with instaloader.anonymous_copy() as anonymous_loader: with instaloader.anonymous_copy() as anonymous_loader:
@@ -182,14 +190,15 @@ def _main(instaloader: Instaloader, targetlist: List[str],
if profiles and download_profile_pic and not instaloader.context.is_logged_in: if profiles and download_profile_pic and not instaloader.context.is_logged_in:
instaloader.context.error("Warning: Use --login to download HD version of profile pictures.") instaloader.context.error("Warning: Use --login to download HD version of profile pictures.")
instaloader.download_profiles(profiles, instaloader.download_profiles(profiles,
download_profile_pic, download_posts, download_tagged, download_highlights, download_profile_pic, download_posts, download_tagged, download_igtv,
download_stories, fast_update, post_filter, storyitem_filter) download_highlights, download_stories,
fast_update, post_filter, storyitem_filter)
if anonymous_retry_profiles: if anonymous_retry_profiles:
instaloader.context.log("Downloading anonymously: {}" instaloader.context.log("Downloading anonymously: {}"
.format(' '.join([p.username for p in anonymous_retry_profiles]))) .format(' '.join([p.username for p in anonymous_retry_profiles])))
with instaloader.anonymous_copy() as anonymous_loader: with instaloader.anonymous_copy() as anonymous_loader:
anonymous_loader.download_profiles(anonymous_retry_profiles, anonymous_loader.download_profiles(anonymous_retry_profiles,
download_profile_pic, download_posts, download_tagged, download_profile_pic, download_posts, download_tagged, download_igtv,
fast_update=fast_update, post_filter=post_filter) fast_update=fast_update, post_filter=post_filter)
except KeyboardInterrupt: except KeyboardInterrupt:
print("\nInterrupted by user.", file=sys.stderr) print("\nInterrupted by user.", file=sys.stderr)
@@ -223,7 +232,8 @@ def main():
help="Download all followees of profile. Requires --login. " help="Download all followees of profile. Requires --login. "
"Consider using :feed rather than @yourself.") "Consider using :feed rather than @yourself.")
g_targets.add_argument('_hashtag', nargs='*', metavar='"#hashtag"', help="Download #hashtag.") g_targets.add_argument('_hashtag', nargs='*', metavar='"#hashtag"', help="Download #hashtag.")
g_targets.add_argument('_location', nargs='*', metavar='%location_id', help="Download %%location_id.") g_targets.add_argument('_location', nargs='*', metavar='%location_id',
help="Download %%location_id. Requires --login.")
g_targets.add_argument('_feed', nargs='*', metavar=":feed", g_targets.add_argument('_feed', nargs='*', metavar=":feed",
help="Download pictures from your feed. Requires --login.") help="Download pictures from your feed. Requires --login.")
g_targets.add_argument('_stories', nargs='*', metavar=":stories", g_targets.add_argument('_stories', nargs='*', metavar=":stories",
@@ -258,7 +268,7 @@ def main():
help='Download geotags when available. Geotags are stored as a ' help='Download geotags when available. Geotags are stored as a '
'text file with the location\'s name and a Google Maps link. ' 'text file with the location\'s name and a Google Maps link. '
'This requires an additional request to the Instagram ' 'This requires an additional request to the Instagram '
'server for each picture, which is why it is disabled by default.') 'server for each picture. Requires --login.')
g_post.add_argument('-C', '--comments', action='store_true', g_post.add_argument('-C', '--comments', action='store_true',
help='Download and update comments for each post. ' help='Download and update comments for each post. '
'This requires an additional request to the Instagram ' 'This requires an additional request to the Instagram '
@@ -283,6 +293,8 @@ def main():
help='Also download highlights of each profile that is downloaded. Requires --login.') help='Also download highlights of each profile that is downloaded. Requires --login.')
g_prof.add_argument('--tagged', action='store_true', g_prof.add_argument('--tagged', action='store_true',
help='Also download posts where each profile is tagged.') help='Also download posts where each profile is tagged.')
g_prof.add_argument('--igtv', action='store_true',
help='Also download IGTV videos.')
g_cond = parser.add_argument_group("Which Posts to Download") g_cond = parser.add_argument_group("Which Posts to Download")
@@ -339,6 +351,8 @@ def main():
help='Tries to ensure downloaded images avoid corruption in case of unexpected interruption. ' help='Tries to ensure downloaded images avoid corruption in case of unexpected interruption. '
'If the last picture is corrupted, Instaloader will fix the picture the next time it is run. ' 'If the last picture is corrupted, Instaloader will fix the picture the next time it is run. '
'Requires the JSON metadata to be saved.') 'Requires the JSON metadata to be saved.')
g_how.add_argument('--request-timeout', metavar='N', type=float,
help='seconds to wait before timing out a connection request')
g_misc = parser.add_argument_group('Miscellaneous Options') g_misc = parser.add_argument_group('Miscellaneous Options')
g_misc.add_argument('-q', '--quiet', action='store_true', g_misc.add_argument('-q', '--quiet', action='store_true',
@@ -393,6 +407,7 @@ def main():
post_metadata_txt_pattern=post_metadata_txt_pattern, post_metadata_txt_pattern=post_metadata_txt_pattern,
storyitem_metadata_txt_pattern=storyitem_metadata_txt_pattern, storyitem_metadata_txt_pattern=storyitem_metadata_txt_pattern,
max_connection_attempts=args.max_connection_attempts, max_connection_attempts=args.max_connection_attempts,
request_timeout=args.request_timeout,
commit_mode=args.commit_mode) commit_mode=args.commit_mode)
_main(loader, _main(loader,
args.profile, args.profile,
@@ -404,6 +419,7 @@ def main():
download_stories=download_stories, download_stories=download_stories,
download_highlights=args.highlights, download_highlights=args.highlights,
download_tagged=args.tagged, download_tagged=args.tagged,
download_igtv=args.igtv,
fast_update=args.fast_update, fast_update=args.fast_update,
max_count=int(args.count) if args.count is not None else None, max_count=int(args.count) if args.count is not None else None,
post_filter_str=args.post_filter, post_filter_str=args.post_filter,
+4 -4
View File
@@ -9,10 +9,6 @@ class QueryReturnedBadRequestException(InstaloaderException):
pass pass
class QueryReturnedNotFoundException(InstaloaderException):
pass
class QueryReturnedForbiddenException(InstaloaderException): class QueryReturnedForbiddenException(InstaloaderException):
pass pass
@@ -62,5 +58,9 @@ class PostChangedException(InstaloaderException):
pass pass
class QueryReturnedNotFoundException(ConnectionException):
pass
class TooManyRequestsException(ConnectionException): class TooManyRequestsException(ConnectionException):
pass pass
+136 -47
View File
@@ -47,8 +47,6 @@ def _requires_login(func: Callable) -> Callable:
if not instaloader.context.is_logged_in: if not instaloader.context.is_logged_in:
raise LoginRequiredException("--login=USERNAME required.") raise LoginRequiredException("--login=USERNAME required.")
return func(instaloader, *args, **kwargs) return func(instaloader, *args, **kwargs)
docstring_text = ":raises LoginRequiredException: If called without being logged in.\n"
call.__doc__ = call.__doc__ + docstring_text if call.__doc__ is not None else docstring_text
return call return call
@@ -144,6 +142,7 @@ class Instaloader:
:param storyitem_metadata_txt_pattern: :option:`--storyitem-metadata-txt`, default is empty (=none) :param storyitem_metadata_txt_pattern: :option:`--storyitem-metadata-txt`, default is empty (=none)
:param max_connection_attempts: :option:`--max-connection-attempts` :param max_connection_attempts: :option:`--max-connection-attempts`
:param commit_mode: :option:`--commit-mode` :param commit_mode: :option:`--commit-mode`
:param request_timeout: :option:`--request-timeout`, set per-request timeout (seconds)
.. attribute:: context .. attribute:: context
@@ -151,7 +150,8 @@ class Instaloader:
""" """
def __init__(self, def __init__(self,
sleep: bool = True, quiet: bool = False, sleep: bool = True,
quiet: bool = False,
user_agent: Optional[str] = None, user_agent: Optional[str] = None,
dirname_pattern: Optional[str] = None, dirname_pattern: Optional[str] = None,
filename_pattern: Optional[str] = None, filename_pattern: Optional[str] = None,
@@ -165,9 +165,10 @@ class Instaloader:
post_metadata_txt_pattern: str = None, post_metadata_txt_pattern: str = None,
storyitem_metadata_txt_pattern: str = None, storyitem_metadata_txt_pattern: str = None,
max_connection_attempts: int = 3, max_connection_attempts: int = 3,
request_timeout: Optional[float] = None,
commit_mode: bool = False): commit_mode: bool = False):
self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts) self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts, request_timeout)
# configuration parameters # configuration parameters
self.dirname_pattern = dirname_pattern or "{target}" self.dirname_pattern = dirname_pattern or "{target}"
@@ -193,15 +194,24 @@ class Instaloader:
@contextmanager @contextmanager
def anonymous_copy(self): def anonymous_copy(self):
"""Yield an anonymous, otherwise equally-configured copy of an Instaloader instance; Then copy its error log.""" """Yield an anonymous, otherwise equally-configured copy of an Instaloader instance; Then copy its error log."""
new_loader = Instaloader(self.context.sleep, self.context.quiet, self.context.user_agent, self.dirname_pattern, new_loader = Instaloader(
self.filename_pattern, download_pictures=self.download_pictures, sleep=self.context.sleep,
download_videos=self.download_videos, quiet=self.context.quiet,
download_video_thumbnails=self.download_video_thumbnails, user_agent=self.context.user_agent,
download_geotags=self.download_geotags, download_comments=self.download_comments, dirname_pattern=self.dirname_pattern,
save_metadata=self.save_metadata, compress_json=self.compress_json, filename_pattern=self.filename_pattern,
post_metadata_txt_pattern=self.post_metadata_txt_pattern, download_pictures=self.download_pictures,
storyitem_metadata_txt_pattern=self.storyitem_metadata_txt_pattern, download_videos=self.download_videos,
max_connection_attempts=self.context.max_connection_attempts) download_video_thumbnails=self.download_video_thumbnails,
download_geotags=self.download_geotags,
download_comments=self.download_comments,
save_metadata=self.save_metadata,
compress_json=self.compress_json,
post_metadata_txt_pattern=self.post_metadata_txt_pattern,
storyitem_metadata_txt_pattern=self.storyitem_metadata_txt_pattern,
max_connection_attempts=self.context.max_connection_attempts,
request_timeout=self.context.request_timeout,
commit_mode=self.commit_mode)
yield new_loader yield new_loader
self.context.error_log.extend(new_loader.context.error_log) self.context.error_log.extend(new_loader.context.error_log)
new_loader.context.error_log = [] # avoid double-printing of errors new_loader.context.error_log = [] # avoid double-printing of errors
@@ -246,6 +256,7 @@ class Instaloader:
filename += '.json.xz' filename += '.json.xz'
else: else:
filename += '.json' filename += '.json'
os.makedirs(os.path.dirname(filename), exist_ok=True)
save_structure_to_file(structure, filename) save_structure_to_file(structure, filename)
if isinstance(structure, (Post, StoryItem)): if isinstance(structure, (Post, StoryItem)):
# log 'json ' message when saving Post or StoryItem # log 'json ' message when saving Post or StoryItem
@@ -256,7 +267,8 @@ class Instaloader:
return {'id': comment.id, return {'id': comment.id,
'created_at': int(comment.created_at_utc.replace(tzinfo=timezone.utc).timestamp()), 'created_at': int(comment.created_at_utc.replace(tzinfo=timezone.utc).timestamp()),
'text': comment.text, 'text': comment.text,
'owner': comment.owner._asdict()} 'owner': comment.owner._asdict(),
'likes_count': comment.likes_count}
def _postcomment_asdict(comment): def _postcomment_asdict(comment):
return {**_postcommentanswer_asdict(comment), return {**_postcommentanswer_asdict(comment),
@@ -273,11 +285,13 @@ class Instaloader:
for x, y in zip(comments_list[:-1], comments_list[1:]): for x, y in zip(comments_list[:-1], comments_list[1:]):
if x['id'] != y['id']: if x['id'] != y['id']:
unique_comments_list.append(y) unique_comments_list.append(y)
elif combine_answers: else:
combined_answers = unique_comments_list[-1].get('answers') or list() unique_comments_list[-1]['likes_count'] = y.get('likes_count')
if 'answers' in y: if combine_answers:
combined_answers.extend(y['answers']) combined_answers = unique_comments_list[-1].get('answers') or list()
unique_comments_list[-1]['answers'] = get_unique_comments(combined_answers) if 'answers' in y:
combined_answers.extend(y['answers'])
unique_comments_list[-1]['answers'] = get_unique_comments(combined_answers)
return unique_comments_list return unique_comments_list
filename += '_comments.json' filename += '_comments.json'
try: try:
@@ -344,46 +358,62 @@ class Instaloader:
self.context.log('geo', end=' ', flush=True) self.context.log('geo', end=' ', flush=True)
@_retry_on_connection_error @_retry_on_connection_error
def download_profilepic(self, profile: Profile, _attempt: int = 1) -> None: def download_title_pic(self, url: str, target: Union[str, Path], name_suffix: str, owner_profile: Profile,
"""Downloads and saves profile pic.""" _attempt: int = 1) -> None:
"""Downloads and saves a picture that does not have an association with a Post or StoryItem, such as a
Profile picture or a Highlight cover picture. Modification time is taken from the HTTP response headers.
.. versionadded:: 4.3"""
def _epoch_to_string(epoch: datetime) -> str: def _epoch_to_string(epoch: datetime) -> str:
return epoch.strftime('%Y-%m-%d_%H-%M-%S_UTC') return epoch.strftime('%Y-%m-%d_%H-%M-%S_UTC')
profile_pic_response = self.context.get_raw(profile.profile_pic_url) http_response = self.context.get_raw(url)
date_object = None # type: Optional[datetime] date_object = None # type: Optional[datetime]
if 'Last-Modified' in profile_pic_response.headers: if 'Last-Modified' in http_response.headers:
date_object = datetime.strptime(profile_pic_response.headers["Last-Modified"], '%a, %d %b %Y %H:%M:%S GMT') date_object = datetime.strptime(http_response.headers["Last-Modified"], '%a, %d %b %Y %H:%M:%S GMT')
profile_pic_bytes = None pic_bytes = None
profile_pic_identifier = _epoch_to_string(date_object) pic_identifier = _epoch_to_string(date_object)
else: else:
profile_pic_bytes = profile_pic_response.content pic_bytes = http_response.content
profile_pic_identifier = md5(profile_pic_bytes).hexdigest()[:16] pic_identifier = md5(pic_bytes).hexdigest()[:16]
profile_pic_extension = 'jpg' pic_extension = 'jpg'
if ((format_string_contains_key(self.dirname_pattern, 'profile') or if ((format_string_contains_key(self.dirname_pattern, 'profile') or
format_string_contains_key(self.dirname_pattern, 'target'))): format_string_contains_key(self.dirname_pattern, 'target'))):
filename = '{0}/{1}_profile_pic.{2}'.format(self.dirname_pattern.format(profile=profile.username.lower(), filename = '{0}/{1}_{2}.{3}'.format(self.dirname_pattern.format(profile=owner_profile.username.lower(),
target=profile.username.lower()), target=target),
profile_pic_identifier, profile_pic_extension) pic_identifier, name_suffix, pic_extension)
else: else:
filename = '{0}/{1}_{2}_profile_pic.{3}'.format(self.dirname_pattern.format(), profile.username.lower(), filename = '{0}/{1}_{2}_{3}.{4}'.format(self.dirname_pattern.format(), target,
profile_pic_identifier, profile_pic_extension) pic_identifier, name_suffix, pic_extension)
content_length = profile_pic_response.headers.get('Content-Length', None) content_length = http_response.headers.get('Content-Length', None)
if os.path.isfile(filename) and (not self.context.is_logged_in or if os.path.isfile(filename) and (not self.context.is_logged_in or
(content_length is not None and (content_length is not None and
os.path.getsize(filename) >= int(content_length))): os.path.getsize(filename) >= int(content_length))):
self.context.log(filename + ' already exists') self.context.log(filename + ' already exists')
return None return None
self.context.write_raw(profile_pic_bytes if profile_pic_bytes else profile_pic_response, filename) os.makedirs(os.path.dirname(filename), exist_ok=True)
self.context.write_raw(pic_bytes if pic_bytes else http_response, filename)
if date_object: if date_object:
os.utime(filename, (datetime.now().timestamp(), date_object.timestamp())) os.utime(filename, (datetime.now().timestamp(), date_object.timestamp()))
self.context.log('') # log output of _get_and_write_raw() does not produce \n self.context.log('') # log output of _get_and_write_raw() does not produce \n
def download_profilepic(self, profile: Profile) -> None:
"""Downloads and saves profile pic."""
self.download_title_pic(profile.profile_pic_url, profile.username.lower(), 'profile_pic', profile)
def download_highlight_cover(self, highlight: Highlight, target: Union[str, Path]) -> None:
"""Downloads and saves Highlight cover picture.
.. versionadded:: 4.3"""
self.download_title_pic(highlight.cover_url, target, 'cover', highlight.owner_profile)
@_requires_login @_requires_login
def save_session_to_file(self, filename: Optional[str] = None) -> None: def save_session_to_file(self, filename: Optional[str] = None) -> None:
"""Saves internally stored :class:`requests.Session` object. """Saves internally stored :class:`requests.Session` object.
:param filename: Filename, or None to use default filename. :param filename: Filename, or None to use default filename.
:raises LoginRequiredException: If called without being logged in.
""" """
if filename is None: if filename is None:
assert self.context.username is not None assert self.context.username is not None
@@ -508,6 +538,7 @@ class Instaloader:
To use this, one needs to be logged in To use this, one needs to be logged in
:param userids: List of user IDs to be processed in terms of downloading their stories, or None. :param userids: List of user IDs to be processed in terms of downloading their stories, or None.
:raises LoginRequiredException: If called without being logged in.
""" """
if not userids: if not userids:
@@ -544,6 +575,7 @@ class Instaloader:
:param filename_target: Replacement for {target} in dirname_pattern and filename_pattern :param filename_target: Replacement for {target} in dirname_pattern and filename_pattern
or None if profile name should be used instead or None if profile name should be used instead
:param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded :param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded
:raises LoginRequiredException: If called without being logged in.
""" """
if not userids: if not userids:
@@ -603,6 +635,7 @@ class Instaloader:
.. versionadded:: 4.1 .. versionadded:: 4.1
:param user: ID or Profile of the user whose highlights should get fetched. :param user: ID or Profile of the user whose highlights should get fetched.
:raises LoginRequiredException: If called without being logged in.
""" """
userid = user if isinstance(user, int) else user.userid userid = user if isinstance(user, int) else user.userid
@@ -627,15 +660,23 @@ class Instaloader:
.. versionadded:: 4.1 .. versionadded:: 4.1
.. versionchanged:: 4.3
Also downloads and saves the Highlight's cover pictures.
:param user: ID or Profile of the user whose highlights should get downloaded. :param user: ID or Profile of the user whose highlights should get downloaded.
:param fast_update: If true, abort when first already-downloaded picture is encountered :param fast_update: If true, abort when first already-downloaded picture is encountered
:param filename_target: Replacement for {target} in dirname_pattern and filename_pattern :param filename_target: Replacement for {target} in dirname_pattern and filename_pattern
or None if profile name and the highlights' titles should be used instead or None if profile name and the highlights' titles should be used instead
:param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded :param storyitem_filter: function(storyitem), which returns True if given StoryItem should be downloaded
:raises LoginRequiredException: If called without being logged in.
""" """
for user_highlight in self.get_highlights(user): for user_highlight in self.get_highlights(user):
name = user_highlight.owner_username name = user_highlight.owner_username
highlight_target = (filename_target
if filename_target
else Path(name) / Path(user_highlight.title)) # type: Union[str, Path]
self.context.log("Retrieving highlights \"{}\" from profile {}".format(user_highlight.title, name)) self.context.log("Retrieving highlights \"{}\" from profile {}".format(user_highlight.title, name))
self.download_highlight_cover(user_highlight, highlight_target)
totalcount = user_highlight.itemcount totalcount = user_highlight.itemcount
count = 1 count = 1
for item in user_highlight.get_items(): for item in user_highlight.get_items():
@@ -646,9 +687,7 @@ class Instaloader:
count += 1 count += 1
with self.context.error_catcher('Download highlights \"{}\" from user {}'.format(user_highlight.title, with self.context.error_catcher('Download highlights \"{}\" from user {}'.format(user_highlight.title,
name)): name)):
downloaded = self.download_storyitem(item, filename_target downloaded = self.download_storyitem(item, highlight_target)
if filename_target
else Path(name) / Path(user_highlight.title))
if fast_update and not downloaded: if fast_update and not downloaded:
break break
@@ -657,6 +696,7 @@ class Instaloader:
"""Get Posts of the user's feed. """Get Posts of the user's feed.
:return: Iterator over Posts of the user's feed. :return: Iterator over Posts of the user's feed.
:raises LoginRequiredException: If called without being logged in.
""" """
data = self.context.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})["data"] data = self.context.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})["data"]
@@ -690,6 +730,7 @@ class Instaloader:
:param max_count: Maximum count of pictures to download :param max_count: Maximum count of pictures to download
:param fast_update: If true, abort when first already-downloaded picture is encountered :param fast_update: If true, abort when first already-downloaded picture is encountered
:param post_filter: function(post), which returns True if given picture should be downloaded :param post_filter: function(post), which returns True if given picture should be downloaded
:raises LoginRequiredException: If called without being logged in.
""" """
self.context.log("Retrieving pictures from your feed...") self.context.log("Retrieving pictures from your feed...")
count = 1 count = 1
@@ -715,6 +756,7 @@ class Instaloader:
:param max_count: Maximum count of pictures to download :param max_count: Maximum count of pictures to download
:param fast_update: If true, abort when first already-downloaded picture is encountered :param fast_update: If true, abort when first already-downloaded picture is encountered
:param post_filter: function(post), which returns True if given picture should be downloaded :param post_filter: function(post), which returns True if given picture should be downloaded
:raises LoginRequiredException: If called without being logged in.
""" """
self.context.log("Retrieving saved posts...") self.context.log("Retrieving saved posts...")
count = 1 count = 1
@@ -732,12 +774,17 @@ class Instaloader:
if fast_update and not downloaded: if fast_update and not downloaded:
break break
@_requires_login
def get_location_posts(self, location: str) -> Iterator[Post]: def get_location_posts(self, location: str) -> Iterator[Post]:
"""Get Posts which are listed by Instagram for a given Location. """Get Posts which are listed by Instagram for a given Location.
:return: Iterator over Posts of a location's posts :return: Iterator over Posts of a location's posts
:raises LoginRequiredException: If called without being logged in.
.. versionadded:: 4.2 .. versionadded:: 4.2
.. versionchanged:: 4.2.9
Require being logged in (as required by Instagram)
""" """
has_next_page = True has_next_page = True
end_cursor = None end_cursor = None
@@ -752,6 +799,7 @@ class Instaloader:
has_next_page = location_data['page_info']['has_next_page'] has_next_page = location_data['page_info']['has_next_page']
end_cursor = location_data['page_info']['end_cursor'] end_cursor = location_data['page_info']['end_cursor']
@_requires_login
def download_location(self, location: str, def download_location(self, location: str,
max_count: Optional[int] = None, max_count: Optional[int] = None,
post_filter: Optional[Callable[[Post], bool]] = None, post_filter: Optional[Callable[[Post], bool]] = None,
@@ -767,8 +815,12 @@ class Instaloader:
:param max_count: Maximum count of pictures to download :param max_count: Maximum count of pictures to download
:param post_filter: function(post), which returns True if given picture should be downloaded :param post_filter: function(post), which returns True if given picture should be downloaded
:param fast_update: If true, abort when first already-downloaded picture is encountered :param fast_update: If true, abort when first already-downloaded picture is encountered
:raises LoginRequiredException: If called without being logged in.
.. versionadded:: 4.2 .. versionadded:: 4.2
.. versionchanged:: 4.2.9
Require being logged in (as required by Instagram)
""" """
self.context.log("Retrieving pictures for location {}...".format(location)) self.context.log("Retrieving pictures for location {}...".format(location))
count = 1 count = 1
@@ -790,6 +842,7 @@ class Instaloader:
"""Get Posts which are worthy of exploring suggested by Instagram. """Get Posts which are worthy of exploring suggested by Instagram.
:return: Iterator over Posts of the user's suggested posts. :return: Iterator over Posts of the user's suggested posts.
:raises LoginRequiredException: If called without being logged in.
""" """
data = self.context.get_json('explore/', {}) data = self.context.get_json('explore/', {})
yield from (Post(self.context, node) yield from (Post(self.context, node)
@@ -858,11 +911,29 @@ class Instaloader:
count += 1 count += 1
if post_filter is not None and not post_filter(post): if post_filter is not None and not post_filter(post):
self.context.log('<{} skipped>'.format(post)) self.context.log('<{} skipped>'.format(post))
continue
with self.context.error_catcher('Download tagged {}'.format(profile.username)): with self.context.error_catcher('Download tagged {}'.format(profile.username)):
downloaded = self.download_post(post, target if target else Path(profile.username) / Path(':tagged')) downloaded = self.download_post(post, target if target else Path(profile.username) / Path(':tagged'))
if fast_update and not downloaded: if fast_update and not downloaded:
break break
def download_igtv(self, profile: Profile, fast_update: bool = False,
post_filter: Optional[Callable[[Post], bool]] = None) -> None:
"""Download IGTV videos of a profile.
.. versionadded:: 4.3"""
self.context.log("Retrieving IGTV videos for profile {}.".format(profile.username))
for number, post in enumerate(profile.get_igtv_posts()):
self.context.log("[{0:{w}d}/{1:{w}d}] ".format(number, profile.igtvcount, w=len(str(profile.igtvcount))),
end="", flush=True)
if post_filter is not None and not post_filter(post):
self.context.log('<{} skipped>'.format(post))
continue
with self.context.error_catcher('Download IGTV {}'.format(post.shortcode)):
downloaded = self.download_post(post, target=profile.username)
if fast_update and not downloaded:
break
def _get_id_filename(self, profile_name: str) -> str: def _get_id_filename(self, profile_name: str) -> str:
if ((format_string_contains_key(self.dirname_pattern, 'profile') or if ((format_string_contains_key(self.dirname_pattern, 'profile') or
format_string_contains_key(self.dirname_pattern, 'target'))): format_string_contains_key(self.dirname_pattern, 'target'))):
@@ -892,16 +963,18 @@ class Instaloader:
:return: Instance of current profile :return: Instance of current profile
""" """
profile = None profile = None
with suppress(ProfileNotExistsException): profile_name_not_exists_err = None
try:
profile = Profile.from_username(self.context, profile_name) profile = Profile.from_username(self.context, profile_name)
profile_exists = profile is not None except ProfileNotExistsException as err:
profile_name_not_exists_err = err
id_filename = self._get_id_filename(profile_name) id_filename = self._get_id_filename(profile_name)
try: try:
with open(id_filename, 'rb') as id_file: with open(id_filename, 'rb') as id_file:
profile_id = int(id_file.read()) profile_id = int(id_file.read())
if (not profile_exists) or \ if (profile is None) or \
(profile_id != profile.userid): (profile_id != profile.userid):
if profile_exists: if profile is not None:
self.context.log("Profile {0} does not match the stored unique ID {1}.".format(profile_name, self.context.log("Profile {0} does not match the stored unique ID {1}.".format(profile_name,
profile_id)) profile_id))
else: else:
@@ -920,17 +993,23 @@ class Instaloader:
os.rename('{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower()), os.rename('{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower()),
'{0}/{1}_id'.format(self.dirname_pattern.format(), newname.lower())) '{0}/{1}_id'.format(self.dirname_pattern.format(), newname.lower()))
return profile_from_id return profile_from_id
# profile exists and profile id matches saved id
return profile return profile
except (FileNotFoundError, ValueError): except (FileNotFoundError, ValueError):
pass pass
if profile_exists: if profile is not None:
self.save_profile_id(profile) self.save_profile_id(profile)
return profile return profile
if profile_name_not_exists_err:
raise profile_name_not_exists_err
raise ProfileNotExistsException("Profile {0} does not exist.".format(profile_name)) raise ProfileNotExistsException("Profile {0} does not exist.".format(profile_name))
def download_profiles(self, profiles: Set[Profile], def download_profiles(self, profiles: Set[Profile],
profile_pic: bool = True, posts: bool = True, profile_pic: bool = True, posts: bool = True,
tagged: bool = False, highlights: bool = False, stories: bool = False, tagged: bool = False,
igtv: bool = False,
highlights: bool = False,
stories: bool = False,
fast_update: bool = False, fast_update: bool = False,
post_filter: Optional[Callable[[Post], bool]] = None, post_filter: Optional[Callable[[Post], bool]] = None,
storyitem_filter: Optional[Callable[[Post], bool]] = None, storyitem_filter: Optional[Callable[[Post], bool]] = None,
@@ -941,6 +1020,7 @@ class Instaloader:
:param profile_pic: not :option:`--no-profile-pic`. :param profile_pic: not :option:`--no-profile-pic`.
:param posts: not :option:`--no-posts`. :param posts: not :option:`--no-posts`.
:param tagged: :option:`--tagged`. :param tagged: :option:`--tagged`.
:param igtv: :option:`--igtv`.
:param highlights: :option:`--highlights`. :param highlights: :option:`--highlights`.
:param stories: :option:`--stories`. :param stories: :option:`--stories`.
:param fast_update: :option:`--fast-update`. :param fast_update: :option:`--fast-update`.
@@ -950,7 +1030,11 @@ class Instaloader:
Whether :exc:`LoginRequiredException` and :exc:`PrivateProfileNotFollowedException` should be raised or Whether :exc:`LoginRequiredException` and :exc:`PrivateProfileNotFollowedException` should be raised or
catched and printed with :meth:`InstaloaderContext.error_catcher`. catched and printed with :meth:`InstaloaderContext.error_catcher`.
.. versionadded:: 4.1""" .. versionadded:: 4.1
.. versionchanged:: 4.3
Add `igtv` parameter.
"""
@contextmanager @contextmanager
def _error_raiser(_str): def _error_raiser(_str):
@@ -976,7 +1060,7 @@ class Instaloader:
self.save_metadata_json(json_filename, profile) self.save_metadata_json(json_filename, profile)
# Catch some errors # Catch some errors
if profile.is_private and (tagged or highlights or posts): if profile.is_private and (tagged or igtv or highlights or posts):
if not self.context.is_logged_in: if not self.context.is_logged_in:
raise LoginRequiredException("--login=USERNAME required.") raise LoginRequiredException("--login=USERNAME required.")
if not profile.followed_by_viewer and self.context.username != profile.username: if not profile.followed_by_viewer and self.context.username != profile.username:
@@ -987,6 +1071,11 @@ class Instaloader:
with self.context.error_catcher('Download tagged of {}'.format(profile_name)): with self.context.error_catcher('Download tagged of {}'.format(profile_name)):
self.download_tagged(profile, fast_update=fast_update, post_filter=post_filter) self.download_tagged(profile, fast_update=fast_update, post_filter=post_filter)
# Download IGTV, if requested
if igtv:
with self.context.error_catcher('Download IGTV of {}'.format(profile_name)):
self.download_igtv(profile, fast_update=fast_update, post_filter=post_filter)
# Download highlights, if requested # Download highlights, if requested
if highlights: if highlights:
with self.context.error_catcher('Download highlights of {}'.format(profile_name)): with self.context.error_catcher('Download highlights of {}'.format(profile_name)):
+63 -12
View File
@@ -10,6 +10,7 @@ import time
import urllib.parse import urllib.parse
from contextlib import contextmanager from contextlib import contextmanager
from datetime import datetime, timedelta from datetime import datetime, timedelta
from functools import partial
from typing import Any, Callable, Dict, Iterator, List, Optional, Union from typing import Any, Callable, Dict, Iterator, List, Optional, Union
import requests import requests
@@ -18,11 +19,15 @@ import requests.utils
from .exceptions import * from .exceptions import *
def copy_session(session: requests.Session) -> requests.Session: def copy_session(session: requests.Session, request_timeout: Optional[float] = None) -> requests.Session:
"""Duplicates a requests.Session.""" """Duplicates a requests.Session."""
new = requests.Session() new = requests.Session()
new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies)) new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
new.headers = session.headers.copy() # type: ignore new.headers = session.headers.copy() # type: ignore
if request_timeout is not None:
# Override default timeout behavior.
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
new.request = partial(new.request, timeout=request_timeout) # type: ignore
return new return new
@@ -47,9 +52,10 @@ class InstaloaderContext:
""" """
def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None, def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
max_connection_attempts: int = 3): max_connection_attempts: int = 3, request_timeout: Optional[float] = None):
self.user_agent = user_agent if user_agent is not None else default_user_agent() self.user_agent = user_agent if user_agent is not None else default_user_agent()
self.request_timeout = request_timeout
self._session = self.get_anonymous_session() self._session = self.get_anonymous_session()
self.username = None self.username = None
self.sleep = sleep self.sleep = sleep
@@ -155,6 +161,10 @@ class InstaloaderContext:
'ig_vw': '1920', 'csrftoken': '', 'ig_vw': '1920', 'csrftoken': '',
's_network': '', 'ds_user_id': ''}) 's_network': '', 'ds_user_id': ''})
session.headers.update(self._default_http_header(empty_session_only=True)) session.headers.update(self._default_http_header(empty_session_only=True))
if self.request_timeout is not None:
# Override default timeout behavior.
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
return session return session
def save_session_to_file(self, sessionfile): def save_session_to_file(self, sessionfile):
@@ -167,6 +177,10 @@ class InstaloaderContext:
session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile)) session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile))
session.headers.update(self._default_http_header()) session.headers.update(self._default_http_header())
session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']}) session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']})
if self.request_timeout is not None:
# Override default timeout behavior.
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
self._session = session self._session = session
self.username = username self.username = username
@@ -183,6 +197,7 @@ class InstaloaderContext:
:raises ConnectionException: If connection to Instagram failed. :raises ConnectionException: If connection to Instagram failed.
:raises TwoFactorAuthRequiredException: First step of 2FA login done, now call :raises TwoFactorAuthRequiredException: First step of 2FA login done, now call
:meth:`Instaloader.two_factor_login`.""" :meth:`Instaloader.two_factor_login`."""
# pylint:disable=import-outside-toplevel
import http.client import http.client
# pylint:disable=protected-access # pylint:disable=protected-access
http.client._MAXHEADERS = 200 http.client._MAXHEADERS = 200
@@ -191,6 +206,10 @@ class InstaloaderContext:
'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '', 'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '',
's_network': '', 'ds_user_id': ''}) 's_network': '', 'ds_user_id': ''})
session.headers.update(self._default_http_header()) session.headers.update(self._default_http_header())
if self.request_timeout is not None:
# Override default timeout behavior.
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
session.get('https://www.instagram.com/web/__mid/') session.get('https://www.instagram.com/web/__mid/')
csrf_token = session.cookies.get_dict()['csrftoken'] csrf_token = session.cookies.get_dict()['csrftoken']
session.headers.update({'X-CSRFToken': csrf_token}) session.headers.update({'X-CSRFToken': csrf_token})
@@ -203,7 +222,7 @@ class InstaloaderContext:
except json.decoder.JSONDecodeError: except json.decoder.JSONDecodeError:
raise ConnectionException("Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason)) raise ConnectionException("Login error: JSON decode fail, {} - {}.".format(login.status_code, login.reason))
if resp_json.get('two_factor_required'): if resp_json.get('two_factor_required'):
two_factor_session = copy_session(session) two_factor_session = copy_session(session, self.request_timeout)
two_factor_session.headers.update({'X-CSRFToken': csrf_token}) two_factor_session.headers.update({'X-CSRFToken': csrf_token})
two_factor_session.cookies.update({'csrftoken': csrf_token}) two_factor_session.cookies.update({'csrftoken': csrf_token})
self.two_factor_auth_pending = (two_factor_session, self.two_factor_auth_pending = (two_factor_session,
@@ -212,7 +231,7 @@ class InstaloaderContext:
raise TwoFactorAuthRequiredException("Login error: two-factor authentication required.") raise TwoFactorAuthRequiredException("Login error: two-factor authentication required.")
if resp_json.get('checkpoint_url'): if resp_json.get('checkpoint_url'):
raise ConnectionException("Login: Checkpoint required. Point your browser to " raise ConnectionException("Login: Checkpoint required. Point your browser to "
"https://www.instagram.com{}, " "https://www.instagram.com{} - "
"follow the instructions, then retry.".format(resp_json.get('checkpoint_url'))) "follow the instructions, then retry.".format(resp_json.get('checkpoint_url')))
if resp_json['status'] != 'ok': if resp_json['status'] != 'ok':
if 'message' in resp_json: if 'message' in resp_json:
@@ -279,9 +298,14 @@ class InstaloaderContext:
def _graphql_request_count_per_sliding_window(self, query_hash: str) -> int: def _graphql_request_count_per_sliding_window(self, query_hash: str) -> int:
"""Return how many GraphQL requests can be done within the sliding window.""" """Return how many GraphQL requests can be done within the sliding window."""
if self.is_logged_in: if self.is_logged_in:
max_reqs = {'1cb6ec562846122743b61e492c85999f': 20, '33ba35852cb50da46f5b5e889df7d159': 20} max_reqs = {'1cb6ec562846122743b61e492c85999f': 20,
'33ba35852cb50da46f5b5e889df7d159': 20,
'iphone': 100,
'other': 100}
else: else:
max_reqs = {'1cb6ec562846122743b61e492c85999f': 200, '33ba35852cb50da46f5b5e889df7d159': 200} max_reqs = {'1cb6ec562846122743b61e492c85999f': 200,
'33ba35852cb50da46f5b5e889df7d159': 200,
'other': 200}
return max_reqs.get(query_hash) or min(max_reqs.values()) return max_reqs.get(query_hash) or min(max_reqs.values())
def _graphql_query_waittime(self, query_hash: str, current_time: float, untracked_queries: bool = False) -> float: def _graphql_query_waittime(self, query_hash: str, current_time: float, untracked_queries: bool = False) -> float:
@@ -346,15 +370,24 @@ class InstaloaderContext:
:raises ConnectionException: When query repeatedly failed. :raises ConnectionException: When query repeatedly failed.
""" """
is_graphql_query = 'query_hash' in params and 'graphql/query' in path is_graphql_query = 'query_hash' in params and 'graphql/query' in path
is_iphone_query = host == 'i.instagram.com'
is_other_query = not is_graphql_query and host == "www.instagram.com"
sess = session if session else self._session sess = session if session else self._session
try: try:
self.do_sleep() self.do_sleep()
if is_graphql_query: if is_graphql_query:
self._ratecontrol_graphql_query(params['query_hash']) self._ratecontrol_graphql_query(params['query_hash'])
if is_iphone_query:
self._ratecontrol_graphql_query('iphone')
if is_other_query:
self._ratecontrol_graphql_query('other')
resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False) resp = sess.get('https://{0}/{1}'.format(host, path), params=params, allow_redirects=False)
while resp.is_redirect: while resp.is_redirect:
redirect_url = resp.headers['location'] redirect_url = resp.headers['location']
self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url)) self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url))
if redirect_url.startswith('https://www.instagram.com/accounts/login'):
# alternate rate limit exceeded behavior
raise TooManyRequestsException("429 Too Many Requests: redirected to login")
if redirect_url.startswith('https://{}/'.format(host)): if redirect_url.startswith('https://{}/'.format(host)):
resp = sess.get(redirect_url if redirect_url.endswith('/') else redirect_url + '/', resp = sess.get(redirect_url if redirect_url.endswith('/') else redirect_url + '/',
params=params, allow_redirects=False) params=params, allow_redirects=False)
@@ -372,8 +405,19 @@ class InstaloaderContext:
if is_html_query: if is_html_query:
match = re.search(r'window\._sharedData = (.*);</script>', resp.text) match = re.search(r'window\._sharedData = (.*);</script>', resp.text)
if match is None: if match is None:
raise ConnectionException("Could not find \"window._sharedData\" in html response.") raise QueryReturnedNotFoundException("Could not find \"window._sharedData\" in html response.")
return json.loads(match.group(1)) resp_json = json.loads(match.group(1))
entry_data = resp_json.get('entry_data')
post_or_profile_page = list(entry_data.values())[0] if entry_data is not None else None
if post_or_profile_page is None:
raise ConnectionException("\"window._sharedData\" does not contain required keys.")
# If GraphQL data is missing in `window._sharedData`, search for it in `__additionalDataLoaded`.
if 'graphql' not in post_or_profile_page[0]:
match = re.search(r'window\.__additionalDataLoaded\([^{]+{"graphql":({.*})}\);</script>',
resp.text)
if match is not None:
post_or_profile_page[0]['graphql'] = json.loads(match.group(1))
return resp_json
else: else:
resp_json = resp.json() resp_json = resp.json()
if 'status' in resp_json and resp_json['status'] != "ok": if 'status' in resp_json and resp_json['status'] != "ok":
@@ -386,11 +430,18 @@ class InstaloaderContext:
except (ConnectionException, json.decoder.JSONDecodeError, requests.exceptions.RequestException) as err: except (ConnectionException, json.decoder.JSONDecodeError, requests.exceptions.RequestException) as err:
error_string = "JSON Query to {}: {}".format(path, err) error_string = "JSON Query to {}: {}".format(path, err)
if _attempt == self.max_connection_attempts: if _attempt == self.max_connection_attempts:
raise ConnectionException(error_string) from err if isinstance(err, QueryReturnedNotFoundException):
raise QueryReturnedNotFoundException(error_string) from err
else:
raise ConnectionException(error_string) from err
self.error(error_string + " [retrying; skip with ^C]", repeat_at_end=False) self.error(error_string + " [retrying; skip with ^C]", repeat_at_end=False)
try: try:
if isinstance(err, TooManyRequestsException): if is_graphql_query and isinstance(err, TooManyRequestsException):
self._ratecontrol_graphql_query(params['query_hash'], untracked_queries=True) self._ratecontrol_graphql_query(params['query_hash'], untracked_queries=True)
if is_iphone_query and isinstance(err, TooManyRequestsException):
self._ratecontrol_graphql_query('iphone', untracked_queries=True)
if is_other_query and isinstance(err, TooManyRequestsException):
self._ratecontrol_graphql_query('other', untracked_queries=True)
return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1) return self.get_json(path=path, params=params, host=host, session=sess, _attempt=_attempt + 1)
except KeyboardInterrupt: except KeyboardInterrupt:
self.error("[skipped by user]", repeat_at_end=False) self.error("[skipped by user]", repeat_at_end=False)
@@ -407,7 +458,7 @@ class InstaloaderContext:
:param rhx_gis: 'rhx_gis' variable as somewhere returned by Instagram, needed to 'sign' request :param rhx_gis: 'rhx_gis' variable as somewhere returned by Instagram, needed to 'sign' request
:return: The server's response dictionary. :return: The server's response dictionary.
""" """
with copy_session(self._session) as tmpsession: with copy_session(self._session, self.request_timeout) as tmpsession:
tmpsession.headers.update(self._default_http_header(empty_session_only=True)) tmpsession.headers.update(self._default_http_header(empty_session_only=True))
del tmpsession.headers['Connection'] del tmpsession.headers['Connection']
del tmpsession.headers['Content-Length'] del tmpsession.headers['Content-Length']
@@ -475,7 +526,7 @@ class InstaloaderContext:
:raises ConnectionException: When query repeatedly failed. :raises ConnectionException: When query repeatedly failed.
.. versionadded:: 4.2.1""" .. versionadded:: 4.2.1"""
with copy_session(self._session) as tempsession: with copy_session(self._session, self.request_timeout) as tempsession:
tempsession.headers['User-Agent'] = 'Instagram 10.3.2 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; ' \ tempsession.headers['User-Agent'] = 'Instagram 10.3.2 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; ' \
'scale=2.00; 750x1334) AppleWebKit/420+' 'scale=2.00; 750x1334) AppleWebKit/420+'
for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With']: for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With']:
+110 -4
View File
@@ -17,11 +17,12 @@ PostSidecarNode.is_video.__doc__ = "Whether this node is a video."
PostSidecarNode.display_url.__doc__ = "URL of image or video thumbnail." PostSidecarNode.display_url.__doc__ = "URL of image or video thumbnail."
PostSidecarNode.video_url.__doc__ = "URL of video or None." PostSidecarNode.video_url.__doc__ = "URL of video or None."
PostCommentAnswer = namedtuple('PostCommentAnswer', ['id', 'created_at_utc', 'text', 'owner']) PostCommentAnswer = namedtuple('PostCommentAnswer', ['id', 'created_at_utc', 'text', 'owner', 'likes_count'])
PostCommentAnswer.id.__doc__ = "ID number of comment." PostCommentAnswer.id.__doc__ = "ID number of comment."
PostCommentAnswer.created_at_utc.__doc__ = ":class:`~datetime.datetime` when comment was created (UTC)." PostCommentAnswer.created_at_utc.__doc__ = ":class:`~datetime.datetime` when comment was created (UTC)."
PostCommentAnswer.text.__doc__ = "Comment text." PostCommentAnswer.text.__doc__ = "Comment text."
PostCommentAnswer.owner.__doc__ = "Owner :class:`Profile` of the comment." PostCommentAnswer.owner.__doc__ = "Owner :class:`Profile` of the comment."
PostCommentAnswer.likes_count.__doc__ = "Number of likes on comment."
PostComment = namedtuple('PostComment', (*PostCommentAnswer._fields, 'answers')) # type: ignore PostComment = namedtuple('PostComment', (*PostCommentAnswer._fields, 'answers')) # type: ignore
for field in PostCommentAnswer._fields: for field in PostCommentAnswer._fields:
@@ -324,6 +325,11 @@ class Post:
@property @property
def comments(self) -> int: def comments(self) -> int:
"""Comment count including answers""" """Comment count including answers"""
# If the count is already present in `self._node`, do not use `self._field` which could trigger fetching the
# full metadata dict.
comments = self._node.get('edge_media_to_comment')
if comments and 'count' in comments:
return comments['count']
try: try:
return self._field('edge_media_to_parent_comment', 'count') return self._field('edge_media_to_parent_comment', 'count')
except KeyError: except KeyError:
@@ -340,7 +346,8 @@ class Post:
return PostCommentAnswer(id=int(node['id']), return PostCommentAnswer(id=int(node['id']),
created_at_utc=datetime.utcfromtimestamp(node['created_at']), created_at_utc=datetime.utcfromtimestamp(node['created_at']),
text=node['text'], text=node['text'],
owner=Profile(self._context, node['owner'])) owner=Profile(self._context, node['owner']),
likes_count=node['edge_liked_by']['count'])
def _postcommentanswers(node): def _postcommentanswers(node):
if 'edge_threaded_comments' not in node: if 'edge_threaded_comments' not in node:
@@ -408,10 +415,17 @@ class Post:
@property @property
def location(self) -> Optional[PostLocation]: def location(self) -> Optional[PostLocation]:
"""If the Post has a location, returns PostLocation namedtuple with fields 'id', 'lat' and 'lng' and 'name'.""" """
If the Post has a location, returns PostLocation namedtuple with fields 'id', 'lat' and 'lng' and 'name'.
.. versionchanged:: 4.2.9
Require being logged in (as required by Instagram), return None if not logged-in.
"""
loc = self._field("location") loc = self._field("location")
if self._location or not loc: if self._location or not loc:
return self._location return self._location
if not self._context.is_logged_in:
return None
location_id = int(loc['id']) location_id = int(loc['id'])
if any(k not in loc for k in ('name', 'slug', 'has_public_page', 'lat', 'lng')): if any(k not in loc for k in ('name', 'slug', 'has_public_page', 'lat', 'lng')):
loc = self._context.get_json("explore/locations/{0}/".format(location_id), loc = self._context.get_json("explore/locations/{0}/".format(location_id),
@@ -504,10 +518,11 @@ class Profile:
def _asdict(self): def _asdict(self):
json_node = self._node.copy() json_node = self._node.copy()
# remove posts # remove posts to avoid "Circular reference detected" exception
json_node.pop('edge_media_collections', None) json_node.pop('edge_media_collections', None)
json_node.pop('edge_owner_to_timeline_media', None) json_node.pop('edge_owner_to_timeline_media', None)
json_node.pop('edge_saved_media', None) json_node.pop('edge_saved_media', None)
json_node.pop('edge_felix_video_timeline', None)
if self._iphone_struct_: if self._iphone_struct_:
json_node['iphone_struct'] = self._iphone_struct_ json_node['iphone_struct'] = self._iphone_struct_
return json_node return json_node
@@ -520,6 +535,13 @@ class Profile:
self._has_full_metadata = True self._has_full_metadata = True
self._rhx_gis = metadata.get('rhx_gis') self._rhx_gis = metadata.get('rhx_gis')
except (QueryReturnedNotFoundException, KeyError) as err: except (QueryReturnedNotFoundException, KeyError) as err:
top_search_results = TopSearchResults(self._context, self.username)
similar_profiles = [profile.username for profile in top_search_results.get_profiles()]
if similar_profiles:
raise ProfileNotExistsException('Profile {} does not exist.\nThe most similar profile{}: {}.'
.format(self.username,
's are' if len(similar_profiles) > 1 else ' is',
', '.join(similar_profiles[0:5]))) from err
raise ProfileNotExistsException('Profile {} does not exist.'.format(self.username)) from err raise ProfileNotExistsException('Profile {} does not exist.'.format(self.username)) from err
def _metadata(self, *keys) -> Any: def _metadata(self, *keys) -> Any:
@@ -577,6 +599,10 @@ class Profile:
def mediacount(self) -> int: def mediacount(self) -> int:
return self._metadata('edge_owner_to_timeline_media', 'count') return self._metadata('edge_owner_to_timeline_media', 'count')
@property
def igtvcount(self) -> int:
return self._metadata('edge_felix_video_timeline', 'count')
@property @property
def followers(self) -> int: def followers(self) -> int:
return self._metadata('edge_followed_by', 'count') return self._metadata('edge_followed_by', 'count')
@@ -720,6 +746,19 @@ class Profile:
lambda d: d['data']['user']['edge_user_to_photos_of_you'], lambda d: d['data']['user']['edge_user_to_photos_of_you'],
self._rhx_gis)) self._rhx_gis))
def get_igtv_posts(self) -> Iterator[Post]:
"""Retrieve all IGTV posts.
.. versionadded:: 4.3"""
self._obtain_metadata()
yield from (Post(self._context, node, self) for node in
self._context.graphql_node_list('bc78b344a68ed16dd5d7f264681c4c76',
{'id': self.userid},
'https://www.instagram.com/{0}/channel/'.format(self.username),
lambda d: d['data']['user']['edge_felix_video_timeline'],
self._rhx_gis,
self._metadata('edge_felix_video_timeline')))
def get_followers(self) -> Iterator['Profile']: def get_followers(self) -> Iterator['Profile']:
""" """
Retrieve list of followers of given profile. Retrieve list of followers of given profile.
@@ -1046,6 +1085,73 @@ class Highlight(Story):
yield from (StoryItem(self._context, item, self.owner_profile) for item in self._items) yield from (StoryItem(self._context, item, self.owner_profile) for item in self._items)
class TopSearchResults:
"""
An invocation of this class triggers a search on Instagram for the provided search string.
Provides methods to access the search results as profiles (:class:`Profile`), locations (:class:`PostLocation`) and
hashtags.
:param context: :attr:`Instaloader.context` used to send the query for the search.
:param searchstring: String to search for with Instagram's "top search".
"""
def __init__(self, context: InstaloaderContext, searchstring: str):
self._context = context
self._searchstring = searchstring
# The `__a` param is only needed to prevent `get_json()` from searching for 'window._sharedData'.
self._node = context.get_json('web/search/topsearch/',
params={'context': 'blended',
'query': searchstring,
'include_reel': False,
'__a': 1})
def get_profiles(self) -> Iterator[Profile]:
"""
Provides the :class:`Profile` instances from the search result.
"""
for user in self._node.get('users', []):
user_node = user['user']
if 'pk' in user_node:
user_node['id'] = user_node['pk']
yield Profile(self._context, user_node)
def get_prefixed_usernames(self) -> Iterator[str]:
"""
Provides all profile names from the search result that start with the search string.
"""
for user in self._node.get('users', []):
username = user.get('user', {}).get('username', '')
if username.startswith(self._searchstring):
yield username
def get_locations(self) -> Iterator[PostLocation]:
"""
Provides instances of :class:`PostLocation` from the search result.
"""
for location in self._node.get('places', []):
place = location.get('place', {})
slug = place.get('slug')
loc = place.get('location', {})
yield PostLocation(int(loc['pk']), loc['name'], slug, None, loc['lat'], loc['lng'])
def get_hashtag_strings(self) -> Iterator[str]:
"""
Provides the hashtags from the search result as strings.
"""
for hashtag in self._node.get('hashtags', []):
name = hashtag.get('hashtag', {}).get('name')
if name:
yield name
@property
def searchstring(self) -> str:
"""
The string that was searched for on Instagram to produce this :class:`TopSearchResults` instance.
"""
return self._searchstring
JsonExportable = Union[Post, Profile, StoryItem] JsonExportable = Union[Post, Profile, StoryItem]
+1
View File
@@ -60,6 +60,7 @@ setup(
'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3 :: Only', 'Programming Language :: Python :: 3 :: Only',
'Topic :: Internet', 'Topic :: Internet',
'Topic :: Multimedia :: Graphics' 'Topic :: Multimedia :: Graphics'
+31 -24
View File
@@ -5,12 +5,14 @@ import shutil
import tempfile import tempfile
import unittest import unittest
from itertools import islice from itertools import islice
from typing import Dict, List
import instaloader import instaloader
PROFILE_WITH_HIGHLIGHTS = 325732271 PROFILE_WITH_HIGHLIGHTS = 325732271
PUBLIC_PROFILE = "selenagomez" PUBLIC_PROFILE = "selenagomez"
PUBLIC_PROFILE_ID = 460563723 PUBLIC_PROFILE_ID = 460563723
PUBLIC_PROFILE_WITH_IGTV = "natgeo"
HASHTAG = "kitten" HASHTAG = "kitten"
LOCATION = "362629379" LOCATION = "362629379"
OWN_USERNAME = "aandergr" OWN_USERNAME = "aandergr"
@@ -22,7 +24,7 @@ EMPTY_PROFILE = "not_public"
EMPTY_PROFILE_ID = 1928659031 EMPTY_PROFILE_ID = 1928659031
# Preserve query timestamps (rate control) between tests to not get rate limited # Preserve query timestamps (rate control) between tests to not get rate limited
instaloadercontext_query_timestamps = dict() instaloadercontext_query_timestamps = dict() # type: Dict[str, List[float]]
class TestInstaloaderAnonymously(unittest.TestCase): class TestInstaloaderAnonymously(unittest.TestCase):
@@ -35,11 +37,13 @@ class TestInstaloaderAnonymously(unittest.TestCase):
download_comments=True, download_comments=True,
save_metadata=True) save_metadata=True)
self.L.context.raise_all_errors = True self.L.context.raise_all_errors = True
self.L.context.query_timestamps = instaloadercontext_query_timestamps.copy() # pylint:disable=protected-access
self.L.context._graphql_query_timestamps = instaloadercontext_query_timestamps.copy()
def tearDown(self): def tearDown(self):
# pylint:disable=global-statement,protected-access
global instaloadercontext_query_timestamps global instaloadercontext_query_timestamps
instaloadercontext_query_timestamps = self.L.context.query_timestamps.copy() instaloadercontext_query_timestamps = self.L.context._graphql_query_timestamps.copy()
self.L.close() self.L.close()
os.chdir('/') os.chdir('/')
print("Removing {}".format(self.dir)) print("Removing {}".format(self.dir))
@@ -53,16 +57,17 @@ class TestInstaloaderAnonymously(unittest.TestCase):
self.assertTrue(post.date_utc < previous_post.date_utc) self.assertTrue(post.date_utc < previous_post.date_utc)
previous_post = post previous_post = post
@unittest.SkipTest @unittest.skip('')
def test_public_profile_download(self): def test_public_profile_download(self):
self.L.download_profile(PUBLIC_PROFILE, profile_pic=False, fast_update=True) profiles = {self.L.check_profile_id(PUBLIC_PROFILE)}
self.L.download_profile(PUBLIC_PROFILE, profile_pic=False, fast_update=True) self.L.download_profiles(profiles, profile_pic=False, fast_update=True, raise_errors=True)
self.L.download_profiles(profiles, profile_pic=False, fast_update=True, raise_errors=True)
def test_public_profile_paging(self): def test_public_profile_paging(self):
self.post_paging_test(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts()) self.post_paging_test(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts())
def test_profile_pic_download(self): def test_profile_pic_download(self):
self.L.download_profile(PUBLIC_PROFILE, profile_pic_only=True) self.L.download_profiles({self.L.check_profile_id(PUBLIC_PROFILE)}, posts=False, raise_errors=True)
def test_hashtag_download(self): def test_hashtag_download(self):
self.L.download_hashtag(HASHTAG, NORMAL_MAX_COUNT) self.L.download_hashtag(HASHTAG, NORMAL_MAX_COUNT)
@@ -73,15 +78,6 @@ class TestInstaloaderAnonymously(unittest.TestCase):
if count == PAGING_MAX_COUNT: if count == PAGING_MAX_COUNT:
break break
def test_location_download(self):
self.L.download_location(LOCATION, NORMAL_MAX_COUNT)
def test_location_paging(self):
for count, post in enumerate(self.L.get_location_posts(LOCATION)):
print(post)
if count == PAGING_MAX_COUNT:
break
def test_get_id_by_username(self): def test_get_id_by_username(self):
self.assertEqual(PUBLIC_PROFILE_ID, self.assertEqual(PUBLIC_PROFILE_ID,
instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).userid) instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).userid)
@@ -109,6 +105,11 @@ class TestInstaloaderAnonymously(unittest.TestCase):
PAGING_MAX_COUNT): PAGING_MAX_COUNT):
print(post) print(post)
def test_public_profile_igtv(self):
for post in islice(instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE_WITH_IGTV).get_igtv_posts(),
PAGING_MAX_COUNT):
print(post)
class TestInstaloaderLoggedIn(TestInstaloaderAnonymously): class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
@@ -116,13 +117,13 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
super().setUp() super().setUp()
self.L.load_session_from_file(OWN_USERNAME) self.L.load_session_from_file(OWN_USERNAME)
@unittest.SkipTest @unittest.skip('')
def test_stories_download(self): def test_stories_download(self):
self.L.download_stories() self.L.download_stories()
@unittest.SkipTest @unittest.skip('')
def test_private_profile_download(self): def test_private_profile_download(self):
self.L.download_profile(PRIVATE_PROFILE, download_stories=True) self.L.download_profiles({self.L.check_profile_id(PRIVATE_PROFILE)}, stories=True, raise_errors=True)
def test_stories_paging(self): def test_stories_paging(self):
for user_story in self.L.get_stories(): for user_story in self.L.get_stories():
@@ -141,9 +142,6 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
def test_private_profile_paging(self): def test_private_profile_paging(self):
self.post_paging_test(instaloader.Profile.from_username(self.L.context, PRIVATE_PROFILE).get_posts()) self.post_paging_test(instaloader.Profile.from_username(self.L.context, PRIVATE_PROFILE).get_posts())
def test_profile_pic_download(self):
self.L.download_profile(PUBLIC_PROFILE, profile_pic_only=True)
def test_feed_download(self): def test_feed_download(self):
self.L.download_feed_posts(NORMAL_MAX_COUNT) self.L.download_feed_posts(NORMAL_MAX_COUNT)
@@ -167,8 +165,8 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
def test_followees_and_stories(self): def test_followees_and_stories(self):
profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME) profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
for f in islice(profile.get_followees(), PAGING_MAX_COUNT): followees = set(islice(profile.get_followees(), PAGING_MAX_COUNT))
self.L.download_profile(f.username, profile_pic=False, download_stories_only=True) self.L.download_profiles(followees, profile_pic=False, stories=True, posts=False, raise_errors=True)
def test_get_followees(self): def test_get_followees(self):
profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME) profile = instaloader.Profile.from_username(self.L.context, OWN_USERNAME)
@@ -192,6 +190,15 @@ class TestInstaloaderLoggedIn(TestInstaloaderAnonymously):
if count == PAGING_MAX_COUNT: if count == PAGING_MAX_COUNT:
break break
def test_location_download(self):
self.L.download_location(LOCATION, NORMAL_MAX_COUNT)
def test_location_paging(self):
for count, post in enumerate(self.L.get_location_posts(LOCATION)):
print(post)
if count == PAGING_MAX_COUNT:
break
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()