Compare commits

..
38 Commits
Author SHA1 Message Date
Alexander Graf e232c82d5e Release of Version 4.4.5 2020-07-18 15:24:31 +02:00
Alexander Graf 601dd0b106 First Release Candidate for Version 4.4.5 2020-07-15 10:26:22 +02:00
Alexander Graf 4c20b573e2 Fix KeyError on Posts with very few comments 2020-07-15 10:17:20 +02:00
Cadence EmberandAlexander Graf db63e5c7e5 Bypass IP-based redirects to /accounts/login (#727)
* Bypass IP-based redirects to /accounts/login

There are two main changes made:

For users, we request /{username}/feed/ instead of /{username}/. For some
reason, this completely bypasses the login redirect. This page doesn't
work in browser while blocked, but fortunately all the data we need is
just present in the HTML page.

For posts, we change from using the /p/ page to using the graphql
endpoint for the same data, which is still subject to graphql rate
limits, but is not subject to login redirects. The data is identical
between the two pages, apart from the object keys being sorted
differently and rhx_gis being missing on graphql.

Yes, this now unblocks access from VPNs, Tor, cloud servers, etc.

* Apply requested patch to fix comments

* Remove rhx_gis from Post and Profile

Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
2020-07-15 09:46:39 +02:00
Alexander Graf e21d34124d Add codesnippet for almost chronological order
Such as for downloading hashtag feeds, as discussed in #666 and contributed by
@e2tovar.

Also change comment color to grey in codesnippets in documentation.
2020-06-21 19:38:16 +02:00
michalens 66b3ec6928 Add explanation on downloading single post (#694) 2020-06-19 19:39:08 +02:00
Alexander Graf f7d82495bd Consistently use native '\' path separator on Windows
Use os.path.join() for joining paths rather than +'/'+.

os.path.join() uses '/' on Unix-ish and '\' on Windows. On Windows, this makes
the following improvements:

- Visual consistency with / and \ not being mixed-up in one path (os.path.join()
  has already been used in many places).

- Allow using the \\?\ bypass method, e.g. with
    --dirname-pattern=\\?\D:\{target}
  The \\?\ bypass method fails when '/' is used as path separator. With the \\?\
  bypass method, we can create directories whose name would otherwise be
  considered invalid, such as directories starting with "aux.".
2020-06-19 19:33:46 +02:00
Alexander Graf b0a430051e Restrict GH Action lint workflow on Python 3.8
Now that Travis-CI status reporting works again, there is no need to run the
tests twice.
2020-06-18 08:33:25 +02:00
Alexander Graf 8b56b90e72 Less intrusive PR template 2020-06-18 08:32:46 +02:00
Alexander Graf 0c93e63615 Update troubleshooting.rst, Minor fixes to contributing.rst 2020-06-16 21:44:22 +02:00
Alexander Graf 65025e61d9 Update issue/PR templates, contributing.rst, CI
- Keep documentation in sync with master branch

- Run PyLint and MyPy on Github Actions

- Update Issue templates, enforce usage of issue templates, add links to
  resources in Instaloader Documentation

- Update contributing.rst
2020-06-16 20:26:58 +02:00
Alexander Graf 11baf493db Release of Version 4.4.4 2020-06-11 20:10:52 +02:00
Alexander Graf 2345d90e0e Fix docs building on Windows 2020-06-09 13:03:48 +02:00
Alexander Graf 71b09aed0f Warn if --login parameter is not a username
Closes #624.
2020-06-09 11:44:06 +02:00
Alexander Graf fd6d8542c9 Add a unit test for an empty private profile 2020-06-09 11:44:06 +02:00
Alexander Graf df6fbbee76 Third Release Candidate for Version 4.4.4 2020-06-08 18:51:46 +02:00
Alexander Graf 1661bf7722 Remove unused import 2020-06-08 18:44:43 +02:00
Alexander Graf 8f9723cf62 Fix "Use --login ..." warning at startup 2020-06-08 18:40:13 +02:00
Alexander Graf e1567a0342 Second Release Candidate for Version 4.4.4 2020-06-08 18:22:06 +02:00
Alexander Graf 610b6242a0 Get higher image quality if logged-in
Addresses a suggestion in #630.
2020-06-08 17:53:39 +02:00
Alexander Graf d90f90988a Fix ValueError when executing from other drive on Windows
Fixes #671.
2020-06-08 17:01:42 +02:00
Alexander Graf d3c747a306 First Release Candidate for Version 4.4.4 2020-06-06 10:52:42 +02:00
Alexander Graf 225353dc21 Download highest-resolution images if logged-in (#662)
If logged-in, Post.url and Post.get_sidecar_nodes() now use the iPhone API
endpoint to obtain the highest-resolution image version. Resolves #630.

A notice "Warning: Use --login to download higher-quality versions of pictures"
is issued if the user tries to download posts without being logged-in.

Further, the message "Errors occurred:", which is displayed when the
InstaloaderContext is closed, has been changed to "Errors or warnings
occurred:".
2020-06-06 10:51:03 +02:00
Alexander Graf ed499cb49c Let --sessionfile default to a more persistent path (#659)
The presence of a sessionfile avoids the usage of the failure-prone login
mechanism.  This commit addresses a suggestion in #615 to store the sessionfile
in a persistent path rather than a path within a temporary directory if no
--sessionfile parameter is given.

The default path is now:

  $XDG_CONFIG_HOME/instaloader/session-USERNAME or
    ~/.config/instaloader/session-USERNAME on Unix,

  %LOCALAPPDATA%\Instaloader\session-USERNAME on Windows.

If no file exists in the new path, Instaloader tries loading from the path
where the sessionfile was stored before this commit, hence it automatically
migrates to the new sessionfile path.
2020-06-06 10:49:41 +02:00
Alexander Graf 1c13d5a30b Windows EXE: Start Powershell instead of cmd
... if started by file explorer.
2020-06-06 10:37:20 +02:00
Alexander Graf 2fd1422619 Windows EXE: Use psutil for startup terminal detection 2020-06-05 20:12:23 +00:00
Alexander Graf c818165db4 Update 615_import_firefox_session.py
This fixes finding the cookies.sqlite file on MacOS as per https://github.com/instaloader/instaloader/issues/92#issuecomment-638361344.
2020-06-03 20:07:52 +02:00
Alexander Graf 26a78463f1 doc: Discourage using --commit-mode
--commit-mode has many bugs, especially #257, #483.  For now we discourage using
it until it is reimplemented.
2020-06-03 18:03:27 +02:00
Alexander Graf 689dd3613b doc: Minor style-related changes to code examples 2020-06-03 18:03:01 +02:00
Alexander Graf 27b7c98be2 Update doc section Troubleshooting -> Login Error
New Firefox cookie import script, based on a comment in #615:
https://github.com/instaloader/instaloader/issues/615#issuecomment-630127064
2020-06-03 17:14:08 +02:00
Alexander Graf 72d86533f2 Downgrade 'old' stale action to more stable v1 2020-05-26 16:03:52 +02:00
Alexander Graf 7bb08a220a Update issue stale action 2020-05-24 11:25:37 +02:00
Alexander Graf 03aabc52bb Release of Version 4.4.3 2020-05-22 18:32:05 +02:00
fireattack 9d4df0d5fb Only substitute characters from auto generated fields (#631)
Only substitute characters from auto generated fields in _PostPathFormatter

This fixes issues of path problem on Windows such as #304, #529, etc.

Also made the following changes:

1. Changes substitution of colon to full-width colon (U+FF1A) instead of MODIFIER LETTER COLON (U+A789) to be consistent with others (we use full-width backslash, pipe, etc. already).
2. Also replaces `\r` on Windows with space, just like `\n`.
2020-05-22 18:30:55 +02:00
Alexander Graf bf2f4f16ed Release of Version 4.4.2 2020-05-19 09:27:03 +02:00
InfinitayandAlexander Graf 1a3712d112 Workaround for Instagram's encrypted login. (#623)
* Workaround for Instagram's encrypted login.

Credit: https://github.com/pgrimaud/
See: https://github.com/pgrimaud/instagram-user-feed/commit/96ad4cf54d1ad331b337f325c73e664999a6d066

Closes #615.

Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
2020-05-19 09:25:08 +02:00
Alexander Graf 65462b01ec Release of Version 4.4.1 2020-05-14 16:22:42 +02:00
Alexander Graf 9857c5fa24 Regex target matching: Fix hashtag regex
Unicode characters are allowed in hashtags, so instead of just
[A-Za-z0-9], we now match all \w.  Fixes #614.
2020-05-14 16:17:07 +02:00
30 changed files with 421 additions and 206 deletions
+1 -1
View File
@@ -24,4 +24,4 @@ If applicable, add error messages and tracebacks to help explain your problem.
**Additional context**
Add any other context about the problem here.
(please also see https://instaloader.github.io/contributing.html for how to report a bug)
<!-- please also see https://instaloader.github.io/contributing.html for how to report a bug -->
+11
View File
@@ -0,0 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: Contributing to Instaloader
url: https://instaloader.github.io/contributing.html
about: Guidelines for how to report an issue.
- name: Instaloader Troubleshooting
url: https://instaloader.github.io/troubleshooting.html
about: Many common problems and their workarounds are described here.
- name: Instaloader Documentation
url: https://instaloader.github.io/
about: Many questions might be already answered here.
+3 -3
View File
@@ -15,10 +15,10 @@ A clear and concise description of what you want to happen.
A clear and concise description of any alternative solutions or features you've considered.
**If the feature request is accepted, would you be willing to submit a PR?**
Yes / No _(Help can be provided if you need assistance submitting a PR)_
Yes / No
<!-- Help can be provided if you need assistance submitting a PR -->
**Additional context**
Add any other context about the feature request here.
(please also see https://instaloader.github.io/contributing.html for how to suggest a feature)
<!-- please also see https://instaloader.github.io/contributing.html for how to suggest a feature -->
+4
View File
@@ -6,3 +6,7 @@ labels: question
---
Your question here...
<!-- Please see our documentation: https://instaloader.github.io/ -->
<!-- Basic questions regarding Instaloader's usage might be more appropriate to ask on Stack Overflow -->
+17
View File
@@ -0,0 +1,17 @@
<!--
Please describe:
- A motivation for this change, e.g.
- Fixes # .
- More general: What problem does the pull request solve?
- The changes proposed in this pull request
- The completeness of this change
- Is it just a proof of concept?
- Is the documentation updated (if appropriate)?
- Do you consider it ready to be merged or is it a draft?
- Can we help you at some point?
-->
+30
View File
@@ -0,0 +1,30 @@
name: PyLint and MyPy
on: [push, pull_request]
jobs:
lint:
name: PyLint and MyPy
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8]
steps:
- name: Checkout Instaloader Repository
uses: actions/checkout@v2
with:
fetch-depth: 0 # needed for building docs
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install Dependencies
run: |
python -m pip install pipenv==2020.6.2
pipenv sync --dev
- name: PyLint
run: pipenv run pylint instaloader
- name: MyPy
run: pipenv run mypy -m instaloader
- name: Build Documentation
run: pipenv run make -C docs html SPHINXOPTS="-W -n"
+12 -2
View File
@@ -2,14 +2,25 @@ name: Mark stale issues and pull requests
on:
schedule:
- cron: "15 16 * * *"
- cron: "10 14 * * *"
jobs:
stale:
name: Mark stale issues and pull requests
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
only-labels: 'question'
stale-issue-message: 'There has been no activity on this question for an extended period of time. This issue will be closed after further 14 days of inactivity.'
stale-issue-label: 'stale'
exempt-issue-labels: 'leave open'
days-before-stale: 15
days-before-close: -1
remove-stale-when-updated: false
- uses: actions/stale@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
@@ -21,4 +32,3 @@ jobs:
exempt-pr-label: 'leave open'
days-before-stale: 90
days-before-close: 14
operations-per-run: 7
+1
View File
@@ -4,6 +4,7 @@ on: [push, pull_request]
jobs:
build:
name: Windows EXE build
runs-on: windows-latest
steps:
- name: Checkout Instaloader repository
+1 -1
View File
@@ -27,7 +27,7 @@ deploy:
repo: instaloader/instaloader.github.io
target-branch: master
on:
tags: true
branch: master
python: 3.6
- provider: script
script: deploy/arch/deploy.sh $TRAVIS_TAG
+6 -21
View File
@@ -8,33 +8,18 @@ import sys
# copy the required files into repo root
shutil.copy('docs/favicon.ico', '.')
shutil.copy('deploy/windows/instaloader.spec', '.')
shutil.unpack_archive('deploy/windows/ps', '.', 'xztar')
code = """
import contextlib
import psutil
import subprocess
def __main():
grandparentpid = psutil.Process(os.getppid()).ppid()
grandparentpidsearchstring = ' ' + str(grandparentpid) + ' '
if hasattr(sys, "_MEIPASS"):
ps = os.path.join(sys._MEIPASS, 'tasklist.exe')
else:
ps = 'tasklist'
popen = subprocess.Popen(ps, stdout=subprocess.PIPE, universal_newlines=True)
for examine in iter(popen.stdout.readline, ""):
if grandparentpidsearchstring in examine:
pname = examine
break
popen.stdout.close()
return_code = popen.wait()
if return_code:
raise subprocess.CalledProcessError(return_code, ps)
if pname[0:12] == 'explorer.exe':
subprocess.Popen("cmd /K \\\"{0}\\\"".format(os.path.splitext(os.path.basename(sys.argv[0]))[0]))
else:
main()
with contextlib.suppress(AttributeError, psutil.Error):
if psutil.Process().parent().parent().name() == "explorer.exe":
subprocess.Popen("powershell -NoExit -Command \\\"& {0}\\\"".format(sys.argv[0]))
return
main()
if __name__ == "__main__":
__main()
+1 -1
View File
@@ -5,7 +5,7 @@ block_cipher = None
a = Analysis(['__main__.py'],
binaries=[],
datas=[('tasklist.exe', '.'), ('framedyn.dll', '.')],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
BIN
View File
Binary file not shown.
+8 -5
View File
@@ -18,14 +18,17 @@ code {
padding: 0.7em;
color: #fff; }
.highlight .c1 {
color: #008d06; }
color: #666; }
.highlight .k, .highlight .kn, .highlight .ow {
color: #f48400;
font-weight: bold; }
.highlight .nb {
color: #008d06; }
.highlight .nb, .highlight .ne, .highlight .nf, .highlight .vm {
color: #f48400; }
.highlight .s1, .highlight .s2, .highlight .mi {
.highlight .s1, .highlight .s2, .highlight .sa, .highlight .se, .highlight .si, .highlight .mi {
color: #ff7efd; }
.highlight .c1, .highlight .nf, .highlight .sa, .highlight .se, .highlight .si {
font-style: italic; }
.highlight .k, .highlight .kn, .highlight .ow {
font-weight: bold; }
.doc-content {
padding-top: 0.5em;
+12 -5
View File
@@ -37,21 +37,28 @@ code {
}
.c1 {
color: $color_instaloader_main
color: #666;
}
.k, .kn, .ow {
color: $color_instaloader_accent;
font-weight: bold
color: $color_instaloader_main
}
.nb {
.nb, .ne, .nf, .vm {
color: $color_instaloader_accent
}
.s1, .s2, .mi {
.s1, .s2, .sa, .se, .si, .mi {
color: $color_instaloader_hyper
}
.c1, .nf, .sa, .se, .si {
font-style: italic;
}
.k, .kn, .ow {
font-weight: bold
}
}
.doc-content {
+14 -9
View File
@@ -51,7 +51,7 @@ invoke it with
instaloader --login=your_username profile [profile ...]
When logging in, Instaloader **stores the session cookies** in a file in your
temporary directory, which will be reused later the next time :option:`--login`
home directory, which will be reused later the next time :option:`--login`
is given. So you can download private profiles **non-interactively** when you
already have a valid session cookie file.
@@ -111,8 +111,12 @@ Instaloader supports the following targets:
``profile`` (requires :option:`--login`).
- ``-post``
The single **post** with the given shortcode. Must be preceeded by ``--`` in
the argument list to not be mistaken as an option flag.
Replace **post** with the post's shortcode to download single post. Must be preceeded by ``--`` in
the argument list to not be mistaken as an option flag::
instaloader -- -B_K4CykAOtf
.. versionadded:: 4.1
@@ -291,17 +295,18 @@ Instaloader as Cronjob
Instaloader is suitable for running as a cronjob to periodically update your
personal Instagram archive. The :option:`--quiet` option disables user
interactions and logging of non-error messages. To non-interactively use
Instaloader logged-in, create a session file, e.g. in your home directory::
Instaloader logged-in, create a session file::
instaloader --login=your_username --sessionfile=~/.instaloader-session
instaloader --login=your_username
Then use the same parameters in your cronjob to load the session and download
Then use the same parameter in your cronjob to load the session and download
the given targets::
instaloader --login=your_username --sessionfile=~/.instaloader-session --quiet <target> [...]
instaloader --login=your_username --quiet <target> [...]
Without :option:`--sessionfile` option, Instaloader saves the session file in
a path within your temporary directory.
Instaloader saves the session file to
``~/.config/instaloader/session-YOUR-USERNAME``. See
:option:`--sessionfile` option for how to override this path.
Programming Instaloader
^^^^^^^^^^^^^^^^^^^^^^^
+7 -3
View File
@@ -193,9 +193,8 @@ Instaloader to login.
.. option:: --sessionfile SESSIONFILE, -f SESSIONFILE
Path for loading and storing session key file. Defaults to a path within
your temporary directory, encoding your local username and your Instagram
profile name.
Path for loading and storing session key file. Defaults to
``~/.config/instaloader/session-YOUR-USERNAME``.
.. option:: --password YOUR-PASSWORD, -p YOUR-PASSWORD
@@ -243,6 +242,11 @@ How to Download
.. versionadded:: 4.2
.. warning::
We discourage from using the "commit mode" due to its bugs when being used
with other command line options (e.g. :issue:`257` with
:option:`--fast-update` and :issue:`483` with :option:`--tagged`).
.. option:: --request-timeout N
Seconds to wait before timing out a connection request.
+11 -2
View File
@@ -28,8 +28,9 @@ Download Posts in a Specific Period
-----------------------------------
To only download Instagram pictures (and metadata) that are within a specific
period, you can play around with :func:`~itertools.dropwhile` and
:func:`~itertools.takewhile` from :mod:`itertools` like in this snippet.
period, you can simply use :func:`~itertools.dropwhile` and
:func:`~itertools.takewhile` from :mod:`itertools` on a generator that returns
Posts in **exact chronological order**, such as :meth:`Profile.get_posts`.
.. literalinclude:: codesnippets/121_since_until.py
@@ -37,6 +38,14 @@ See also :class:`Post`, :meth:`Instaloader.download_post`.
Discussed in :issue:`121`.
The code example with :func:`~itertools.dropwhile` and
:func:`~itertools.takewhile` makes the assumption that the post iterator returns
posts in exact chronological order. As discussed in :issue:`666`, the following
approach fits for an **almost chronological order**, where up to *k* older posts
are inserted into an otherwise chronological order, such as an Hashtag feed.
.. literalinclude:: codesnippets/666_historical_hashtag_data.py
Likes of a Profile / Ghost Followers
------------------------------------
+2 -4
View File
@@ -5,13 +5,11 @@ import instaloader
L = instaloader.Instaloader()
posts = instaloader.Hashtag.from_name(L.context, 'urbanphotography').get_posts()
# or
# posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts()
posts = instaloader.Profile.from_username(L.context, "instagram").get_posts()
SINCE = datetime(2015, 5, 1)
UNTIL = datetime(2015, 3, 1)
for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
print(post.date)
L.download_post(post, '#urbanphotography')
L.download_post(post, "instagram")
@@ -0,0 +1,53 @@
from argparse import ArgumentParser
from glob import glob
from os.path import expanduser
from platform import system
from sqlite3 import OperationalError, connect
try:
from instaloader import ConnectionException, Instaloader
except ModuleNotFoundError:
raise SystemExit("Instaloader not found.\n pip install [--user] instaloader")
def get_cookiefile():
default_cookiefile = {
"Windows": "~/AppData/Roaming/Mozilla/Firefox/Profiles/*/cookies.sqlite",
"Darwin": "~/Library/Application Support/Firefox/Profiles/*/cookies.sqlite",
}.get(system(), "~/.mozilla/firefox/*/cookies.sqlite")
cookiefiles = glob(expanduser(default_cookiefile))
if not cookiefiles:
raise SystemExit("No Firefox cookies.sqlite file found. Use -c COOKIEFILE.")
return cookiefiles[0]
def import_session(cookiefile, sessionfile):
print("Using cookies from {}.".format(cookiefile))
conn = connect(cookiefile)
try:
cookie_data = conn.execute(
"SELECT name, value FROM moz_cookies WHERE baseDomain='instagram.com'"
)
except OperationalError:
cookie_data = conn.execute(
"SELECT name, value FROM moz_cookies WHERE host LIKE '%instagram.com'"
)
instaloader = Instaloader(max_connection_attempts=1)
instaloader.context._session.cookies.update(cookie_data)
username = instaloader.test_login()
if not username:
raise SystemExit("Not logged in. Are you logged in successfully in Firefox?")
print("Imported session cookie for {}.".format(username))
instaloader.context.username = username
instaloader.save_session_to_file(sessionfile)
if __name__ == "__main__":
p = ArgumentParser()
p.add_argument("-c", "--cookiefile")
p.add_argument("-f", "--sessionfile")
args = p.parse_args()
try:
import_session(args.cookiefile or get_cookiefile(), args.sessionfile)
except (ConnectionException, OperationalError) as e:
raise SystemExit("Cookie import failed: {}".format(e))
@@ -0,0 +1,30 @@
from datetime import datetime
import instaloader
L = instaloader.Instaloader()
posts = instaloader.Hashtag.from_name(L.context, "urbanphotography").get_posts()
SINCE = datetime(2020, 5, 10) # further from today, inclusive
UNTIL = datetime(2020, 5, 11) # closer to today, not inclusive
k = 0 # initiate k
k_list = [] # uncomment this to tune k
for post in posts:
postdate = post.date
if postdate > UNTIL:
continue
elif postdate <= SINCE:
k += 1
if k == 50:
break
else:
continue
else:
L.download_post(post, "#urbanphotography")
k = 0 # set k to 0
# if you want to tune k, uncomment below to get your k max
#k_list.append(k)
#max(k_list)
@@ -1,23 +0,0 @@
from glob import glob
from os.path import expanduser
from sqlite3 import connect
from instaloader import ConnectionException, Instaloader
# FIREFOXCOOKIEFILE = "/home/alex/.mozilla/firefox/l96w6b90.default/cookies.sqlite"
FIREFOXCOOKIEFILE = glob(expanduser("~/.mozilla/firefox/*.default/cookies.sqlite"))[0]
instaloader = Instaloader(max_connection_attempts=1)
instaloader.context._session.cookies.update(connect(FIREFOXCOOKIEFILE)
.execute("SELECT name, value FROM moz_cookies "
"WHERE baseDomain='instagram.com'"))
try:
username = instaloader.test_login()
if not username:
raise ConnectionException()
except ConnectionException:
raise SystemExit("Cookie import failed. Are you logged in successfully in Firefox?")
instaloader.context.username = username
instaloader.save_session_to_file()
+19 -8
View File
@@ -18,6 +18,7 @@
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import platform
import subprocess
import sys
import typing
@@ -49,7 +50,23 @@ autodoc_member_order = 'bysource'
intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
'requests': ('https://requests.kennethreitz.org/en/master/', None)}
extlinks = {'issue': ('https://github.com/instaloader/instaloader/issues/%s', 'issue #')}
current_release = subprocess.check_output(["git", "describe", "--abbrev=0"]).decode("ascii")[1:-1]
date_format = "%e %b %Y" if platform.system() != "Windows" else "%d %b %Y"
current_release_date = subprocess.check_output(
["git", "log", "-1", "--tags", "--format=%ad", "--date=format:"+date_format]).decode("ascii")[:-1]
html_context = {'current_release': current_release, 'current_release_date': current_release_date}
extlinks = {
'issue': (
'https://github.com/instaloader/instaloader/issues/%s',
'Issue #'
),
'example': (
'https://raw.githubusercontent.com/instaloader/instaloader/v' + current_release + '/docs/codesnippets/%s',
'Example '
),
}
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -69,7 +86,7 @@ master_doc = 'index'
# General information about the project.
project = 'Instaloader'
copyright = '2017, Alexander Graf and Andre Koch-Kramer'
copyright = '2017-2020, Alexander Graf and Andre Koch-Kramer'
author = 'Alexander Graf and Andre Koch-Kramer'
# The version info for the project you're documenting, acts as replacement for
@@ -353,12 +370,6 @@ texinfo_documents = [
# texinfo_no_detailmenu = False
current_release = subprocess.check_output(["git", "describe", "--abbrev=0"]).decode("ascii")[1:-1]
current_release_date = subprocess.check_output(["git", "log", "-1", "--tags", "--format=%ad", "--date=format:%e %b %Y"]).decode("ascii")[:-1]
html_context = {'current_release': current_release, 'current_release_date': current_release_date}
def setup(app):
typing.TYPE_CHECKING = True
app.add_stylesheet("instaloader.css")
+27 -26
View File
@@ -35,8 +35,8 @@ reporting a problem, please keep the following in mind:
- If not obvious, describe **which behavior you expected**
instead of what actually happened.
- If you think an issue has been closed accidentally or inappropriately, feel
free to reopen it.
- If we have closed an issue apparently inadvertently or inappropriately, please
let us know.
Writing Code or Improving the Documentation
-------------------------------------------
@@ -45,28 +45,25 @@ Changes of the Instaloader source can be proposed as a
`Pull Request <https://github.com/instaloader/instaloader/pulls>`__. There are only
few things to consider:
- Sometimes, the most current code is not in the ``master`` branch. Check that
you forked from the most recent branch.
- Base your Pull Request on the ``master`` branch if it fixes a bug,
or the ``upcoming/v4.X`` branch (if it exists at the moment of submitting the PR)
otherwise.
- We use `Pylint <https://www.pylint.org/>`__ for error and syntax checking of
the source. The file ``.travis.yml`` in the project's root directory
shows how it is invoked. Note that sometimes it might be better to disable a
warning rather than adapting the code to Pylint's desires.
the source and `MyPy <https://github.com/python/mypy>`__ for type checking.
Beware that sometimes it might be better to disable a warning rather than
adapting the code to a tool's desires.
- The documentation source is located in the ``docs`` folder. The file
``cli-options.rst`` is merely an RST-formatted copy of ``instaloader --help``
output, of which the source is in ``instaloader/__main__.py``.
- Feel free to contact us, even if you "only" have Proof-of-Concepts or
not-fully integrated changes. They already might be an advance for the
project.
Suggesting Features
-------------------
Proposing Features
------------------
.. goal-start
Instaloader's goal is to mimick the browser's behavior to access the data that
Instaloader's goal is to mimic the browser's behavior to access the data that
is available through the Instagram web interface, pack this data into complete
and easily-(re)usable python objects, and provide a user interface for the most
common downloading and metadata collection tasks, without changing any of the
@@ -74,10 +71,9 @@ online data.
.. goal-end
If you have an idea of how Instaloader should be enhanced, but do not want to
implement the feature yourself, feel free to open a ticket in our
`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__.
Please consider the following:
Prior spending effort on implementing a new feature, it might be appropriate to
clarify how it could fit into the project's scope or discuss implementation
details. If you feel the need to do so, please create a "feature suggestion".
- Instaloader already has plenty of features. **Check the documentation**
beforehand to ensure your desired suggestion is not already implemented.
@@ -86,16 +82,21 @@ Please consider the following:
an issue suggesting the same or a similar feature, share your thoughts in a
comment there, instead of opening a new issue.
- If possible, provide us a **use case of the feature**: How could the user
invoke the new function? Which problem would it solve? If new information is
obtained, how would it be further processed?
- **Motivate the feature**, i.e.
- If not obvious, briefly motivate how your suggested feature **conforms with
Instaloader's project goal**.
- Provide us a **use case of the feature**: How could the user
invoke the new function? Which problem would it solve? If new information is
obtained, how would it be further processed?
- **Be patient**. Naturally, bugs and pull requests have a higher priority than
feature suggestions. Keep in mind that this is a free software project, and
unfortunately we only have limited time to work on it.
- Describe already-working **alternatives of the feature** and how they
compare to your proposed feature.
- Briefly describe how your suggested feature **conforms with Instaloader's
project goal**.
- Explain your **solution ideas**. Describe your ideas on how the feature could
be implemented and the underlying problem could be solved. Also **describe
alternatives** that you have considered.
Donations
---------
+37 -13
View File
@@ -27,12 +27,26 @@ expired. In case the retry continuously fails for some reason, which should not
happen in normal conditions, consider adjusting the
:option:`--max-connection-attempts` option.
There have been observations that services, that in their nature offer
promiscious IP addresses, such as cloud, VPN and public proxy services, might be
subject to significantly stricter limits for anonymous access. However,
logged-in accesses (see :option:`--login`) do not seem to be affected.
Too many queries in the last time
---------------------------------
**"Too many queries in the last time"** is not an error. It is a notice that the
rate limit has almost been reached, according to Instaloader's own rate
accounting mechanism. We regularly adjust this mechanism to match Instagram's
current rate limiting.
Login error
Private but not followed
------------------------
You have to follow a private account to access most of its associated
information.
Login Error
-----------
Instaloader's login *should* work fine, both with and without
@@ -40,21 +54,31 @@ Two-Factor-Authentication. It also supports handling the *checkpoint challenge*,
issued when Instagram suspects authentication activity on your account, by
pointing the user to an URL to be opened in a browser.
Nevertheless, in :issue:`92` users report problems with logging in. To still use
Instaloader's logged-in functionality, you may use the following script to
workaround login problems by importing the session cookies from Firefox and
bypassing Instaloader's login.
Nevertheless, in :issue:`92` and :issue:`615` users reported problems with
logging in. We recommend to always keep the sessionfile which Instaloader
creates when using :option:`--login`. If a sessionfile is present,
:option:`--login` does not make make use of the failure-prone login procedure.
Also, sessionfiles usually do not expire and can be copied between different
computers without any problems.
.. literalinclude:: codesnippets/92_import_firefox_session.py
If you do not have a sessionfile present, you may use the following script
(:example:`615_import_firefox_session.py`) to workaround login problems by
importing the session cookies from Firefox and bypassing Instaloader's login and
so still use Instaloader's logged-in functionality.
To use this,
.. literalinclude:: codesnippets/615_import_firefox_session.py
#. login to Instagram in Firefox,
To use this script,
#. execute the snippet,
#. Download the script: :example:`615_import_firefox_session.py`,
#. then, ``instaloader -l USERNAME`` should work fine.
#. Login to Instagram in Firefox,
If you do not use your default firefox profile, or your operating system has the
paths differently set up, you may have to alter the ``FIREFOXCOOKIEFILE``
variable first.
#. Execute the snippet, e.g. with ``python 615_import_firefox_session.py``,
#. Then, ``instaloader -l USERNAME`` should work fine.
This script also supports specifying a cookiefile path, which may be useful if
you use multiple Firefox profiles or if your operating system has the directory
structure differently set up. Also, you can specify an alternative sessionfile
path.
+1 -1
View File
@@ -1,7 +1,7 @@
"""Download pictures (or videos) along with their captions and other metadata from Instagram."""
__version__ = '4.4'
__version__ = '4.4.5'
try:
+5 -3
View File
@@ -82,6 +82,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
instaloader.context.log('Only download storyitems with property "{}".'.format(storyitem_filter_str))
# Login, if desired
if username is not None:
if not re.match(r"^[A-Za-z0-9._]+$", username):
instaloader.context.error("Warning: Parameter \"{}\" for --login is not a valid username.".format(username))
try:
instaloader.load_session_from_file(username, sessionfile)
except FileNotFoundError as err:
@@ -143,7 +145,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
for followee in profile.get_followees():
instaloader.save_profile_id(followee)
profiles.add(followee)
elif re.match(r"^#[A-Za-z0-9]+$", target):
elif re.match(r"^#\w+$", target):
instaloader.download_hashtag(hashtag=target[1:], max_count=max_count, fast_update=fast_update,
post_filter=post_filter,
profile_pic=download_profile_pic, posts=download_posts)
@@ -196,8 +198,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
if len(profiles) > 1:
instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
' '.join([p.username for p in profiles])))
if profiles and download_profile_pic and not instaloader.context.is_logged_in:
instaloader.context.error("Warning: Use --login to download HD version of profile pictures.")
if profiles and (download_profile_pic or download_posts) and not instaloader.context.is_logged_in:
instaloader.context.error("Warning: Use --login to download higher-quality versions of pictures.")
instaloader.download_profiles(profiles,
download_profile_pic, download_posts, download_tagged, download_igtv,
download_highlights, download_stories,
+45 -26
View File
@@ -27,6 +27,20 @@ from .structures import (Hashtag, Highlight, JsonExportable, Post, PostLocation,
def get_default_session_filename(username: str) -> str:
"""Returns default session filename for given username."""
sessionfilename = "session-{}".format(username)
if platform.system() == "Windows":
# on Windows, use %LOCALAPPDATA%\Instaloader\session-USERNAME
localappdata = os.getenv("LOCALAPPDATA")
if localappdata is not None:
return os.path.join(localappdata, "Instaloader", sessionfilename)
# legacy fallback - store in temp dir if %LOCALAPPDATA% is not set
return os.path.join(tempfile.gettempdir(), ".instaloader-" + getpass.getuser(), sessionfilename)
# on Unix, use ~/.config/instaloader/session-USERNAME
return os.path.join(os.getenv("XDG_CONFIG_HOME", os.path.expanduser("~/.config")), "instaloader", sessionfilename)
def get_legacy_session_filename(username: str) -> str:
"""Returns legacy (until v4.4.3) default session filename for given username."""
dirname = tempfile.gettempdir() + "/" + ".instaloader-" + getpass.getuser()
filename = dirname + "/" + "session-" + username
return filename.lower()
@@ -103,22 +117,20 @@ class _ArbitraryItemFormatter(string.Formatter):
class _PostPathFormatter(_ArbitraryItemFormatter):
def vformat(self, format_string, args, kwargs):
"""Override :meth:`string.Formatter.vformat` for character substitution in paths for Windows, see issue #84."""
ret = super().vformat(format_string, args, kwargs)
if platform.system() == 'Windows':
ret = ret.replace(':', '\ua789').replace('<', '\ufe64').replace('>', '\ufe65').replace('\"', '\uff02')
ret = ret.replace('\\', '\ufe68').replace('|', '\uff5c').replace('?', '\ufe16').replace('*', '\uff0a')
return ret
def get_value(self, key, args, kwargs):
"""Replaces '/' with similar looking Division Slash and on windows newline with space"""
ret = super().get_value(key, args, kwargs)
if not isinstance(ret, str):
return ret
return self.sanitize_path(ret)
@staticmethod
def sanitize_path(ret: str) -> str:
"""Replaces '/' with similar looking Division Slash and some other illegal filename characters on Windows."""
ret = ret.replace('/', '\u2215')
if platform.system() == 'Windows':
ret = ret.replace('\n', ' ')
ret = ret.replace(':', '\uff1a').replace('<', '\ufe64').replace('>', '\ufe65').replace('\"', '\uff02')
ret = ret.replace('\\', '\ufe68').replace('|', '\uff5c').replace('?', '\ufe16').replace('*', '\uff0a')
ret = ret.replace('\n', ' ').replace('\r', ' ')
return ret
@@ -381,12 +393,12 @@ class Instaloader:
if ((format_string_contains_key(self.dirname_pattern, 'profile') or
format_string_contains_key(self.dirname_pattern, 'target'))):
profile_str = owner_profile.username.lower() if owner_profile is not None else target
filename = '{0}/{1}_{2}.{3}'.format(self.dirname_pattern.format(profile=profile_str,
target=target),
pic_identifier, name_suffix, pic_extension)
filename = os.path.join(self.dirname_pattern.format(profile=profile_str,
target=target),
'{0}_{1}.{2}'.format(pic_identifier, name_suffix, pic_extension))
else:
filename = '{0}/{1}_{2}_{3}.{4}'.format(self.dirname_pattern.format(), target,
pic_identifier, name_suffix, pic_extension)
filename = os.path.join(self.dirname_pattern.format(),
'{0}_{1}_{2}.{3}'.format(target, pic_identifier, name_suffix, pic_extension))
content_length = http_response.headers.get('Content-Length', None)
if os.path.isfile(filename) and (not self.context.is_logged_in or
(content_length is not None and
@@ -443,6 +455,8 @@ class Instaloader:
"""
if filename is None:
filename = get_default_session_filename(username)
if not os.path.exists(filename):
filename = get_legacy_session_filename(username)
with open(filename, 'rb') as sessionfile:
self.context.load_session_from_file(username, sessionfile)
self.context.log("Loaded session from %s." % filename)
@@ -487,7 +501,7 @@ class Instaloader:
"""
dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target)
filename = dirname + '/' + self.format_filename(post, target=target)
filename = os.path.join(dirname, self.format_filename(post, target=target))
os.makedirs(os.path.dirname(filename), exist_ok=True)
# Download the image(s) / video thumbnail and videos within sidecars if desired
@@ -616,7 +630,7 @@ class Instaloader:
date_local = item.date_local
dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target)
filename = dirname + '/' + self.format_filename(item, target=target)
filename = os.path.join(dirname, self.format_filename(item, target=target))
os.makedirs(os.path.dirname(filename), exist_ok=True)
downloaded = False
if not item.is_video or self.download_video_thumbnails is True:
@@ -681,7 +695,8 @@ class Instaloader:
name = user_highlight.owner_username
highlight_target = (filename_target
if filename_target
else Path(name) / Path(user_highlight.title)) # type: Union[str, Path]
else (Path(_PostPathFormatter.sanitize_path(name)) /
_PostPathFormatter.sanitize_path(user_highlight.title))) # type: Union[str, Path]
self.context.log("Retrieving highlights \"{}\" from profile {}".format(user_highlight.title, name))
self.download_highlight_cover(user_highlight, highlight_target)
totalcount = user_highlight.itemcount
@@ -942,8 +957,10 @@ class Instaloader:
.. versionadded:: 4.1"""
self.context.log("Retrieving tagged posts for profile {}.".format(profile.username))
self.posts_download_loop(profile.get_tagged_posts(),
target if target else Path(profile.username) / Path(':tagged'), fast_update,
post_filter)
target if target
else (Path(_PostPathFormatter.sanitize_path(profile.username)) /
_PostPathFormatter.sanitize_path(':tagged')),
fast_update, post_filter)
def download_igtv(self, profile: Profile, fast_update: bool = False,
post_filter: Optional[Callable[[Post], bool]] = None) -> None:
@@ -957,10 +974,12 @@ class Instaloader:
def _get_id_filename(self, profile_name: str) -> str:
if ((format_string_contains_key(self.dirname_pattern, 'profile') or
format_string_contains_key(self.dirname_pattern, 'target'))):
return '{0}/id'.format(self.dirname_pattern.format(profile=profile_name.lower(),
target=profile_name.lower()))
return os.path.join(self.dirname_pattern.format(profile=profile_name.lower(),
target=profile_name.lower()),
'id')
else:
return '{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower())
return os.path.join(self.dirname_pattern.format(),
'{0}_id'.format(profile_name.lower()))
def save_profile_id(self, profile: Profile):
"""
@@ -1074,9 +1093,9 @@ class Instaloader:
# Save metadata as JSON if desired.
if self.save_metadata:
json_filename = '{0}/{1}_{2}'.format(self.dirname_pattern.format(profile=profile_name,
target=profile_name),
profile_name, profile.userid)
json_filename = os.path.join(self.dirname_pattern.format(profile=profile_name,
target=profile_name),
'{0}_{1}'.format(profile_name, profile.userid))
self.save_metadata_json(json_filename, profile)
# Catch some errors
+7 -4
View File
@@ -113,7 +113,7 @@ class InstaloaderContext:
def close(self):
"""Print error log and close session"""
if self.error_log and not self.quiet:
print("\nErrors occurred:", file=sys.stderr)
print("\nErrors or warnings occurred:", file=sys.stderr)
for err in self.error_log:
print(err, file=sys.stderr)
self._session.close()
@@ -215,8 +215,11 @@ class InstaloaderContext:
session.headers.update({'X-CSRFToken': csrf_token})
# Not using self.get_json() here, because we need to access csrftoken cookie
self.do_sleep()
# Workaround credits to pgrimaud.
# See: https://github.com/pgrimaud/instagram-user-feed/commit/96ad4cf54d1ad331b337f325c73e664999a6d066
enc_password = '#PWD_INSTAGRAM_BROWSER:0:{}:{}'.format(int(datetime.now().timestamp()), passwd)
login = session.post('https://www.instagram.com/accounts/login/ajax/',
data={'password': passwd, 'username': user}, allow_redirects=True)
data={'enc_password': enc_password, 'username': user}, allow_redirects=True)
try:
resp_json = login.json()
except json.decoder.JSONDecodeError:
@@ -533,8 +536,8 @@ class InstaloaderContext:
.. versionadded:: 4.2.1"""
with copy_session(self._session, self.request_timeout) as tempsession:
tempsession.headers['User-Agent'] = 'Instagram 10.3.2 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; ' \
'scale=2.00; 750x1334) AppleWebKit/420+'
tempsession.headers['User-Agent'] = 'Instagram 123.1.0.26.115 (iPhone12,1; iOS 13_3; en_US; en-US; ' \
'scale=2.00; 1656x3584; 190542906)'
for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With']:
tempsession.headers.pop(header, None)
return self.get_json(path, params, 'i.instagram.com', tempsession)
+52 -45
View File
@@ -69,8 +69,11 @@ class Post:
self._node = node
self._owner_profile = owner_profile
self._full_metadata_dict = None # type: Optional[Dict[str, Any]]
self._rhx_gis_str = None # type: Optional[str]
self._location = None # type: Optional[PostLocation]
self._iphone_struct_ = None
if 'iphone_struct' in node:
# if loaded from JSON with load_structure_from_file()
self._iphone_struct_ = node['iphone_struct']
@classmethod
def from_shortcode(cls, context: InstaloaderContext, shortcode: str):
@@ -111,6 +114,8 @@ class Post:
node['owner'] = self.owner_profile._asdict()
if self._location:
node['location'] = self._location._asdict()
if self._iphone_struct_:
node['iphone_struct'] = self._iphone_struct_
return node
@property
@@ -136,9 +141,11 @@ class Post:
def _obtain_metadata(self):
if not self._full_metadata_dict:
pic_json = self._context.get_json("p/{0}/".format(self.shortcode), params={})
self._full_metadata_dict = pic_json['entry_data']['PostPage'][0]['graphql']['shortcode_media']
self._rhx_gis_str = pic_json.get('rhx_gis')
pic_json = self._context.graphql_query(
'2b0673e0dc4580674a88d426fe00ea90',
{'shortcode': self.shortcode}
)
self._full_metadata_dict = pic_json['data']['shortcode_media']
if self._full_metadata_dict is None:
# issue #449
self._context.error("Fetching Post metadata failed (issue #449). "
@@ -156,9 +163,13 @@ class Post:
return self._full_metadata_dict
@property
def _rhx_gis(self) -> Optional[str]:
self._obtain_metadata()
return self._rhx_gis_str
def _iphone_struct(self) -> Dict[str, Any]:
if not self._context.is_logged_in:
raise LoginRequiredException("--login required to access iPhone media info endpoint.")
if not self._iphone_struct_:
data = self._context.get_iphone_json(path='api/v1/media/{}/info/'.format(self.mediaid), params={})
self._iphone_struct_ = data['items'][0]
return self._iphone_struct_
def _field(self, *keys) -> Any:
"""Lookups given fields in _node, and if not found in _full_metadata. Raises KeyError if not found anywhere."""
@@ -225,6 +236,13 @@ class Post:
@property
def url(self) -> str:
"""URL of the picture / video thumbnail of the post"""
if self.typename == "GraphImage" and self._context.is_logged_in:
try:
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
return url
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
return self._node["display_url"] if "display_url" in self._node else self._node["display_src"]
@property
@@ -239,10 +257,18 @@ class Post:
if any(edge['node']['is_video'] for edge in edges):
# video_url is only present in full metadata, issue #558.
edges = self._full_metadata['edge_sidecar_to_children']['edges']
for edge in edges:
for idx, edge in enumerate(edges):
node = edge['node']
is_video = node['is_video']
yield PostSidecarNode(is_video=is_video, display_url=node['display_url'],
display_url = node['display_url']
if not is_video and self._context.is_logged_in:
try:
carousel_media = self._iphone_struct['carousel_media']
orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
except (InstaloaderException, KeyError, IndexError) as err:
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
yield PostSidecarNode(is_video=is_video, display_url=display_url,
video_url=node['video_url'] if is_video else None)
@property
@@ -362,7 +388,7 @@ class Post:
created_at_utc=datetime.utcfromtimestamp(node['created_at']),
text=node['text'],
owner=Profile(self._context, node['owner']),
likes_count=node['edge_liked_by']['count'])
likes_count=node.get('edge_liked_by', {}).get('count', 0))
def _postcommentanswers(node):
if 'edge_threaded_comments' not in node:
@@ -388,14 +414,9 @@ class Post:
if self.comments == 0:
# Avoid doing additional requests if there are no comments
return
try:
comment_edges = self._field('edge_media_to_parent_comment', 'edges')
answers_count = sum([edge['node']['edge_threaded_comments']['count'] for edge in comment_edges])
threaded_comments_available = True
except KeyError:
comment_edges = self._field('edge_media_to_comment', 'edges')
answers_count = 0
threaded_comments_available = False
comment_edges = self._field('edge_media_to_comment', 'edges')
answers_count = sum([edge['node'].get('edge_threaded_comments', {}).get('count', 0) for edge in comment_edges])
if self.comments == len(comment_edges) + answers_count:
# If the Post's metadata already contains all parent comments, don't do GraphQL requests to obtain them
@@ -403,14 +424,10 @@ class Post:
return
yield from (_postcomment(node) for node in
self._context.graphql_node_list(
"97b41c52301f77ce508f55e66d17620e" if threaded_comments_available
else "f0986789a5c5d17c2400faebf16efd0d",
"97b41c52301f77ce508f55e66d17620e",
{'shortcode': self.shortcode},
'https://www.instagram.com/p/' + self.shortcode + '/',
lambda d:
d['data']['shortcode_media'][
'edge_media_to_parent_comment' if threaded_comments_available else 'edge_media_to_comment'],
self._rhx_gis))
lambda d: d['data']['shortcode_media']['edge_media_to_parent_comment']))
def get_likes(self) -> Iterator['Profile']:
"""Iterate over all likes of the post. A :class:`Profile` instance of each likee is yielded."""
@@ -425,8 +442,7 @@ class Post:
yield from (Profile(self._context, node) for node in
self._context.graphql_node_list("1cb6ec562846122743b61e492c85999f", {'shortcode': self.shortcode},
'https://www.instagram.com/p/' + self.shortcode + '/',
lambda d: d['data']['shortcode_media']['edge_liked_by'],
self._rhx_gis))
lambda d: d['data']['shortcode_media']['edge_liked_by']))
@property
def is_sponsored(self) -> bool:
@@ -507,7 +523,6 @@ class Profile:
self._has_public_story = None # type: Optional[bool]
self._node = node
self._has_full_metadata = False
self._rhx_gis = None
self._iphone_struct_ = None
if 'iphone_struct' in node:
# if loaded from JSON with load_structure_from_file()
@@ -569,10 +584,9 @@ class Profile:
def _obtain_metadata(self):
try:
if not self._has_full_metadata:
metadata = self._context.get_json('{}/'.format(self.username), params={})
metadata = self._context.get_json('{}/feed/'.format(self.username), params={})
self._node = metadata['entry_data']['ProfilePage'][0]['graphql']['user']
self._has_full_metadata = True
self._rhx_gis = metadata.get('rhx_gis')
except (QueryReturnedNotFoundException, KeyError) as err:
top_search_results = TopSearchResults(self._context, self.username)
similar_profiles = [profile.username for profile in top_search_results.get_profiles()]
@@ -705,8 +719,7 @@ class Profile:
'include_reel': False, 'include_suggested_users': False,
'include_logged_out_extras': True,
'include_highlight_reels': False},
'https://www.instagram.com/{}/'.format(self.username),
self._rhx_gis)
'https://www.instagram.com/{}/'.format(self.username))
self._has_public_story = data['data']['user']['has_public_story']
assert self._has_public_story is not None
return self._has_public_story
@@ -765,8 +778,7 @@ class Profile:
{'id': self.userid},
'https://www.instagram.com/{0}/'.format(self.username),
lambda d: d['data']['user']['edge_owner_to_timeline_media'],
self._rhx_gis,
self._metadata('edge_owner_to_timeline_media')))
first_data=self._metadata('edge_owner_to_timeline_media')))
def get_saved_posts(self) -> Iterator[Post]:
"""Get Posts that are marked as saved by the user."""
@@ -780,8 +792,7 @@ class Profile:
{'id': self.userid},
'https://www.instagram.com/{0}/'.format(self.username),
lambda d: d['data']['user']['edge_saved_media'],
self._rhx_gis,
self._metadata('edge_saved_media')))
first_data=self._metadata('edge_saved_media')))
def get_tagged_posts(self) -> Iterator[Post]:
"""Retrieve all posts where a profile is tagged.
@@ -792,8 +803,7 @@ class Profile:
self._context.graphql_node_list("e31a871f7301132ceaab56507a66bbb7",
{'id': self.userid},
'https://www.instagram.com/{0}/'.format(self.username),
lambda d: d['data']['user']['edge_user_to_photos_of_you'],
self._rhx_gis))
lambda d: d['data']['user']['edge_user_to_photos_of_you']))
def get_igtv_posts(self) -> Iterator[Post]:
"""Retrieve all IGTV posts.
@@ -805,8 +815,7 @@ class Profile:
{'id': self.userid},
'https://www.instagram.com/{0}/channel/'.format(self.username),
lambda d: d['data']['user']['edge_felix_video_timeline'],
self._rhx_gis,
self._metadata('edge_felix_video_timeline')))
first_data=self._metadata('edge_felix_video_timeline')))
def get_followers(self) -> Iterator['Profile']:
"""
@@ -820,8 +829,7 @@ class Profile:
self._context.graphql_node_list("37479f2b8209594dde7facb0d904896a",
{'id': str(self.userid)},
'https://www.instagram.com/' + self.username + '/',
lambda d: d['data']['user']['edge_followed_by'],
self._rhx_gis))
lambda d: d['data']['user']['edge_followed_by']))
def get_followees(self) -> Iterator['Profile']:
"""
@@ -835,8 +843,7 @@ class Profile:
self._context.graphql_node_list("58712303d941c6855d4e888c5f0cd22f",
{'id': str(self.userid)},
'https://www.instagram.com/' + self.username + '/',
lambda d: d['data']['user']['edge_follow'],
self._rhx_gis))
lambda d: d['data']['user']['edge_follow']))
def get_similar_accounts(self) -> Iterator['Profile']:
"""
@@ -851,8 +858,8 @@ class Profile:
yield from (Profile(self._context, edge["node"]) for edge in
self._context.graphql_query("ad99dd9d3646cc3c0dda65debcd266a7",
{"user_id": str(self.userid), "include_chaining": True},
"https://www.instagram.com/{0}/".format(self.username),
self._rhx_gis)["data"]["user"]["edge_chaining"]["edges"])
"https://www.instagram.com/{0}/"
.format(self.username))["data"]["user"]["edge_chaining"]["edges"])
class StoryItem:
+4
View File
@@ -94,6 +94,10 @@ class TestInstaloaderAnonymously(unittest.TestCase):
self.assertEqual(EMPTY_PROFILE.lower(),
instaloader.Profile.from_id(self.L.context, EMPTY_PROFILE_ID).username)
def test_get_username_by_name_empty(self):
self.assertEqual(EMPTY_PROFILE_ID,
instaloader.Profile.from_username(self.L.context, EMPTY_PROFILE).userid)
def test_post_from_mediaid(self):
for post in instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts():
post2 = instaloader.Post.from_mediaid(self.L.context, post.mediaid)