mirror of
https://github.com/instaloader/instaloader.git
synced 2026-10-16 21:52:17 -07:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 André Koch-Kramer
|
195569410b |
@@ -13,7 +13,7 @@ Steps to reproduce the behavior:
|
||||
(e.g. Instaloader command line)
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen (even if it seems obvious).
|
||||
A clear and concise description of what you expected to happen (if not obvious).
|
||||
|
||||
**Error messages and tracebacks**
|
||||
If applicable, add error messages and tracebacks to help explain your problem.
|
||||
@@ -24,9 +24,4 @@ If applicable, add error messages and tracebacks to help explain your problem.
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
|
||||
<!--
|
||||
Also see https://instaloader.github.io/contributing.html for how to report a bug.
|
||||
|
||||
Please understand that a bug issue that lacks the required information cannot be
|
||||
processed.
|
||||
-->
|
||||
<!-- please also see https://instaloader.github.io/contributing.html for how to report a bug -->
|
||||
|
||||
@@ -7,12 +7,6 @@ labels: question
|
||||
|
||||
Your question here...
|
||||
|
||||
<!--
|
||||
People will be better able and more motivated to provide help if your question
|
||||
<!-- Please see our documentation: https://instaloader.github.io/ -->
|
||||
|
||||
- has a concise and meaningful title,
|
||||
- is not covered by the documentation: https://instaloader.github.io/,
|
||||
- is actually related to Instaloader (this is not a Python help forum),
|
||||
- has not already been answered here or on other sites such as Stack Overflow: https://stackoverflow.com/questions/tagged/instaloader,
|
||||
- is written in a polite and welcoming tone.
|
||||
-->
|
||||
<!-- Basic questions regarding Instaloader's usage might be more appropriate to ask on Stack Overflow -->
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
custom: "https://instaloader.github.io/contributing.html#donations"
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
|
||||
<!--
|
||||
Thanks for your willingness to contribute to Instaloader! Please briefly
|
||||
describe
|
||||
Please describe:
|
||||
|
||||
- A motivation for this change, e.g.
|
||||
- Fixes # .
|
||||
|
||||
@@ -6,4 +6,3 @@ Instaloader is written by
|
||||
- Alexander Graf (@aandergr)
|
||||
- André Koch-Kramer (@Thammus)
|
||||
- Lars Lindqvist (@e5150)
|
||||
- ... and many more, see https://github.com/instaloader/instaloader/graphs/contributors
|
||||
|
||||
@@ -1,135 +0,0 @@
|
||||
|
||||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in our
|
||||
community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socio-economic status,
|
||||
nationality, personal appearance, race, religion, or sexual identity
|
||||
and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming,
|
||||
diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment for our
|
||||
community include:
|
||||
|
||||
* Demonstrating empathy and kindness toward other people
|
||||
* Being respectful of differing opinions, viewpoints, and experiences
|
||||
* Giving and gracefully accepting constructive feedback
|
||||
* Accepting responsibility and apologizing to those affected by our mistakes,
|
||||
and learning from the experience
|
||||
* Focusing on what is best not just for us as individuals, but for the
|
||||
overall community
|
||||
|
||||
Examples of unacceptable behavior include:
|
||||
|
||||
* The use of sexualized language or imagery, and sexual attention or
|
||||
advances of any kind
|
||||
* Trolling, insulting or derogatory comments, and personal or political attacks
|
||||
* Public or private harassment
|
||||
* Publishing others' private information, such as a physical or email
|
||||
address, without their explicit permission
|
||||
* Other conduct which could reasonably be considered inappropriate in a
|
||||
professional setting
|
||||
|
||||
## Enforcement Responsibilities
|
||||
|
||||
Community leaders are responsible for clarifying and enforcing our standards of
|
||||
acceptable behavior and will take appropriate and fair corrective action in
|
||||
response to any behavior that they deem inappropriate, threatening, offensive,
|
||||
or harmful.
|
||||
|
||||
Community leaders have the right and responsibility to remove, edit, or reject
|
||||
comments, commits, code, wiki edits, issues, and other contributions that are
|
||||
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
||||
decisions when appropriate.
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces, and also applies when
|
||||
an individual is officially representing the community in public spaces.
|
||||
Examples of representing our community include using an official e-mail address,
|
||||
posting via an official social media account, or acting as an appointed
|
||||
representative at an online or offline event.
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the community leaders responsible for enforcement by opening an
|
||||
issue or contacting one or more of the project maintainers.
|
||||
|
||||
All complaints will be reviewed and investigated promptly and fairly.
|
||||
|
||||
All community leaders are obligated to respect the privacy and security of the
|
||||
reporter of any incident.
|
||||
|
||||
## Enforcement Guidelines
|
||||
|
||||
Community leaders will follow these Community Impact Guidelines in determining
|
||||
the consequences for any action they deem in violation of this Code of Conduct:
|
||||
|
||||
### 1. Correction
|
||||
|
||||
**Community Impact**: Use of inappropriate language or other behavior deemed
|
||||
unprofessional or unwelcome in the community.
|
||||
|
||||
**Consequence**: A private, written warning from community leaders, providing
|
||||
clarity around the nature of the violation and an explanation of why the
|
||||
behavior was inappropriate. A public apology may be requested.
|
||||
|
||||
### 2. Warning
|
||||
|
||||
**Community Impact**: A violation through a single incident or series
|
||||
of actions.
|
||||
|
||||
**Consequence**: A warning with consequences for continued behavior. No
|
||||
interaction with the people involved, including unsolicited interaction with
|
||||
those enforcing the Code of Conduct, for a specified period of time. This
|
||||
includes avoiding interactions in community spaces as well as external channels
|
||||
like social media. Violating these terms may lead to a temporary or
|
||||
permanent ban.
|
||||
|
||||
### 3. Temporary Ban
|
||||
|
||||
**Community Impact**: A serious violation of community standards, including
|
||||
sustained inappropriate behavior.
|
||||
|
||||
**Consequence**: A temporary ban from any sort of interaction or public
|
||||
communication with the community for a specified period of time. No public or
|
||||
private interaction with the people involved, including unsolicited interaction
|
||||
with those enforcing the Code of Conduct, is allowed during this period.
|
||||
Violating these terms may lead to a permanent ban.
|
||||
|
||||
### 4. Permanent Ban
|
||||
|
||||
**Community Impact**: Demonstrating a pattern of violation of community
|
||||
standards, including sustained inappropriate behavior, harassment of an
|
||||
individual, or aggression toward or disparagement of classes of individuals.
|
||||
|
||||
**Consequence**: A permanent ban from any sort of public interaction within
|
||||
the community.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
||||
version 2.0, available at
|
||||
[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
|
||||
|
||||
Community Impact Guidelines were inspired by
|
||||
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
|
||||
|
||||
For answers to common questions about this code of conduct, see the FAQ at
|
||||
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
|
||||
at [https://www.contributor-covenant.org/translations][translations].
|
||||
|
||||
[homepage]: https://www.contributor-covenant.org
|
||||
[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
|
||||
[Mozilla CoC]: https://github.com/mozilla/diversity
|
||||
[FAQ]: https://www.contributor-covenant.org/faq
|
||||
[translations]: https://www.contributor-covenant.org/translations
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2016-2020 Alexander Graf and André Koch-Kramer.
|
||||
Copyright (c) 2016-2019 Alexander Graf and André Koch-Kramer.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
||||
@@ -171,9 +171,6 @@ dirname pattern. The following tokens are defined for usage with
|
||||
- ``{mediaid}``
|
||||
Integer representation of shortcode.
|
||||
|
||||
- ``{filename}``
|
||||
Instagram's internal filename.
|
||||
|
||||
- ``{date_utc}`` (same as ``{date}``)
|
||||
Creation time in UTC timezone.
|
||||
`strftime()-style formatting options <https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>`__
|
||||
|
||||
+1
-14
@@ -82,16 +82,6 @@ What to Download of each Post
|
||||
Template to write in txt file for each StoryItem. See
|
||||
:ref:`metadata-text-files`.
|
||||
|
||||
.. option:: --slide
|
||||
|
||||
Download only selected images of a sidecar. You can select single images using their
|
||||
index in the sidecar starting with the leftmost or you can specify a range of images
|
||||
with the following syntax: ``start_index-end_index``. Example:
|
||||
``--slide 1`` will select only the first image, ``--slide last`` only the last one and ``--slide 1-3`` will select only
|
||||
the first three images.
|
||||
|
||||
.. versionadded:: 4.6
|
||||
|
||||
.. option:: --no-metadata-json
|
||||
|
||||
Do not create a JSON file containing the metadata of each post.
|
||||
@@ -248,13 +238,10 @@ How to Download
|
||||
|
||||
.. option:: --request-timeout N
|
||||
|
||||
Seconds to wait before timing out a connection request. Defaults to 300.
|
||||
Seconds to wait before timing out a connection request.
|
||||
|
||||
.. versionadded:: 4.3
|
||||
|
||||
.. versionchanged:: 4.6
|
||||
Enabled this option by default with a timeout of 300 seconds.
|
||||
|
||||
Miscellaneous Options
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ def get_cookiefile():
|
||||
|
||||
def import_session(cookiefile, sessionfile):
|
||||
print("Using cookies from {}.".format(cookiefile))
|
||||
conn = connect(f"file:{cookiefile}?immutable=1", uri=True)
|
||||
conn = connect(cookiefile)
|
||||
try:
|
||||
cookie_data = conn.execute(
|
||||
"SELECT name, value FROM moz_cookies WHERE baseDomain='instagram.com'"
|
||||
|
||||
+1
-1
@@ -65,7 +65,7 @@ extlinks = {
|
||||
'Issue #'
|
||||
),
|
||||
'example': (
|
||||
'https://raw.githubusercontent.com/instaloader/instaloader/master/docs/codesnippets/%s',
|
||||
'https://raw.githubusercontent.com/instaloader/instaloader/v' + current_release + '/docs/codesnippets/%s',
|
||||
'Example '
|
||||
),
|
||||
}
|
||||
|
||||
+14
-48
@@ -11,23 +11,6 @@ Instaloader's development is organized on
|
||||
`GitHub <https://github.com/instaloader/instaloader>`__, where Issues and Pull
|
||||
Requests are discussed.
|
||||
|
||||
Contributing to an open source project is an unmatchable opportunity to improve
|
||||
and manifest programming skills. For issues that are good starting points to get
|
||||
involved in Instaloader development, as they do not require deep insight into
|
||||
Instaloader's code base, have a look at our
|
||||
`Issues labelled 'good first issue' <https://github.com/instaloader/instaloader/contribute>`__.
|
||||
|
||||
This document covers several ways of how you can contribute back to Instaloader.
|
||||
|
||||
Answering Questions
|
||||
-------------------
|
||||
|
||||
The easiest way to help out is to answer questions. If you are interested in
|
||||
answering questions regarding Instaloader, good places to begin are
|
||||
|
||||
- `Questions tagged 'instaloader' on Stack Overflow <https://stackoverflow.com/questions/tagged/instaloader>`__,
|
||||
- `Instaloader Issues labeled 'question' <https://github.com/instaloader/instaloader/issues?q=is%3Aissue+is%3Aopen+label%3Aquestion>`__.
|
||||
|
||||
Reporting Bugs
|
||||
--------------
|
||||
|
||||
@@ -49,7 +32,7 @@ reporting a problem, please keep the following in mind:
|
||||
|
||||
- Include all **error messages and tracebacks** in the report.
|
||||
|
||||
- Even if it seems obvious, describe **which behavior you expected**
|
||||
- If not obvious, describe **which behavior you expected**
|
||||
instead of what actually happened.
|
||||
|
||||
- If we have closed an issue apparently inadvertently or inappropriately, please
|
||||
@@ -58,39 +41,22 @@ reporting a problem, please keep the following in mind:
|
||||
Writing Code or Improving the Documentation
|
||||
-------------------------------------------
|
||||
|
||||
Improvements of the Instaloader source or its documentation can be proposed as a
|
||||
`Pull Request <https://github.com/instaloader/instaloader/pulls>`__.
|
||||
Changes of the Instaloader source can be proposed as a
|
||||
`Pull Request <https://github.com/instaloader/instaloader/pulls>`__. There are only
|
||||
few things to consider:
|
||||
|
||||
- Please base your Pull Request on
|
||||
- Base your Pull Request on the ``master`` branch if it fixes a bug,
|
||||
or the ``upcoming/v4.X`` branch (if it exists at the moment of submitting the PR)
|
||||
otherwise.
|
||||
|
||||
- ``master``, which will be released with the next minor release, if it is
|
||||
- We use `Pylint <https://www.pylint.org/>`__ for error and syntax checking of
|
||||
the source and `MyPy <https://github.com/python/mypy>`__ for type checking.
|
||||
Beware that sometimes it might be better to disable a warning rather than
|
||||
adapting the code to a tool's desires.
|
||||
|
||||
- a bug fix that does not require extensive testing,
|
||||
- an improvement to the documentation,
|
||||
|
||||
- ``upcoming/v4.X``, if it is
|
||||
|
||||
- a new feature,
|
||||
- a bug fix that does require thorough testing before being released to a
|
||||
final version.
|
||||
|
||||
- All Pull Requests are analyzed by `Pylint <https://www.pylint.org/>`__ for
|
||||
error and syntax checking of the source and
|
||||
`Mypy <https://github.com/python/mypy>`__ for type checking. You can run them
|
||||
locally with::
|
||||
|
||||
pylint instaloader
|
||||
mypy -m instaloader
|
||||
|
||||
- Improvements to the documentation are very welcome. The documentation is
|
||||
created with `Sphinx <https://www.sphinx-doc.org/en/2.0/>`__, version 2,
|
||||
and can be build locally using::
|
||||
|
||||
make -C docs html
|
||||
|
||||
- Feel free to create an issue to make sure someone from the Instaloader team
|
||||
agrees that the change might be an improvement, or if you want to discuss
|
||||
your basic proposal, before working on a pull request.
|
||||
- The documentation source is located in the ``docs`` folder. The file
|
||||
``cli-options.rst`` is merely an RST-formatted copy of ``instaloader --help``
|
||||
output, of which the source is in ``instaloader/__main__.py``.
|
||||
|
||||
Proposing Features
|
||||
------------------
|
||||
|
||||
@@ -33,9 +33,6 @@ To **upgrade Instaloader** to its current version, do::
|
||||
- On **Windows 10**, you may download the standalone executable from the
|
||||
`current release page <https://github.com/instaloader/instaloader/releases/latest>`__.
|
||||
|
||||
- On **Android**, you can use Instaloader with `Termux <https://play.google.com/store/apps/details?id=com.termux>`__
|
||||
after typing ``pkg install python`` and ``pip3 install instaloader``.
|
||||
|
||||
- To test the most current **pre-release** version of Instaloader::
|
||||
|
||||
pip3 install --pre instaloader
|
||||
|
||||
+10
-17
@@ -9,7 +9,10 @@ Troubleshooting
|
||||
---------------------
|
||||
|
||||
Instaloader has a logic to keep track of its requests to Instagram and to obey
|
||||
their rate limits. The rate controller assumes that
|
||||
their rate limits. Since they are nowhere documented, we try them out
|
||||
experimentally. We have a daily cron job running to confirm that Instaloader
|
||||
still stays within the rate limits. Nevertheless, the rate control logic assumes
|
||||
that
|
||||
|
||||
- at one time, Instaloader is the only application that consumes requests, i.e.
|
||||
neither the Instagram browser interface, nor a mobile app, nor another
|
||||
@@ -18,13 +21,7 @@ their rate limits. The rate controller assumes that
|
||||
- no requests had been consumed when Instaloader starts.
|
||||
|
||||
The latter one implies that restarting or reinstantiating Instaloader often
|
||||
within short time is prone to cause a 429.
|
||||
|
||||
Since the behavior of the rate controller might change between different
|
||||
versions of Instaloader, make sure to use the current version of Instaloader,
|
||||
especially when encountering many 429 errors.
|
||||
|
||||
If a request is denied with a 429,
|
||||
within short time is prone to cause a 429. If a request is denied with a 429,
|
||||
Instaloader retries the request as soon as the temporary ban is assumed to be
|
||||
expired. In case the retry continuously fails for some reason, which should not
|
||||
happen under normal conditions, consider adjusting the
|
||||
@@ -35,18 +32,13 @@ promiscuous IP addresses, such as cloud, VPN and public proxy services, might be
|
||||
subject to significantly stricter limits for anonymous access. However,
|
||||
logged-in accesses (see :option:`--login`) do not seem to be affected.
|
||||
|
||||
Instaloader allows to adjust the rate controlling behavior by overriding
|
||||
:class:`instaloader.RateController`.
|
||||
|
||||
Too many queries in the last time
|
||||
---------------------------------
|
||||
|
||||
**"Too many queries in the last time"** is not an error. It is a notice that the
|
||||
rate limit has almost been reached, according to Instaloader's own rate
|
||||
accounting mechanism.
|
||||
|
||||
Instaloader allows to adjust the rate controlling behavior by overriding
|
||||
:class:`instaloader.RateController`.
|
||||
accounting mechanism. We regularly adjust this mechanism to match Instagram's
|
||||
current rate limiting.
|
||||
|
||||
Private but not followed
|
||||
------------------------
|
||||
@@ -65,8 +57,9 @@ pointing the user to an URL to be opened in a browser.
|
||||
Nevertheless, in :issue:`92` and :issue:`615` users reported problems with
|
||||
logging in. We recommend to always keep the session file which Instaloader
|
||||
creates when using :option:`--login`. If a session file is present,
|
||||
:option:`--login` does not make use of the failure-prone login procedure.
|
||||
Also, session files usually do not expire.
|
||||
:option:`--login` does not make make use of the failure-prone login procedure.
|
||||
Also, session files usually do not expire and can be copied between different
|
||||
computers without any problems.
|
||||
|
||||
If you do not have a session file present, you may use the following script
|
||||
(:example:`615_import_firefox_session.py`) to workaround login problems by
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Download pictures (or videos) along with their captions and other metadata from Instagram."""
|
||||
|
||||
|
||||
__version__ = '4.6'
|
||||
__version__ = '4.5'
|
||||
|
||||
|
||||
try:
|
||||
|
||||
@@ -222,13 +222,14 @@ def _main(instaloader: Instaloader, targetlist: List[str],
|
||||
# Instaloader did at least save a session file
|
||||
instaloader.context.log("No targets were specified, thus nothing has been downloaded.")
|
||||
else:
|
||||
# Instaloader did not do anything
|
||||
# Instloader did not do anything
|
||||
instaloader.context.log("usage:" + usage_string())
|
||||
|
||||
|
||||
def main():
|
||||
parser = ArgumentParser(description=__doc__, add_help=False, usage=usage_string(),
|
||||
epilog="The complete documentation can be found at "
|
||||
epilog="Report issues at https://github.com/instaloader/instaloader/issues. "
|
||||
"The complete documentation can be found at "
|
||||
"https://instaloader.github.io/.",
|
||||
fromfile_prefix_chars='+')
|
||||
|
||||
@@ -267,8 +268,6 @@ def main():
|
||||
help="Do not download regular posts.")
|
||||
g_prof.add_argument('--no-profile-pic', action='store_true',
|
||||
help='Do not download profile picture.')
|
||||
g_post.add_argument('--slide', action='store',
|
||||
help='Set what image/interval of a sidecar you want to download.')
|
||||
g_post.add_argument('--no-pictures', action='store_true',
|
||||
help='Do not download post pictures. Cannot be used together with --fast-update. '
|
||||
'Implies --no-video-thumbnails, does not imply --no-videos.')
|
||||
@@ -367,8 +366,8 @@ def main():
|
||||
'connection fails, it can be manually skipped by hitting CTRL+C. Set this to 0 to retry '
|
||||
'infinitely.')
|
||||
g_how.add_argument('--commit-mode', action='store_true', help=SUPPRESS)
|
||||
g_how.add_argument('--request-timeout', metavar='N', type=float, default=300.0,
|
||||
help='Seconds to wait before timing out a connection request. Defaults to 300.')
|
||||
g_how.add_argument('--request-timeout', metavar='N', type=float,
|
||||
help='seconds to wait before timing out a connection request')
|
||||
|
||||
g_misc = parser.add_argument_group('Miscellaneous Options')
|
||||
g_misc.add_argument('-q', '--quiet', action='store_true',
|
||||
@@ -426,8 +425,7 @@ def main():
|
||||
max_connection_attempts=args.max_connection_attempts,
|
||||
request_timeout=args.request_timeout,
|
||||
resume_prefix=resume_prefix,
|
||||
check_resume_bbd=not args.use_aged_resume_files,
|
||||
slide=args.slide)
|
||||
check_resume_bbd=not args.use_aged_resume_files)
|
||||
_main(loader,
|
||||
args.profile,
|
||||
username=args.login.lower() if args.login is not None else None,
|
||||
|
||||
+45
-102
@@ -13,8 +13,7 @@ from functools import wraps
|
||||
from hashlib import md5
|
||||
from io import BytesIO
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, IO, Iterator, List, Optional, Set, Union, cast
|
||||
from urllib.parse import urlparse
|
||||
from typing import Any, Callable, ContextManager, IO, Iterator, List, Optional, Set, Union, cast
|
||||
|
||||
import requests
|
||||
import urllib3 # type: ignore
|
||||
@@ -23,7 +22,7 @@ from .exceptions import *
|
||||
from .instaloadercontext import InstaloaderContext, RateController
|
||||
from .nodeiterator import NodeIterator, resumable_iteration
|
||||
from .structures import (Hashtag, Highlight, JsonExportable, Post, PostLocation, Profile, Story, StoryItem,
|
||||
load_structure_from_file, save_structure_to_file, PostSidecarNode)
|
||||
load_structure_from_file, save_structure_to_file)
|
||||
|
||||
|
||||
def get_default_session_filename(username: str) -> str:
|
||||
@@ -102,8 +101,6 @@ class _ArbitraryItemFormatter(string.Formatter):
|
||||
|
||||
def get_value(self, key, args, kwargs):
|
||||
"""Override to substitute {ATTRIBUTE} by attributes of our _item."""
|
||||
if key == 'filename' and isinstance(self._item, (Post, StoryItem, PostSidecarNode)):
|
||||
return "{filename}"
|
||||
if hasattr(self._item, key):
|
||||
return getattr(self._item, key)
|
||||
return super().get_value(key, args, kwargs)
|
||||
@@ -160,7 +157,6 @@ class Instaloader:
|
||||
:param rate_controller: Generator for a :class:`RateController` to override rate controlling behavior
|
||||
:param resume_prefix: :option:`--resume-prefix`, or None for :option:`--no-resume`.
|
||||
:param check_resume_bbd: Whether to check the date of expiry of resume files and reject them if expired.
|
||||
:param slide: :option:`--slide`
|
||||
|
||||
.. attribute:: context
|
||||
|
||||
@@ -183,11 +179,10 @@ class Instaloader:
|
||||
post_metadata_txt_pattern: str = None,
|
||||
storyitem_metadata_txt_pattern: str = None,
|
||||
max_connection_attempts: int = 3,
|
||||
request_timeout: float = 300.0,
|
||||
request_timeout: Optional[float] = None,
|
||||
rate_controller: Optional[Callable[[InstaloaderContext], RateController]] = None,
|
||||
resume_prefix: Optional[str] = "iterator",
|
||||
check_resume_bbd: bool = True,
|
||||
slide: Optional[str] = None):
|
||||
check_resume_bbd: bool = True):
|
||||
|
||||
self.context = InstaloaderContext(sleep, quiet, user_agent, max_connection_attempts,
|
||||
request_timeout, rate_controller)
|
||||
@@ -209,31 +204,6 @@ class Instaloader:
|
||||
self.resume_prefix = resume_prefix
|
||||
self.check_resume_bbd = check_resume_bbd
|
||||
|
||||
self.slide = slide or ""
|
||||
self.slide_start = 0
|
||||
self.slide_end = -1
|
||||
if self.slide != "":
|
||||
splitted = self.slide.split('-')
|
||||
if len(splitted) == 1:
|
||||
if splitted[0] == 'last':
|
||||
# download only last image of a sidecar
|
||||
self.slide_start = -1
|
||||
else:
|
||||
if int(splitted[0]) > 0:
|
||||
self.slide_start = self.slide_end = int(splitted[0])-1
|
||||
else:
|
||||
raise InvalidArgumentException("--slide parameter must be greater than 0.")
|
||||
elif len(splitted) == 2:
|
||||
if splitted[1] == 'last':
|
||||
self.slide_start = int(splitted[0])-1
|
||||
elif 0 < int(splitted[0]) < int(splitted[1]):
|
||||
self.slide_start = int(splitted[0])-1
|
||||
self.slide_end = int(splitted[1])-1
|
||||
else:
|
||||
raise InvalidArgumentException("Invalid data for --slide parameter.")
|
||||
else:
|
||||
raise InvalidArgumentException("Invalid data for --slide parameter.")
|
||||
|
||||
@contextmanager
|
||||
def anonymous_copy(self):
|
||||
"""Yield an anonymous, otherwise equally-configured copy of an Instaloader instance; Then copy its error log."""
|
||||
@@ -255,8 +225,7 @@ class Instaloader:
|
||||
max_connection_attempts=self.context.max_connection_attempts,
|
||||
request_timeout=self.context.request_timeout,
|
||||
resume_prefix=self.resume_prefix,
|
||||
check_resume_bbd=self.check_resume_bbd,
|
||||
slide=self.slide)
|
||||
check_resume_bbd=self.check_resume_bbd)
|
||||
yield new_loader
|
||||
self.context.error_log.extend(new_loader.context.error_log)
|
||||
new_loader.context.error_log = [] # avoid double-printing of errors
|
||||
@@ -501,7 +470,13 @@ class Instaloader:
|
||||
|
||||
def test_login(self) -> Optional[str]:
|
||||
"""Returns the Instagram username to which given :class:`requests.Session` object belongs, or None."""
|
||||
return self.context.test_login()
|
||||
data = self.context.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {}).get('data', {}).get('user')
|
||||
if data is not None:
|
||||
profile = Profile.from_node(self.context, {'id': data.get('id'),
|
||||
'username': data.get('username'),
|
||||
'profile_pic_url_hd': data.get('profile_pic_url')})
|
||||
return profile.username
|
||||
return None
|
||||
|
||||
def login(self, user: str, passwd: str) -> None:
|
||||
"""Log in to instagram with given username and password and internally store session object.
|
||||
@@ -523,20 +498,7 @@ class Instaloader:
|
||||
.. versionadded:: 4.2"""
|
||||
self.context.two_factor_login(two_factor_code)
|
||||
|
||||
@staticmethod
|
||||
def __prepare_filename(filename_template: str, url: Callable[[], str]) -> str:
|
||||
"""Replace filename token inside filename_template with url's filename and assure the directories exist.
|
||||
|
||||
.. versionadded:: 4.6"""
|
||||
if "{filename}" in filename_template:
|
||||
filename = filename_template.replace("{filename}",
|
||||
os.path.splitext(os.path.basename(urlparse(url()).path))[0])
|
||||
else:
|
||||
filename = filename_template
|
||||
os.makedirs(os.path.dirname(filename), exist_ok=True)
|
||||
return filename
|
||||
|
||||
def format_filename(self, item: Union[Post, StoryItem, PostSidecarNode], target: Optional[Union[str, Path]] = None):
|
||||
def format_filename(self, item: Union[Post, StoryItem], target: Optional[Union[str, Path]] = None):
|
||||
"""Format filename of a :class:`Post` or :class:`StoryItem` according to ``filename-pattern`` parameter.
|
||||
|
||||
.. versionadded:: 4.1"""
|
||||
@@ -552,46 +514,31 @@ class Instaloader:
|
||||
"""
|
||||
|
||||
dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target)
|
||||
filename_template = os.path.join(dirname, self.format_filename(post, target=target))
|
||||
filename = self.__prepare_filename(filename_template, lambda: post.url)
|
||||
filename = os.path.join(dirname, self.format_filename(post, target=target))
|
||||
os.makedirs(os.path.dirname(filename), exist_ok=True)
|
||||
|
||||
# Download the image(s) / video thumbnail and videos within sidecars if desired
|
||||
downloaded = True
|
||||
if post.typename == 'GraphSidecar':
|
||||
if self.download_pictures or self.download_videos:
|
||||
for edge_number, sidecar_node in enumerate(
|
||||
post.get_sidecar_nodes(self.slide_start, self.slide_end),
|
||||
start=post.mediacount if self.slide_start < 0 else self.slide_start + 1
|
||||
):
|
||||
if self.download_pictures and (not sidecar_node.is_video or self.download_video_thumbnails):
|
||||
suffix = str(edge_number)
|
||||
if '{filename}' in self.filename_pattern:
|
||||
suffix = ''
|
||||
# pylint:disable=cell-var-from-loop
|
||||
filename = self.__prepare_filename(filename_template, lambda: sidecar_node.display_url)
|
||||
# Download sidecar picture or video thumbnail (--no-pictures implies --no-video-thumbnails)
|
||||
if self.download_pictures:
|
||||
if post.typename == 'GraphSidecar':
|
||||
edge_number = 1
|
||||
for sidecar_node in post.get_sidecar_nodes():
|
||||
# Download picture or video thumbnail
|
||||
if not sidecar_node.is_video or self.download_video_thumbnails is True:
|
||||
downloaded &= self.download_pic(filename=filename, url=sidecar_node.display_url,
|
||||
mtime=post.date_local, filename_suffix=suffix)
|
||||
if sidecar_node.is_video and self.download_videos:
|
||||
suffix = str(edge_number)
|
||||
if '{filename}' in self.filename_pattern:
|
||||
suffix = ''
|
||||
# pylint:disable=cell-var-from-loop
|
||||
filename = self.__prepare_filename(filename_template, lambda: sidecar_node.video_url)
|
||||
# Download sidecar video if desired
|
||||
mtime=post.date_local, filename_suffix=str(edge_number))
|
||||
# Additionally download video if available and desired
|
||||
if sidecar_node.is_video and self.download_videos is True:
|
||||
downloaded &= self.download_pic(filename=filename, url=sidecar_node.video_url,
|
||||
mtime=post.date_local, filename_suffix=suffix)
|
||||
elif post.typename == 'GraphImage':
|
||||
# Download picture
|
||||
if self.download_pictures:
|
||||
mtime=post.date_local, filename_suffix=str(edge_number))
|
||||
edge_number += 1
|
||||
elif post.typename == 'GraphImage':
|
||||
downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
|
||||
elif post.typename == 'GraphVideo':
|
||||
# Download video thumbnail (--no-pictures implies --no-video-thumbnails)
|
||||
if self.download_pictures and self.download_video_thumbnails:
|
||||
with self.context.error_catcher("Video thumbnail of {}".format(post)):
|
||||
elif post.typename == 'GraphVideo':
|
||||
if self.download_video_thumbnails is True:
|
||||
downloaded = self.download_pic(filename=filename, url=post.url, mtime=post.date_local)
|
||||
else:
|
||||
self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename))
|
||||
else:
|
||||
self.context.error("Warning: {0} has unknown typename: {1}".format(post, post.typename))
|
||||
|
||||
# Save caption if desired
|
||||
metadata_string = _ArbitraryItemFormatter(post).format(self.post_metadata_txt_pattern).strip()
|
||||
@@ -599,7 +546,7 @@ class Instaloader:
|
||||
self.save_caption(filename=filename, mtime=post.date_local, caption=metadata_string)
|
||||
|
||||
# Download video if desired
|
||||
if post.is_video and self.download_videos:
|
||||
if post.is_video and self.download_videos is True:
|
||||
downloaded &= self.download_pic(filename=filename, url=post.video_url, mtime=post.date_local)
|
||||
|
||||
# Download geotags if desired
|
||||
@@ -607,11 +554,11 @@ class Instaloader:
|
||||
self.save_location(filename, post.location, post.date_local)
|
||||
|
||||
# Update comments if desired
|
||||
if self.download_comments:
|
||||
if self.download_comments is True:
|
||||
self.update_comments(filename=filename, post=post)
|
||||
|
||||
# Save metadata as JSON if desired.
|
||||
if self.save_metadata:
|
||||
if self.save_metadata is not False:
|
||||
self.save_metadata_json(filename, post)
|
||||
|
||||
self.context.log()
|
||||
@@ -636,12 +583,12 @@ class Instaloader:
|
||||
|
||||
def _userid_chunks():
|
||||
assert userids is not None
|
||||
userids_per_query = 50
|
||||
userids_per_query = 100
|
||||
for i in range(0, len(userids), userids_per_query):
|
||||
yield userids[i:i + userids_per_query]
|
||||
|
||||
for userid_chunk in _userid_chunks():
|
||||
stories = self.context.graphql_query("303a4ae99711322310f25250d988f3b7",
|
||||
stories = self.context.graphql_query("bf41e22b1c4ba4c9f31b844ebb7d9056",
|
||||
{"reel_ids": userid_chunk, "precomposed_overlay": False})["data"]
|
||||
yield from (Story(self.context, media) for media in stories['reels_media'])
|
||||
|
||||
@@ -695,14 +642,13 @@ class Instaloader:
|
||||
|
||||
date_local = item.date_local
|
||||
dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target)
|
||||
filename_template = os.path.join(dirname, self.format_filename(item, target=target))
|
||||
filename = self.__prepare_filename(filename_template, lambda: item.url)
|
||||
filename = os.path.join(dirname, self.format_filename(item, target=target))
|
||||
os.makedirs(os.path.dirname(filename), exist_ok=True)
|
||||
downloaded = False
|
||||
if not item.is_video or self.download_video_thumbnails is True:
|
||||
url = item.url
|
||||
downloaded = self.download_pic(filename=filename, url=url, mtime=date_local)
|
||||
if item.is_video and self.download_videos is True:
|
||||
filename = self.__prepare_filename(filename_template, lambda: str(item.video_url))
|
||||
downloaded |= self.download_pic(filename=filename, url=item.video_url, mtime=date_local)
|
||||
# Save caption if desired
|
||||
metadata_string = _ArbitraryItemFormatter(item).format(self.storyitem_metadata_txt_pattern).strip()
|
||||
@@ -915,7 +861,7 @@ class Instaloader:
|
||||
"""
|
||||
self.context.log("Retrieving saved posts...")
|
||||
assert self.context.username is not None # safe due to @_requires_login; required by typechecker
|
||||
node_iterator = Profile.own_profile(self.context).get_saved_posts()
|
||||
node_iterator = Profile.from_username(self.context, self.context.username).get_saved_posts()
|
||||
self.posts_download_loop(node_iterator, ":saved",
|
||||
fast_update, post_filter,
|
||||
max_count=max_count, total_count=node_iterator.count)
|
||||
@@ -1167,11 +1113,11 @@ class Instaloader:
|
||||
def _error_raiser(_str):
|
||||
yield
|
||||
|
||||
# error_handler type is Callable[[Optional[str]], ContextManager[None]] (not supported with Python 3.5.0..3.5.3)
|
||||
error_handler = _error_raiser if raise_errors else self.context.error_catcher
|
||||
error_handler = cast(Callable[[Optional[str]], ContextManager[None]],
|
||||
_error_raiser if raise_errors else self.context.error_catcher)
|
||||
|
||||
for profile in profiles:
|
||||
with error_handler(profile.username): # type: ignore # (ignore type for Python 3.5 support)
|
||||
with error_handler(profile.username):
|
||||
profile_name = profile.username
|
||||
|
||||
# Download profile picture
|
||||
@@ -1187,13 +1133,10 @@ class Instaloader:
|
||||
self.save_metadata_json(json_filename, profile)
|
||||
|
||||
# Catch some errors
|
||||
if tagged or igtv or highlights or posts:
|
||||
if (not self.context.is_logged_in and
|
||||
profile.is_private):
|
||||
if profile.is_private and (tagged or igtv or highlights or posts):
|
||||
if not self.context.is_logged_in:
|
||||
raise LoginRequiredException("--login=USERNAME required.")
|
||||
if (self.context.username != profile.username and
|
||||
profile.is_private and
|
||||
not profile.followed_by_viewer):
|
||||
if not profile.followed_by_viewer and self.context.username != profile.username:
|
||||
raise PrivateProfileNotFollowedException("Private but not followed.")
|
||||
|
||||
# Download tagged, if requested
|
||||
|
||||
@@ -25,9 +25,10 @@ def copy_session(session: requests.Session, request_timeout: Optional[float] = N
|
||||
new = requests.Session()
|
||||
new.cookies = requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
|
||||
new.headers = session.headers.copy()
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
new.request = partial(new.request, timeout=request_timeout) # type: ignore
|
||||
if request_timeout is not None:
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
new.request = partial(new.request, timeout=request_timeout) # type: ignore
|
||||
return new
|
||||
|
||||
|
||||
@@ -52,7 +53,7 @@ class InstaloaderContext:
|
||||
"""
|
||||
|
||||
def __init__(self, sleep: bool = True, quiet: bool = False, user_agent: Optional[str] = None,
|
||||
max_connection_attempts: int = 3, request_timeout: float = 300.0,
|
||||
max_connection_attempts: int = 3, request_timeout: Optional[float] = None,
|
||||
rate_controller: Optional[Callable[["InstaloaderContext"], "RateController"]] = None):
|
||||
|
||||
self.user_agent = user_agent if user_agent is not None else default_user_agent()
|
||||
@@ -77,6 +78,9 @@ class InstaloaderContext:
|
||||
# Cache profile from id (mapping from id to Profile)
|
||||
self.profile_id_cache = dict() # type: Dict[int, Any]
|
||||
|
||||
# Cache profile from name (mapping from username to Profile)
|
||||
self.profile_name_cache = dict() # type: Dict[str, Any]
|
||||
|
||||
@contextmanager
|
||||
def anonymous_copy(self):
|
||||
session = self._session
|
||||
@@ -160,9 +164,10 @@ class InstaloaderContext:
|
||||
'ig_vw': '1920', 'csrftoken': '',
|
||||
's_network': '', 'ds_user_id': ''})
|
||||
session.headers.update(self._default_http_header(empty_session_only=True))
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
|
||||
if self.request_timeout is not None:
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
|
||||
return session
|
||||
|
||||
def save_session_to_file(self, sessionfile):
|
||||
@@ -175,14 +180,20 @@ class InstaloaderContext:
|
||||
session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile))
|
||||
session.headers.update(self._default_http_header())
|
||||
session.headers.update({'X-CSRFToken': session.cookies.get_dict()['csrftoken']})
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
|
||||
if self.request_timeout is not None:
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
|
||||
self._session = session
|
||||
self.username = username
|
||||
|
||||
def test_login(self) -> Optional[str]:
|
||||
"""Not meant to be used directly, use :meth:`Instaloader.test_login`."""
|
||||
"""Not meant to be used directly, use :meth:`Instaloader.test_login`.
|
||||
|
||||
.. deprecated:: 4.5
|
||||
:meth:`Instaloader.test_login` now calls :meth:`InstaloaderContext.graphql_query` directly
|
||||
without using this function.
|
||||
"""
|
||||
data = self.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})
|
||||
return data["data"]["user"]["username"] if data["data"]["user"] is not None else None
|
||||
|
||||
@@ -203,9 +214,10 @@ class InstaloaderContext:
|
||||
'ig_vw': '1920', 'ig_cb': '1', 'csrftoken': '',
|
||||
's_network': '', 'ds_user_id': ''})
|
||||
session.headers.update(self._default_http_header())
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
|
||||
if self.request_timeout is not None:
|
||||
# Override default timeout behavior.
|
||||
# Need to silence mypy bug for this. See: https://github.com/python/mypy/issues/2427
|
||||
session.request = partial(session.request, timeout=self.request_timeout) # type: ignore
|
||||
session.get('https://www.instagram.com/web/__mid/')
|
||||
csrf_token = session.cookies.get_dict()['csrftoken']
|
||||
session.headers.update({'X-CSRFToken': csrf_token})
|
||||
@@ -320,10 +332,8 @@ class InstaloaderContext:
|
||||
redirect_url = resp.headers['location']
|
||||
self.log('\nHTTP redirect from https://{0}/{1} to {2}'.format(host, path, redirect_url))
|
||||
if redirect_url.startswith('https://www.instagram.com/accounts/login'):
|
||||
if not self.is_logged_in:
|
||||
raise LoginRequiredException("Redirected to login page. Use --login.")
|
||||
# alternate rate limit exceeded behavior
|
||||
raise TooManyRequestsException("Redirected to login")
|
||||
raise TooManyRequestsException("429 Too Many Requests: redirected to login")
|
||||
if redirect_url.startswith('https://{}/'.format(host)):
|
||||
resp = sess.get(redirect_url if redirect_url.endswith('/') else redirect_url + '/',
|
||||
params=params, allow_redirects=False)
|
||||
@@ -349,10 +359,10 @@ class InstaloaderContext:
|
||||
raise ConnectionException("\"window._sharedData\" does not contain required keys.")
|
||||
# If GraphQL data is missing in `window._sharedData`, search for it in `__additionalDataLoaded`.
|
||||
if 'graphql' not in post_or_profile_page[0]:
|
||||
match = re.search(r'window\.__additionalDataLoaded\(.*?({.*"graphql":.*})\);</script>',
|
||||
match = re.search(r'window\.__additionalDataLoaded\([^{]+{"graphql":({.*})}\);</script>',
|
||||
resp.text)
|
||||
if match is not None:
|
||||
post_or_profile_page[0]['graphql'] = json.loads(match.group(1))['graphql']
|
||||
post_or_profile_page[0]['graphql'] = json.loads(match.group(1))
|
||||
return resp_json
|
||||
else:
|
||||
resp_json = resp.json()
|
||||
@@ -469,7 +479,7 @@ class InstaloaderContext:
|
||||
|
||||
.. versionadded:: 4.2.1"""
|
||||
with copy_session(self._session, self.request_timeout) as tempsession:
|
||||
tempsession.headers['User-Agent'] = 'Instagram 146.0.0.27.125 (iPhone12,1; iOS 13_3; en_US; en-US; ' \
|
||||
tempsession.headers['User-Agent'] = 'Instagram 123.1.0.26.115 (iPhone12,1; iOS 13_3; en_US; en-US; ' \
|
||||
'scale=2.00; 1656x3584; 190542906)'
|
||||
for header in ['Host', 'Origin', 'X-Instagram-AJAX', 'X-Requested-With']:
|
||||
tempsession.headers.pop(header, None)
|
||||
@@ -485,7 +495,7 @@ class InstaloaderContext:
|
||||
shutil.copyfileobj(resp.raw, file)
|
||||
else:
|
||||
file.write(resp)
|
||||
os.replace(filename + '.temp', filename)
|
||||
os.rename(filename + '.temp', filename)
|
||||
|
||||
def get_raw(self, url: str, _attempt=1) -> requests.Response:
|
||||
"""Downloads a file anonymously.
|
||||
@@ -547,8 +557,8 @@ class RateController:
|
||||
|
||||
def __init__(self, context: InstaloaderContext):
|
||||
self._context = context
|
||||
self._query_timestamps = dict() # type: Dict[str, List[float]]
|
||||
self._earliest_next_request_time = 0.0
|
||||
self._graphql_query_timestamps = dict() # type: Dict[str, List[float]]
|
||||
self._graphql_earliest_next_request_time = 0.0
|
||||
|
||||
def sleep(self, secs: float):
|
||||
"""Wait given number of seconds."""
|
||||
@@ -557,75 +567,38 @@ class RateController:
|
||||
# pylint:disable=no-self-use
|
||||
time.sleep(secs)
|
||||
|
||||
def _dump_query_timestamps(self, current_time: float, failed_query_type: str):
|
||||
windows = [10, 11, 20, 22, 30, 60]
|
||||
self._context.error("Requests within last {} minutes grouped by type:"
|
||||
.format('/'.join(str(w) for w in windows)),
|
||||
repeat_at_end=False)
|
||||
for query_type, times in self._query_timestamps.items():
|
||||
reqs_in_sliding_window = [sum(t > current_time - w * 60 for t in times) for w in windows]
|
||||
self._context.error(" {} {:>32}: {}".format(
|
||||
"*" if query_type == failed_query_type else " ",
|
||||
query_type,
|
||||
" ".join("{:4}".format(reqs) for reqs in reqs_in_sliding_window)
|
||||
), repeat_at_end=False)
|
||||
def _dump_query_timestamps(self, current_time: float):
|
||||
windows = [10, 11, 15, 20, 30, 60]
|
||||
print("GraphQL requests:", file=sys.stderr)
|
||||
for query_hash, times in self._graphql_query_timestamps.items():
|
||||
print(" {}".format(query_hash), file=sys.stderr)
|
||||
for window in windows:
|
||||
reqs_in_sliding_window = sum(t > current_time - window * 60 for t in times)
|
||||
print(" last {} minutes: {} requests".format(window, reqs_in_sliding_window), file=sys.stderr)
|
||||
|
||||
def count_per_sliding_window(self, query_type: str) -> int:
|
||||
"""Return how many requests can be done within the sliding window."""
|
||||
"""Return how many GraphQL requests can be done within the sliding window."""
|
||||
# Not static, to allow for the count_per_sliding_window to depend on context-inherent properties, such as
|
||||
# whether we are logged in.
|
||||
# pylint:disable=no-self-use
|
||||
return 75 if query_type in ['iphone', 'other'] else 200
|
||||
|
||||
def _reqs_in_sliding_window(self, query_type: Optional[str], current_time: float, window: float) -> List[float]:
|
||||
if query_type is not None:
|
||||
# timestamps of type query_type
|
||||
relevant_timestamps = self._query_timestamps[query_type]
|
||||
else:
|
||||
# all GraphQL queries, i.e. not 'iphone' or 'other'
|
||||
graphql_query_timestamps = filter(lambda tp: tp[0] not in ['iphone', 'other'],
|
||||
self._query_timestamps.items())
|
||||
relevant_timestamps = [t for times in (tp[1] for tp in graphql_query_timestamps) for t in times]
|
||||
return list(filter(lambda t: t > current_time - window, relevant_timestamps))
|
||||
# pylint:disable=no-self-use,unused-argument
|
||||
return 200
|
||||
|
||||
def query_waittime(self, query_type: str, current_time: float, untracked_queries: bool = False) -> float:
|
||||
"""Calculate time needed to wait before query can be executed."""
|
||||
per_type_sliding_window = 660
|
||||
if query_type not in self._query_timestamps:
|
||||
self._query_timestamps[query_type] = []
|
||||
self._query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60,
|
||||
self._query_timestamps[query_type]))
|
||||
|
||||
def per_type_next_request_time():
|
||||
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, per_type_sliding_window)
|
||||
if len(reqs_in_sliding_window) < self.count_per_sliding_window(query_type):
|
||||
return 0.0
|
||||
else:
|
||||
return min(reqs_in_sliding_window) + per_type_sliding_window + 6
|
||||
|
||||
def gql_accumulated_next_request_time():
|
||||
if query_type in ['iphone', 'other']:
|
||||
return 0.0
|
||||
gql_accumulated_sliding_window = 600
|
||||
gql_accumulated_max_count = 275
|
||||
reqs_in_sliding_window = self._reqs_in_sliding_window(None, current_time, gql_accumulated_sliding_window)
|
||||
if len(reqs_in_sliding_window) < gql_accumulated_max_count:
|
||||
return 0.0
|
||||
else:
|
||||
return min(reqs_in_sliding_window) + gql_accumulated_sliding_window
|
||||
|
||||
def untracked_next_request_time():
|
||||
if untracked_queries:
|
||||
reqs_in_sliding_window = self._reqs_in_sliding_window(query_type, current_time, per_type_sliding_window)
|
||||
self._earliest_next_request_time = min(reqs_in_sliding_window) + per_type_sliding_window + 6
|
||||
return self._earliest_next_request_time
|
||||
|
||||
return max(0.0,
|
||||
max(
|
||||
per_type_next_request_time(),
|
||||
gql_accumulated_next_request_time(),
|
||||
untracked_next_request_time(),
|
||||
) - current_time)
|
||||
"""Calculate time needed to wait before GraphQL query can be executed."""
|
||||
sliding_window = 660
|
||||
if query_type not in self._graphql_query_timestamps:
|
||||
self._graphql_query_timestamps[query_type] = []
|
||||
self._graphql_query_timestamps[query_type] = list(filter(lambda t: t > current_time - 60 * 60,
|
||||
self._graphql_query_timestamps[query_type]))
|
||||
reqs_in_sliding_window = list(filter(lambda t: t > current_time - sliding_window,
|
||||
self._graphql_query_timestamps[query_type]))
|
||||
count_per_sliding_window = self.count_per_sliding_window(query_type)
|
||||
if len(reqs_in_sliding_window) < count_per_sliding_window and not untracked_queries:
|
||||
return max(0.0, self._graphql_earliest_next_request_time - current_time)
|
||||
next_request_time = min(reqs_in_sliding_window) + sliding_window + 6
|
||||
if untracked_queries:
|
||||
self._graphql_earliest_next_request_time = next_request_time
|
||||
return max(next_request_time, self._graphql_earliest_next_request_time) - current_time
|
||||
|
||||
def wait_before_query(self, query_type: str) -> None:
|
||||
"""This method is called before a query to Instagram. It calls :meth:`RateController.sleep` to wait
|
||||
@@ -637,25 +610,24 @@ class RateController:
|
||||
.format(round(waittime), datetime.now() + timedelta(seconds=waittime)))
|
||||
if waittime > 0:
|
||||
self.sleep(waittime)
|
||||
if query_type not in self._query_timestamps:
|
||||
self._query_timestamps[query_type] = [time.monotonic()]
|
||||
if query_type not in self._graphql_query_timestamps:
|
||||
self._graphql_query_timestamps[query_type] = [time.monotonic()]
|
||||
else:
|
||||
self._query_timestamps[query_type].append(time.monotonic())
|
||||
self._graphql_query_timestamps[query_type].append(time.monotonic())
|
||||
|
||||
def handle_429(self, query_type: str) -> None:
|
||||
"""This method is called to handle a 429 Too Many Requests response. It calls :meth:`RateController.sleep` to
|
||||
wait until we can repeat the same request."""
|
||||
text_for_429 = ("HTTP error code 429 was returned because too many queries occurred in the last time. "
|
||||
"Please do not use Instagram in your browser or run multiple instances of Instaloader "
|
||||
"in parallel.")
|
||||
print(textwrap.fill(text_for_429), file=sys.stderr)
|
||||
current_time = time.monotonic()
|
||||
waittime = self.query_waittime(query_type, current_time, True)
|
||||
assert waittime >= 0
|
||||
self._dump_query_timestamps(current_time, query_type)
|
||||
text_for_429 = ("Instagram responded with HTTP error \"429 - Too Many Requests\". Please do not run multiple "
|
||||
"instances of Instaloader in parallel or within short sequence. Also, do not use any Instagram "
|
||||
"App while Instaloader is running.")
|
||||
self._context.error(textwrap.fill(text_for_429), repeat_at_end=False)
|
||||
if waittime > 1.5:
|
||||
self._context.error("The request will be retried in {} seconds, at {:%H:%M}."
|
||||
.format(round(waittime), datetime.now() + timedelta(seconds=waittime)),
|
||||
repeat_at_end=False)
|
||||
if waittime > 15:
|
||||
self._context.log("The request will be retried in {} seconds, at {:%H:%M}."
|
||||
.format(round(waittime), datetime.now() + timedelta(seconds=waittime)))
|
||||
self._dump_query_timestamps(current_time)
|
||||
if waittime > 0:
|
||||
self.sleep(waittime)
|
||||
|
||||
@@ -81,13 +81,11 @@ class NodeIterator(Iterator[T]):
|
||||
self._node_wrapper = node_wrapper
|
||||
self._query_variables = query_variables if query_variables is not None else {}
|
||||
self._query_referer = query_referer
|
||||
self._data = first_data
|
||||
self._page_index = 0
|
||||
self._total_index = 0
|
||||
if first_data is not None:
|
||||
self._data = first_data
|
||||
self._best_before = datetime.now() + NodeIterator._shelf_life
|
||||
else:
|
||||
self._data = self._query()
|
||||
self._best_before = (None if first_data is None else
|
||||
datetime.now() + NodeIterator._shelf_life)
|
||||
|
||||
def _query(self, after: Optional[str] = None) -> Dict:
|
||||
pagination_variables = {'first': NodeIterator._graphql_page_length} # type: Dict[str, Any]
|
||||
@@ -115,6 +113,8 @@ class NodeIterator(Iterator[T]):
|
||||
return self
|
||||
|
||||
def __next__(self) -> T:
|
||||
if self._data is None:
|
||||
self._data = self._query()
|
||||
if self._page_index < len(self._data['edges']):
|
||||
node = self._data['edges'][self._page_index]['node']
|
||||
page_index, total_index = self._page_index, self._total_index
|
||||
@@ -193,12 +193,8 @@ class NodeIterator(Iterator[T]):
|
||||
self._query_referer != frozen.query_referer or
|
||||
self._context.username != frozen.context_username):
|
||||
raise InvalidArgumentException("Mismatching resume information.")
|
||||
if not frozen.best_before:
|
||||
raise InvalidArgumentException("\"best before\" date missing.")
|
||||
if frozen.remaining_data is None:
|
||||
raise InvalidArgumentException("\"remaining_data\" missing.")
|
||||
self._total_index = frozen.total_index
|
||||
self._best_before = datetime.fromtimestamp(frozen.best_before)
|
||||
self._best_before = datetime.fromtimestamp(frozen.best_before) if frozen.best_before else None
|
||||
self._data = frozen.remaining_data
|
||||
|
||||
|
||||
@@ -263,8 +259,7 @@ def resumable_iteration(context: InstaloaderContext,
|
||||
try:
|
||||
yield is_resuming, start_index
|
||||
except KeyboardInterrupt:
|
||||
if os.path.dirname(resume_file_path):
|
||||
os.makedirs(os.path.dirname(resume_file_path), exist_ok=True)
|
||||
os.makedirs(os.path.dirname(resume_file_path), exist_ok=True)
|
||||
save(iterator.freeze(), resume_file_path)
|
||||
context.log("\nSaved resume information to {}.".format(resume_file_path))
|
||||
raise
|
||||
|
||||
+53
-114
@@ -147,6 +147,10 @@ class Post:
|
||||
)
|
||||
self._full_metadata_dict = pic_json['data']['shortcode_media']
|
||||
if self._full_metadata_dict is None:
|
||||
# issue #449
|
||||
self._context.error("Fetching Post metadata failed (issue #449). "
|
||||
"The following data has been returned:\n"
|
||||
+ json.dumps(pic_json['entry_data'], indent=2))
|
||||
raise BadResponseException("Fetching Post metadata failed.")
|
||||
if self.shortcode != self._full_metadata_dict['shortcode']:
|
||||
self._node.update(self._full_metadata_dict)
|
||||
@@ -192,7 +196,7 @@ class Post:
|
||||
# is contained.
|
||||
# Note that we cannot use Profile.from_id() here since that would lead us into a recursion.
|
||||
owner_struct = self._full_metadata['owner']
|
||||
self._owner_profile = Profile(self._context, owner_struct)
|
||||
self._owner_profile = Profile.from_node(self._context, owner_struct)
|
||||
return self._owner_profile
|
||||
|
||||
@property
|
||||
@@ -203,13 +207,7 @@ class Post:
|
||||
@property
|
||||
def owner_id(self) -> int:
|
||||
"""The ID of the Post's owner."""
|
||||
# The ID may already be available, e.g. if the post instance was created
|
||||
# from an `hashtag.get_posts()` iterator, so no need to make another
|
||||
# http request.
|
||||
if 'owner' in self._node and 'id' in self._node['owner']:
|
||||
return self._node['owner']['id']
|
||||
else:
|
||||
return self.owner_profile.userid
|
||||
return self.owner_profile.userid
|
||||
|
||||
@property
|
||||
def date_local(self) -> datetime:
|
||||
@@ -252,49 +250,26 @@ class Post:
|
||||
"""Type of post, GraphImage, GraphVideo or GraphSidecar"""
|
||||
return self._field('__typename')
|
||||
|
||||
@property
|
||||
def mediacount(self) -> int:
|
||||
"""
|
||||
The number of media in a sidecar Post, or 1 if the Post it not a sidecar.
|
||||
|
||||
.. versionadded:: 4.6
|
||||
"""
|
||||
if self.typename == 'GraphSidecar':
|
||||
edges = self._field('edge_sidecar_to_children', 'edges')
|
||||
return len(edges)
|
||||
return 1
|
||||
|
||||
def get_sidecar_nodes(self, start=0, end=-1) -> Iterator[PostSidecarNode]:
|
||||
"""
|
||||
Sidecar nodes of a Post with typename==GraphSidecar.
|
||||
|
||||
.. versionchanged:: 4.6
|
||||
Added parameters *start* and *end* to specify a slice of sidecar media.
|
||||
"""
|
||||
def get_sidecar_nodes(self) -> Iterator[PostSidecarNode]:
|
||||
"""Sidecar nodes of a Post with typename==GraphSidecar."""
|
||||
if self.typename == 'GraphSidecar':
|
||||
edges = self._field('edge_sidecar_to_children', 'edges')
|
||||
if any(edge['node']['is_video'] for edge in edges):
|
||||
# video_url is only present in full metadata, issue #558.
|
||||
edges = self._full_metadata['edge_sidecar_to_children']['edges']
|
||||
if end < 0:
|
||||
end = len(edges)-1
|
||||
if start < 0:
|
||||
start = len(edges)-1
|
||||
for idx, edge in enumerate(edges):
|
||||
if start <= idx <= end:
|
||||
node = edge['node']
|
||||
is_video = node['is_video']
|
||||
display_url = node['display_url']
|
||||
if not is_video and self._context.is_logged_in:
|
||||
try:
|
||||
carousel_media = self._iphone_struct['carousel_media']
|
||||
orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
|
||||
display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
|
||||
except (InstaloaderException, KeyError, IndexError) as err:
|
||||
self._context.error('{} Unable to fetch high quality image version of {}.'.format(
|
||||
err, self))
|
||||
yield PostSidecarNode(is_video=is_video, display_url=display_url,
|
||||
video_url=node['video_url'] if is_video else None)
|
||||
node = edge['node']
|
||||
is_video = node['is_video']
|
||||
display_url = node['display_url']
|
||||
if not is_video and self._context.is_logged_in:
|
||||
try:
|
||||
carousel_media = self._iphone_struct['carousel_media']
|
||||
orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
|
||||
display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
|
||||
except (InstaloaderException, KeyError, IndexError) as err:
|
||||
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
|
||||
yield PostSidecarNode(is_video=is_video, display_url=display_url,
|
||||
video_url=node['video_url'] if is_video else None)
|
||||
|
||||
@property
|
||||
def caption(self) -> Optional[str]:
|
||||
@@ -353,12 +328,6 @@ class Post:
|
||||
def video_url(self) -> Optional[str]:
|
||||
"""URL of the video, or None."""
|
||||
if self.is_video:
|
||||
if self._context.is_logged_in:
|
||||
try:
|
||||
url = self._iphone_struct['video_versions'][0]['url']
|
||||
return url
|
||||
except (InstaloaderException, KeyError, IndexError) as err:
|
||||
self._context.error('{} Unable to fetch high quality video version of {}.'.format(err, self))
|
||||
return self._field('video_url')
|
||||
return None
|
||||
|
||||
@@ -418,7 +387,7 @@ class Post:
|
||||
return PostCommentAnswer(id=int(node['id']),
|
||||
created_at_utc=datetime.utcfromtimestamp(node['created_at']),
|
||||
text=node['text'],
|
||||
owner=Profile(self._context, node['owner']),
|
||||
owner=Profile.from_node(self._context, node['owner']),
|
||||
likes_count=node.get('edge_liked_by', {}).get('count', 0))
|
||||
|
||||
def _postcommentanswers(node):
|
||||
@@ -466,27 +435,20 @@ class Post:
|
||||
)
|
||||
|
||||
def get_likes(self) -> Iterator['Profile']:
|
||||
"""
|
||||
Iterate over all likes of the post. A :class:`Profile` instance of each likee is yielded.
|
||||
|
||||
.. versionchanged:: 4.5.4
|
||||
Require being logged in (as required by Instagram).
|
||||
"""
|
||||
if not self._context.is_logged_in:
|
||||
raise LoginRequiredException("--login required to access likes of a post.")
|
||||
"""Iterate over all likes of the post. A :class:`Profile` instance of each likee is yielded."""
|
||||
if self.likes == 0:
|
||||
# Avoid doing additional requests if there are no comments
|
||||
return
|
||||
likes_edges = self._field('edge_media_preview_like', 'edges')
|
||||
if self.likes == len(likes_edges):
|
||||
# If the Post's metadata already contains all likes, don't do GraphQL requests to obtain them
|
||||
yield from (Profile(self._context, like['node']) for like in likes_edges)
|
||||
yield from (Profile.from_node(self._context, like['node']) for like in likes_edges)
|
||||
return
|
||||
yield from NodeIterator(
|
||||
self._context,
|
||||
'1cb6ec562846122743b61e492c85999f',
|
||||
lambda d: d['data']['shortcode_media']['edge_liked_by'],
|
||||
lambda n: Profile(self._context, n),
|
||||
lambda n: Profile.from_node(self._context, n),
|
||||
{'shortcode': self.shortcode},
|
||||
'https://www.instagram.com/p/{0}/'.format(self.shortcode),
|
||||
)
|
||||
@@ -512,7 +474,7 @@ class Post:
|
||||
.. versionadded:: 4.4
|
||||
"""
|
||||
return ([] if not self.is_sponsored else
|
||||
[Profile(self._context, edge['node']['sponsor']) for edge in
|
||||
[Profile.from_node(self._context, edge['node']['sponsor']) for edge in
|
||||
self._field('edge_media_to_sponsor_user', 'edges')])
|
||||
|
||||
@property
|
||||
@@ -575,6 +537,17 @@ class Profile:
|
||||
# if loaded from JSON with load_structure_from_file()
|
||||
self._iphone_struct_ = node['iphone_struct']
|
||||
|
||||
@classmethod
|
||||
def from_node(cls, context: InstaloaderContext, node: Dict[str, Any]) -> 'Profile':
|
||||
assert 'username' in node
|
||||
if node['username'] in context.profile_name_cache:
|
||||
return context.profile_name_cache[node['username']]
|
||||
profile = cls(context, node)
|
||||
if 'id' in node:
|
||||
context.profile_id_cache[node['id']] = profile
|
||||
context.profile_name_cache[profile.username] = profile
|
||||
return profile
|
||||
|
||||
@classmethod
|
||||
def from_username(cls, context: InstaloaderContext, username: str):
|
||||
"""Create a Profile instance from a given username, raise exception if it does not exist.
|
||||
@@ -586,8 +559,11 @@ class Profile:
|
||||
:raises: :class:`ProfileNotExistsException`
|
||||
"""
|
||||
# pylint:disable=protected-access
|
||||
profile = cls(context, {'username': username.lower()})
|
||||
profile._obtain_metadata() # to raise ProfileNotExistsException now in case username is invalid
|
||||
if username.lower() in context.profile_name_cache:
|
||||
return context.profile_name_cache[username.lower()]
|
||||
profile = cls.from_node(context, {'username': username.lower()})
|
||||
profile._obtain_metadata() # to raise ProfileNotExistException now in case username is invalid
|
||||
context.profile_id_cache[profile.userid] = profile
|
||||
return profile
|
||||
|
||||
@classmethod
|
||||
@@ -610,23 +586,10 @@ class Profile:
|
||||
'include_highlight_reels': False},
|
||||
rhx_gis=context.root_rhx_gis)['data']['user']
|
||||
if data:
|
||||
profile = cls(context, data['reel']['owner'])
|
||||
return cls.from_node(context, data['reel']['owner'])
|
||||
else:
|
||||
raise ProfileNotExistsException("No profile found, the user may have blocked you (ID: " +
|
||||
str(profile_id) + ").")
|
||||
context.profile_id_cache[profile_id] = profile
|
||||
return profile
|
||||
|
||||
@classmethod
|
||||
def own_profile(cls, context: InstaloaderContext):
|
||||
"""Return own profile if logged-in.
|
||||
|
||||
:param context: :attr:`Instaloader.context`
|
||||
|
||||
.. versionadded:: 4.5.2"""
|
||||
if not context.is_logged_in:
|
||||
raise LoginRequiredException("--login required to access own profile.")
|
||||
return cls(context, context.graphql_query("d6f4427fbe92d846298cf93df0b937d3", {})["data"]["user"])
|
||||
|
||||
def _asdict(self):
|
||||
json_node = self._node.copy()
|
||||
@@ -851,6 +814,7 @@ class Profile:
|
||||
if self.username != self._context.username:
|
||||
raise LoginRequiredException("--login={} required to get that profile's saved posts.".format(self.username))
|
||||
|
||||
self._obtain_metadata()
|
||||
return NodeIterator(
|
||||
self._context,
|
||||
'f883d95537fbcd400f466f63d42bd8a1',
|
||||
@@ -858,6 +822,7 @@ class Profile:
|
||||
lambda n: Post(self._context, n),
|
||||
{'id': self.userid},
|
||||
'https://www.instagram.com/{0}/'.format(self.username),
|
||||
self._metadata('edge_saved_media'),
|
||||
)
|
||||
|
||||
def get_tagged_posts(self) -> NodeIterator[Post]:
|
||||
@@ -907,7 +872,7 @@ class Profile:
|
||||
self._context,
|
||||
'37479f2b8209594dde7facb0d904896a',
|
||||
lambda d: d['data']['user']['edge_followed_by'],
|
||||
lambda n: Profile(self._context, n),
|
||||
lambda n: Profile.from_node(self._context, n),
|
||||
{'id': str(self.userid)},
|
||||
'https://www.instagram.com/{0}/'.format(self.username),
|
||||
)
|
||||
@@ -926,7 +891,7 @@ class Profile:
|
||||
self._context,
|
||||
'58712303d941c6855d4e888c5f0cd22f',
|
||||
lambda d: d['data']['user']['edge_follow'],
|
||||
lambda n: Profile(self._context, n),
|
||||
lambda n: Profile.from_node(self._context, n),
|
||||
{'id': str(self.userid)},
|
||||
'https://www.instagram.com/{0}/'.format(self.username),
|
||||
)
|
||||
@@ -941,7 +906,7 @@ class Profile:
|
||||
if not self._context.is_logged_in:
|
||||
raise LoginRequiredException("--login required to get a profile's similar accounts.")
|
||||
self._obtain_metadata()
|
||||
yield from (Profile(self._context, edge["node"]) for edge in
|
||||
yield from (Profile.from_node(self._context, edge["node"]) for edge in
|
||||
self._context.graphql_query("ad99dd9d3646cc3c0dda65debcd266a7",
|
||||
{"user_id": str(self.userid), "include_chaining": True},
|
||||
"https://www.instagram.com/{0}/"
|
||||
@@ -963,17 +928,11 @@ class StoryItem:
|
||||
self._context = context
|
||||
self._node = node
|
||||
self._owner_profile = owner_profile
|
||||
self._iphone_struct_ = None
|
||||
if 'iphone_struct' in node:
|
||||
# if loaded from JSON with load_structure_from_file()
|
||||
self._iphone_struct_ = node['iphone_struct']
|
||||
|
||||
def _asdict(self):
|
||||
node = self._node
|
||||
if self._owner_profile:
|
||||
node['owner'] = self._owner_profile._asdict()
|
||||
if self._iphone_struct_:
|
||||
node['iphone_struct'] = self._iphone_struct_
|
||||
return node
|
||||
|
||||
@property
|
||||
@@ -998,15 +957,6 @@ class StoryItem:
|
||||
def __hash__(self) -> int:
|
||||
return hash(self.mediaid)
|
||||
|
||||
@property
|
||||
def _iphone_struct(self) -> Dict[str, Any]:
|
||||
if not self._context.is_logged_in:
|
||||
raise LoginRequiredException("--login required to access iPhone media info endpoint.")
|
||||
if not self._iphone_struct_:
|
||||
data = self._context.get_iphone_json(path='api/v1/media/{}/info/'.format(self.mediaid), params={})
|
||||
self._iphone_struct_ = data['items'][0]
|
||||
return self._iphone_struct_
|
||||
|
||||
@property
|
||||
def owner_profile(self) -> Profile:
|
||||
""":class:`Profile` instance of the story item's owner."""
|
||||
@@ -1058,13 +1008,6 @@ class StoryItem:
|
||||
@property
|
||||
def url(self) -> str:
|
||||
"""URL of the picture / video thumbnail of the StoryItem"""
|
||||
if self.typename == "GraphStoryImage" and self._context.is_logged_in:
|
||||
try:
|
||||
orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
|
||||
url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
|
||||
return url
|
||||
except (InstaloaderException, KeyError, IndexError) as err:
|
||||
self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
|
||||
return self._node['display_resources'][-1]['src']
|
||||
|
||||
@property
|
||||
@@ -1137,14 +1080,14 @@ class Story:
|
||||
|
||||
@property
|
||||
def last_seen_local(self) -> Optional[datetime]:
|
||||
"""Timestamp of the most recent StoryItem that has been watched or None (local time zone)."""
|
||||
"""Timestamp when the story has last been watched or None (local time zone)."""
|
||||
if self._node['seen']:
|
||||
return datetime.fromtimestamp(self._node['seen'])
|
||||
return None
|
||||
|
||||
@property
|
||||
def last_seen_utc(self) -> Optional[datetime]:
|
||||
"""Timestamp of the most recent StoryItem that has been watched or None (UTC)."""
|
||||
"""Timestamp when the story has last been watched or None (UTC)."""
|
||||
if self._node['seen']:
|
||||
return datetime.utcfromtimestamp(self._node['seen'])
|
||||
return None
|
||||
@@ -1168,7 +1111,7 @@ class Story:
|
||||
def owner_profile(self) -> Profile:
|
||||
""":class:`Profile` instance of the story owner."""
|
||||
if not self._owner_profile:
|
||||
self._owner_profile = Profile(self._context, self._node['user'])
|
||||
self._owner_profile = Profile.from_node(self._context, self._node['user'])
|
||||
return self._owner_profile
|
||||
|
||||
@property
|
||||
@@ -1226,7 +1169,7 @@ class Highlight(Story):
|
||||
def owner_profile(self) -> Profile:
|
||||
""":class:`Profile` instance of the highlights' owner."""
|
||||
if not self._owner_profile:
|
||||
self._owner_profile = Profile(self._context, self._node['owner'])
|
||||
self._owner_profile = Profile.from_node(self._context, self._node['owner'])
|
||||
return self._owner_profile
|
||||
|
||||
@property
|
||||
@@ -1410,13 +1353,9 @@ class Hashtag:
|
||||
next_other = next(other_posts, None)
|
||||
while next_top is not None or next_other is not None:
|
||||
if next_other is None:
|
||||
assert next_top is not None
|
||||
yield next_top
|
||||
yield from sorted_top_posts
|
||||
break
|
||||
if next_top is None:
|
||||
assert next_other is not None
|
||||
yield next_other
|
||||
yield from other_posts
|
||||
break
|
||||
if next_top == next_other:
|
||||
@@ -1461,7 +1400,7 @@ class TopSearchResults:
|
||||
user_node = user['user']
|
||||
if 'pk' in user_node:
|
||||
user_node['id'] = user_node['pk']
|
||||
yield Profile(self._context, user_node)
|
||||
yield Profile.from_node(self._context, user_node)
|
||||
|
||||
def get_prefixed_usernames(self) -> Iterator[str]:
|
||||
"""
|
||||
@@ -1554,7 +1493,7 @@ def load_structure_from_file(context: InstaloaderContext, filename: str) -> Json
|
||||
if node_type == "Post":
|
||||
return Post(context, json_structure['node'])
|
||||
elif node_type == "Profile":
|
||||
return Profile(context, json_structure['node'])
|
||||
return Profile.from_node(context, json_structure['node'])
|
||||
elif node_type == "StoryItem":
|
||||
return StoryItem(context, json_structure['node'])
|
||||
elif node_type == "Hashtag":
|
||||
|
||||
@@ -38,7 +38,6 @@ setup(
|
||||
name='instaloader',
|
||||
version=get_version(),
|
||||
packages=['instaloader'],
|
||||
package_data={'instaloader': ['py.typed']},
|
||||
url='https://instaloader.github.io/',
|
||||
license='MIT',
|
||||
author='Alexander Graf, André Koch-Kramer',
|
||||
@@ -49,7 +48,7 @@ setup(
|
||||
install_requires=requirements,
|
||||
python_requires='>=3.5',
|
||||
entry_points={'console_scripts': ['instaloader=instaloader.__main__:main']},
|
||||
zip_safe=False,
|
||||
zip_safe=True,
|
||||
keywords=keywords,
|
||||
classifiers=[
|
||||
'Development Status :: 5 - Production/Stable',
|
||||
|
||||
Reference in New Issue
Block a user