Compare commits

..
64 Commits
Author SHA1 Message Date
André Koch-Kramer 52492456ed Release of version 1.3 2017-04-22 18:01:54 +02:00
Alexander Graf e0924e8d08 Fix --geotags 2017-04-22 17:54:21 +02:00
Alexander Graf 655dbb552d Further clarify meaning of --count 2017-04-22 17:34:49 +02:00
André Koch-Kramer fdb8e94c64 Fixed ":feed-liked" functionality 2017-04-22 17:26:48 +02:00
Alexander Graf b3c83f420c --count to limit posts at #hashtag and :feed-* 2017-04-22 17:21:02 +02:00
Alexander Graf df00e2b03a Minor enhancements to README 2017-04-22 12:11:56 +02:00
Alexander Graf b60f0b4836 README: Document all of Instaloader's features 2017-04-22 11:58:59 +02:00
Alexander Graf 35e71192c5 travis: Enable more Pylint inspections 2017-04-22 11:58:12 +02:00
André Koch-Kramer 2106c2d5f6 Satisfy pylint after their update 2017-04-22 11:13:10 +02:00
André Koch-Kramer 0e943189e5 Parse graphql structure for sidecars 2017-04-22 10:50:12 +02:00
André Koch-Kramer 8e77a1c125 Fixed download_node() 2017-04-22 01:39:52 +02:00
André Koch-Kramer 361445519a Adapt new graphql layout used by Instagram
Fixes #19.
2017-04-21 18:10:41 +02:00
Alexander Graf 6b345e1f52 Adapt video downloading to new format
This should the video downloading issue reported at #18.
2017-04-20 09:17:59 +02:00
Alexander Graf c841bca81b Silence new pylint warnings 2017-04-20 09:17:32 +02:00
Alexander Graf 3e1360160d Download pictures with #hashtag
Instaloader is now capable of downloading all pictures associated with
one #hashtag with:
instaloader #hashtag

This implements the feature requested with #18.
2017-04-17 12:16:22 +02:00
Alexander Graf a7d1c5bbb0 Make check_id() exceptions Non-fatal
The check_id() step, including the get_username_by_id(), which is used
to determine whether a profile's account name has changed since the last
download using it's unique id when Instaloader is operating logged-in,
is actually optional and should not cause termination in any case.
2017-04-10 21:05:58 +02:00
Alexander Graf dc748a0541 Download all pictures of Sidecar nodes 2017-03-25 21:08:54 +01:00
Alexander Graf 72c647829a Don't fail if --sessionfile does not contain '/' 2017-03-21 14:58:13 +01:00
Alexander Graf 0a039b9c8c Release of version 1.2.2 2017-03-19 14:52:45 +01:00
Alexander Graf b1014bfbfd README: Mention how to install from our Git repo 2017-03-19 14:42:02 +01:00
Alexander Graf d246268630 Retry download anonymously if profile not exists
In case you are blocked by a public profile which you intend to
download, the server responds as if the profile would not exist. Now in
this case, we retry the download without using an anonymous session.
2017-03-19 12:52:07 +01:00
Alexander Graf 23a0e32e8e Clarify --login is required for download followees 2017-03-19 12:51:20 +01:00
Alexander Graf 597e5977b2 Release of version 1.2.1 2017-02-13 10:25:54 +01:00
Alexander Graf 00f6f47fa9 fix get_id_by_username() 2017-02-13 10:20:45 +01:00
Alexander Graf 61711f3b6d Fix Typos in README.rst 2017-02-13 10:03:34 +01:00
Alexander Graf be477e8a88 Fix very minor packaging issues
- State in README.rst and setup.py metainfo that we require Python>=3.5

- Let Travis-CI test against newer versions of Python

- Let instaloader --help show where to report issues
2017-02-13 09:57:03 +01:00
Alexander Graf 02509d3c40 Fix downloading (set max_id only if not zero)
This should fix #17.
2017-02-13 09:50:20 +01:00
Alexander Graf 6e3c13b5d3 Fix typo in README.rst 2017-01-06 10:33:38 +01:00
Alexander Graf 84c2a823c4 fix typing 2016-12-22 16:05:25 +01:00
Alexander Graf f2660f7c02 Require Python >= 3.5 2016-12-22 13:27:08 +01:00
Alexander Graf 73f85b3316 Adjust travis yml 2016-12-22 13:24:50 +01:00
Alexander Graf 86f8b2f018 Annotate all types 2016-12-22 13:20:41 +01:00
André Koch-Kramer 13ebcb782e Release of version 1.2 2016-09-22 19:04:25 +02:00
André Koch-Kramer 50faad0a04 Minimal updated description 2016-09-22 19:02:26 +02:00
André Koch-Kramer 98c2847afd Implemented feature: store geotags/locations 2016-09-22 18:28:13 +02:00
Alexander Graf be8546812a Use reStructuredText as file format for README
rst seems to be cooler than md, and is the only format accepted by pypi,
and is accepted by github as well.

There is an intro to rst at:
http://docutils.sourceforge.net/docs/user/rst/quickref.html
2016-09-19 19:45:15 +02:00
Alexander Graf 1d506b5f95 Minor documentation improvements 2016-09-19 19:26:59 +02:00
Alexander Graf 1036fe45b5 README.md: Note pip3 as way to install 2016-09-18 17:44:27 +02:00
Alexander Graf bbcebb25eb setup.py: print message if python version is too old 2016-09-18 17:43:57 +02:00
Alexander Graf 8408183b79 minor, irrelevant setup.py fix 2016-09-18 17:27:10 +02:00
Alexander Graf 66cd9695df Release of version 1.1 2016-09-18 17:02:53 +02:00
Alexander Graf 508c629d2b Equalify summary in *.py and README.md 2016-09-18 16:41:43 +02:00
Alexander Graf 1ff6dd9d30 Mini refactoring and docstrings
Closes #12.
2016-09-18 16:35:25 +02:00
Alexander Graf d5c13b1295 Globally disable pylint too-many-arguments warning 2016-09-18 15:43:24 +02:00
Alexander Graf 3ac8ffbc84 Reduce code duplication introducing download_node 2016-09-18 15:41:12 +02:00
Alexander Graf 7f500d775e minor README.md enhancements 2016-09-18 14:51:35 +02:00
Alexander Graf c2957e389f Have setuptools setup.py for serious distribution
This is a) cooler and b) a requirement for deploying it on PyPI.

It removes need of __all__ shit (which is hard to keep updated), and
allows installing instaloader easily as a global module and executable.
Additionally it removes __init__.py.
2016-09-18 14:43:12 +02:00
Alexander Graf 70c91e000e Targets :feed-all and :feed-liked to load feed
Closes #14.
2016-09-17 20:53:03 +02:00
Alexander Graf 9cd93c9414 Have function providing access to user's feed
Closes #13.
2016-09-16 23:24:28 +02:00
Alexander Graf 5dc9be47cb Make instaloader usable as package
This commit allows doing `import instaloader` when instaloader is
located in a subdirectory "instaloader". This makes it easier to use
instaloader e.g. when it is imported using git submodules feature.
2016-08-18 10:04:54 +02:00
André Koch-Kramer 0088ee5e9e Added handling of UnicodeEncodeError
- Only try to print captions if possible
- Added option '--shorter-output' to disable output of captions
2016-08-04 19:36:36 +02:00
Alexander Graf 0678a8118a Properly escape \ in regex string 2016-08-03 20:29:36 +02:00
André Koch-Kramer b71179371d Do not sleep when --no-sleep is given 2016-08-03 20:25:16 +02:00
Alexander Graf 05104b7438 Have better error handling when working on files
try ... except FileNotFoundError is better than os.path.isfile.
2016-08-03 13:51:25 +02:00
Alexander Graf ce8bdb18e0 Have newline in id files
This is better. A line in a textfile must terminate with a \n character.
2016-08-03 13:50:47 +02:00
André Koch-Kramer 77d0d272fc Implementation of get_id_by_username()
+ updated README.md
2016-08-02 21:27:39 +02:00
André Koch-Kramer e6ca038e25 Store ID of profile to find it after name change
Closes #9.
2016-08-02 18:54:30 +02:00
André Koch-Kramer 4fb574253e change import order to satisfy pylint 2016-08-02 18:41:57 +02:00
André Koch-Kramer 584c3cc76c Release of version 1.0.1 2016-08-01 18:16:47 +02:00
André Koch-Kramer dd383d8887 Fix Windows Unicode Problems
Closes #11.

instaloader now tries to use win-unicode-console when recognizing
windows.
2016-08-01 18:10:35 +02:00
Alexander Graf 82ef71836f Minor UI improvements
Improvement to help text as well as trivial improvements to exception
texts.
2016-07-29 18:03:19 +02:00
Alexander Graf 2f53debffa Disable pylint warning for download() 2016-07-29 18:02:36 +02:00
Alexander Graf ebff2205b1 @<profile> to download all followees of profile
This closes #10.
2016-07-29 17:59:07 +02:00
André Koch-Kramer 8524c8a2f2 Output captions while downloading profiles 2016-07-29 16:53:18 +02:00
6 changed files with 766 additions and 187 deletions
+4
View File
@@ -0,0 +1,4 @@
__pycache__/
build/
dist/
instaloader.egg-info/
+4 -3
View File
@@ -1,11 +1,12 @@
language: python
python:
- "3.3"
- "3.4"
- "3.5"
- "3.5-dev"
- "3.6"
- "3.6-dev"
- "3.7-dev"
- "nightly"
install:
- pip install pylint requests
script:
- python3 -m pylint -r n -d bad-whitespace,bad-continuation,missing-docstring,anomalous-backslash-in-string,multiple-imports,dangerous-default-value,locally-disabled instaloader
- python3 -m pylint -r n -d bad-whitespace,bad-continuation,missing-docstring,multiple-imports,too-many-arguments,locally-disabled,line-too-long instaloader
-67
View File
@@ -1,67 +0,0 @@
# instaloader
Simple downloader to fetch all Instagram pictures and captions from a given profile.
## Usage
Ensure having [Python](https://www.python.org/) (at least version 3.3) and
[python3-requests](https://pypi.python.org/pypi/requests/) installed.
After having [downloaded instaloader.py](https://github.com/Thammus/instaloader/releases), you invoke it with
```
./instaloader.py profile [profile ...]
```
where `profile` is the name of a profile you want to download. Instead of only one profile, you may
also specify a list of profiles.
To later update your local copy of that profile, you may run
```
./instaloader.py --fast-update profile [profile ...]
```
When `--fast-update` is given, instaloder terminates when arriving at the first already-downloaded
picture.
Instaloader can also be used to **download private profiles**. To do so, invoke it with
```
./instaloader.py --login=your_username profile [profile ...]
```
When invoked like this, it also **stores the session cookies** in a file in `/tmp`, which will be
reused later when `--login` is given. So you can download private profiles **non-interactively**
when you already have a valid session cookies file.
The `--quiet` option makes it also **suitable as a cron job**.
To get a list of other helpful flags, run `./instaloader.py --help`.
## Usage as library
You may also use parts of instaloader as library to do other interesting things.
For example, to get a list of all followers of a profile as well as their follower count, do
```python
import instaloader
# login
session = instaloader.get_logged_in_session(USERNAME)
# get followees
followees = instaloader.get_followees(PROFILE, session)
for f in followees:
print("%i\t%s\t%s" % (f['follower_count'], f['username'], f['full_name']))
```
Then, you may download all pictures of all followees with
```python
for f in followees:
try:
instaloader.download(f['username'], session)
except instaloader.NonfatalException:
pass
```
`get_followees()` also returns unique IDs for all loaded followees. These IDs stay unchanged even
if a user changes his/her username. To get the current username of a profile, given this unique ID
`get_username_by_id()` can be used. For example:
```python
instaloader.get_username_by_id(session, followees[0]['id'])
```
+185
View File
@@ -0,0 +1,185 @@
Instaloader
===========
Tool to automatically download pictures (or videos) of given profiles
from Instagram.
Installation
------------
Instaloader is written in Python, thus ensure having
`Python <https://www.python.org/>`__ (at least version 3.5) installed.
If you intend to use this tool under Windows, it is recommended
to install
`win-unicode-console <https://pypi.python.org/pypi/win_unicode_console>`__.
If you have `pip <https://pypi.python.org/pypi/pip>`__ installed, you
may install Instaloader using
::
pip3 install instaloader
Alternatively, to get the most current version of Instaloader from our
`Git repository <https://github.com/Thammus/instaloader>`__:
::
pip3 install git+https://github.com/Thammus/instaloader
(pass ``--upgrade`` to upgrade if Instaloader is already installed)
Instaloader requires
`requests <https://pypi.python.org/pypi/requests>`__, which
will be installed automatically, if it is not already installed.
How to automatically download pictures from Instagram
-----------------------------------------------------
To **download all pictures and videos of a profile**, as well as the
**profile picture**, do
::
instaloader profile [profile ...]
where ``profile`` is the name of a profile you want to download. Instead
of only one profile, you may also specify a list of profiles.
To later **update your local copy** of that profiles, you may run
::
instaloader --fast-update profile [profile ...]
When ``--fast-update`` is given, Instaloader stops when arriving at
the first already-downloaded picture.
Instaloader can also be used to **download private profiles**. To do so,
invoke it with
::
instaloader --login=your_username profile [profile ...]
When invoked like this, it also **stores the session cookies** in a file
in your temporary directory, which will be reused later when ``--login`` is given. So
you can download private profiles **non-interactively** when you already
have a valid session cookie file.
You may also download
**the most recent pictures by hashtag**:
::
instaloader "#hashtag"
If you want to **download all followees of a given profile**, call
::
instaloader --login=your_username @profile
To **download all the pictures from your feed which you have liked**, call
::
instaloader --login=your_username :feed-liked
or to **download all pictures from your feed**:
::
instaloader --login=your_username :feed-all
Advanced Options
----------------
The following flags can be given to Instaloader to specify how profiles should
be downloaded.
--fast-update Stop when encountering the first already-downloaded post
of a profile.
--profile-pic-only Only download profile pictures. Without this flag, the current
profile picture and all the profile's posts are downloaded.
--skip-videos Skip posts which are videos.
--geotags Also **download geotags** and store Google Maps links in
separate textfiles.
--count COUNT If used with ``#hashtag``, ``:feed-all`` or
``:feed-liked``: Do not attempt to download more than COUNT
posts.
--quiet Do not output any messages except warnings and errors. This
option makes Instaloader **suitable as a cron job**.
--no-sleep Normally, Instaloader waits a few seconds between requests
to the Instagram servers. This flag inhibits this behavior.
--password PASSWORD If used with ``--login``, use parameter as password if no
valid session file is found, instead of asking
interactively.
--sessionfile FILE Specify an alternative place for loading and storing the
session cookies. Without this flag, they are stored in a path
within your temporary directory, encoding your local
username and your instagram profile name.
To get a list of all flags, run ``instaloader --help``.
Usage as Python module
----------------------
You may also use parts of Instaloader as library to do other interesting
things.
For example, to get a list of all followees of a profile as well as
their follower count, do
.. code:: python
import instaloader
# login
session = instaloader.get_logged_in_session(USERNAME)
# get followees
followees = instaloader.get_followees(PROFILE, session)
for f in followees:
print("%i\t%s\t%s" % (f['follower_count'], f['username'], f['full_name']))
Then, you may download all pictures of all followees with
.. code:: python
for f in followees:
try:
instaloader.download(f['username'], session)
except instaloader.NonfatalException:
pass
You could also download your last 20 liked pics with
.. code:: python
instaloader.download_feed_pics(session, max_count=20, fast_update=True,
filter_func=lambda node:
not node["likes"]["viewer_has_liked"] if "likes" in node else not node["viewer_has_liked"])
To download the last 20 pictures with hashtag #cat, do
.. code:: python
instaloader.download_hashtag('cat', session=instaloader.get_anonymous_session(), max_count=20)
Each Instagram profile has its own unique ID which stays unmodified even
if a user changes his/her username. To get said ID, given the profile's
name, you may call
.. code:: python
instaloader.get_id_by_username(PROFILE_NAME)
``get_followees()`` also returns unique IDs for all loaded followees. To
get the current username of a profile, given this unique ID
``get_username_by_id()`` can be used. For example:
.. code:: python
instaloader.get_username_by_id(session, followees[0]['id'])
+535 -117
View File
@@ -1,9 +1,34 @@
#!/usr/bin/env python3
"""Tool to download pictures (or videos) and captions from Instagram, from a given set
of profiles (even if private), from your feed or from all followees of a given profile."""
import re, json, datetime, shutil, os, time, random, sys, pickle, getpass, tempfile
from argparse import ArgumentParser
from io import BytesIO
from numbers import Real
from typing import List, Optional, Any, Dict, Callable
import requests, requests.utils
# To get version from setup.py for instaloader --version
import pkg_resources
try:
# pylint:disable=no-member
__version__ = pkg_resources.get_distribution('instaloader').version
except pkg_resources.DistributionNotFound:
__version__ = 'Run ./setup.py --version'
try:
# pylint:disable=wrong-import-position
import win_unicode_console
except ImportError:
WINUNICODE = False
else:
win_unicode_console.enable()
WINUNICODE = True
class InstaloaderException(Exception):
"""Base exception for this script"""
pass
@@ -15,6 +40,9 @@ class NonfatalException(InstaloaderException):
class ProfileNotExistsException(NonfatalException):
pass
class ProfileAccessDeniedException(NonfatalException):
pass
class ProfileHasNoPicsException(NonfatalException):
pass
@@ -31,57 +59,67 @@ class ConnectionException(InstaloaderException):
pass
def log(*msg, sep='', end='\n', flush=False, quiet=False):
def _log(*msg, sep='', end='\n', flush=False, quiet=False):
if not quiet:
print(*msg, sep=sep, end=end, flush=flush)
def get_json(name, session, max_id=0, sleep_min_max=[1,5]):
resp = session.get('http://www.instagram.com/'+name, \
params={'max_id': max_id})
time.sleep(abs(sleep_min_max[1]-sleep_min_max[0])*random.random()+abs(sleep_min_max[0]))
match = re.search('window\._sharedData = .*<', resp.text)
if match is None:
return None
def get_json(name: str, session: requests.Session,
max_id: Optional[str] = None, sleep: bool = True) -> Optional[Dict[str, Any]]:
"""Return JSON of a profile"""
if not max_id:
resp = session.get('https://www.instagram.com/'+name)
else:
resp = session.get('https://www.instagram.com/'+name, params={'max_id': max_id})
if sleep:
time.sleep(4 * random.random() + 1)
match = re.search('window\\._sharedData = .*<', resp.text)
if match is not None:
return json.loads(match.group(0)[21:-2])
def get_last_id(data):
if len(data["entry_data"]) == 0 or \
len(data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"]) == 0:
return None
else:
data = data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"]
return int(data[len(data)-1]["id"])
def get_username_by_id(session, profile_id):
def get_username_by_id(session: requests.Session, profile_id: int) -> str:
"""To get the current username of a profile, given its unique ID, this function can be used.
session is required to be a logged-in (i.e. non-anonymous) session."""
tempsession = copy_session(session)
tempsession.headers.update({'Content-Type' : 'application/json'})
tempsession.headers.update({'Content-Type' : 'application/x-www-form-urlencoded'})
resp = tempsession.post('https://www.instagram.com/query/', data='q=ig_user(' +
str(profile_id) +')+%7B%0A++username%0A%7D%0A')
if resp.status_code == 200:
data = json.loads(resp.text)
if 'username' in data:
return json.loads(resp.text)['username']
raise ProfileNotExistsException("no profile found, the user may have blocked " +
"you (id: " + str(profile_id) + ")")
raise ProfileNotExistsException("No profile found, the user may have blocked " +
"you (id: " + str(profile_id) + ").")
else:
if test_login(session):
raise ConnectionException("username could not be determined due to ConnectionError" +
" (id: "+ str(profile_id) +")")
raise ProfileAccessDeniedException("Username could not be determined due to error {0} (id: {1})."
.format(str(resp.status_code), str(profile_id)))
raise LoginRequiredException("Login required to determine username (id: " +
str(profile_id) + ")")
str(profile_id) + ").")
def epoch_to_string(epoch):
def get_id_by_username(profile: str) -> int:
"""Each Instagram profile has its own unique ID which stays unmodified even if a user changes
his/her username. To get said ID, given the profile's name, you may call this function."""
data = get_json(profile, get_anonymous_session())
if "ProfilePage" not in data["entry_data"]:
raise ProfileNotExistsException("Profile {0} does not exist.".format(profile))
return int(data['entry_data']['ProfilePage'][0]['user']['id'])
def _epoch_to_string(epoch: Real) -> str:
return datetime.datetime.fromtimestamp(epoch).strftime('%Y-%m-%d_%H-%M-%S')
def get_file_extension(url):
match = re.search('\.[a-z]*\?', url)
if match is None:
return url[-3:]
else:
return match.group(0)[1:-1]
def get_followees(profile, session):
def get_followees(profile: str, session: requests.Session) -> List[Dict[str, Any]]:
"""
Retrieve list of followees of given profile
:param profile: Name of profile to lookup followees
:param session: Session belonging to a user, i.e. not an anonymous session
:return: List of followees (list of dictionaries), as returned by instagram server
"""
tmpsession = copy_session(session)
data = get_json(profile, tmpsession)
profile_id = data['entry_data']['ProfilePage'][0]['user']['id']
@@ -107,7 +145,7 @@ def get_followees(profile, session):
"&ref=relationships%3A%3Afollow_list"]
tmpsession.headers.update(default_http_header())
tmpsession.headers.update({'Referer' : 'https://www.instagram.com/'+profile+'/following/'})
tmpsession.headers.update({'Content-Type' : 'application/json'})
tmpsession.headers.update({'Content-Type' : 'application/x-www-form-urlencoded'})
resp = tmpsession.post('https://www.instagram.com/query/', data=query[0]+"first("+query[1])
if resp.status_code == 200:
data = json.loads(resp.text)
@@ -127,20 +165,28 @@ def get_followees(profile, session):
return followees
if test_login(tmpsession):
raise ConnectionException("ConnectionError("+str(resp.status_code)+"): "
"unable to gather followees")
raise LoginRequiredException("Login required to gather followees")
"unable to gather followees.")
raise LoginRequiredException("Login required to gather followees.")
def download_pic(name, url, date_epoch, outputlabel=None, quiet=False):
# Returns true, if file was actually downloaded, i.e. updated
def download_pic(name: str, url: str, date_epoch: Real, outputlabel: Optional[str] = None, quiet: bool = False,
filename_suffix: Optional[str] = None) -> bool:
"""Downloads and saves picture with given url under given directory with given timestamp.
Returns true, if file was actually downloaded, i.e. updated."""
if outputlabel is None:
outputlabel = epoch_to_string(date_epoch)
filename = name.lower() + '/' + epoch_to_string(date_epoch) + '.' + get_file_extension(url)
outputlabel = _epoch_to_string(date_epoch)
urlmatch = re.search('\\.[a-z]*\\?', url)
file_extension = url[-3:] if urlmatch is None else urlmatch.group(0)[1:-1]
filename = name.lower() + '/' + _epoch_to_string(date_epoch)
if filename_suffix is not None:
filename += '_' + filename_suffix
filename += '.' + file_extension
if os.path.isfile(filename):
log(outputlabel + ' exists', end=' ', flush=True, quiet=quiet)
_log(outputlabel + ' exists', end=' ', flush=True, quiet=quiet)
return False
resp = get_anonymous_session().get(url, stream=True)
if resp.status_code == 200:
log(outputlabel, end=' ', flush=True, quiet=quiet)
_log(outputlabel, end=' ', flush=True, quiet=quiet)
os.makedirs(name.lower(), exist_ok=True)
with open(filename, 'wb') as file:
resp.raw.decode_content = True
@@ -148,75 +194,114 @@ def download_pic(name, url, date_epoch, outputlabel=None, quiet=False):
os.utime(filename, (datetime.datetime.now().timestamp(), date_epoch))
return True
else:
raise ConnectionException("file \'" + url + "\' could not be downloaded")
raise ConnectionException("File \'" + url + "\' could not be downloaded.")
def save_caption(name, date_epoch, caption, quiet=False):
filename = name.lower() + '/' + epoch_to_string(date_epoch) + '.txt'
if os.path.isfile(filename):
with open(filename, 'r') as file:
def save_caption(name: str, date_epoch: Real, caption: str, shorter_output: bool = False, quiet: bool = False) -> None:
"""Updates picture caption"""
filename = name.lower() + '/' + _epoch_to_string(date_epoch) + '.txt'
pcaption = caption.replace('\n', ' ').strip()
caption = caption.encode("UTF-8")
if shorter_output:
pcaption = "txt"
else:
pcaption = '[' + ((pcaption[:29]+u"\u2026") if len(pcaption)>31 else pcaption) + ']'
try:
with open(filename, 'rb') as file:
file_caption = file.read()
if file_caption == caption:
log('txt unchanged', end=' ', flush=True, quiet=quiet)
if file_caption.replace(b'\r\n', b'\n') == caption.replace(b'\r\n', b'\n'):
try:
_log(pcaption + ' unchanged', end=' ', flush=True, quiet=quiet)
except UnicodeEncodeError:
_log('txt unchanged', end=' ', flush=True, quiet=quiet)
return None
else:
def get_filename(index):
return filename if index==0 else (filename[:-4] + '_old_' + \
return filename if index==0 else (filename[:-4] + '_old_' +
(str(0) if index<10 else str()) + str(index) + filename[-4:])
i = 0
while os.path.isfile(get_filename(i)):
i = i + 1
for index in range(i, 0, -1):
os.rename(get_filename(index-1), get_filename(index))
log('txt updated', end=' ', flush=True, quiet=quiet)
log('txt', end=' ', flush=True, quiet=quiet)
try:
_log(pcaption + ' updated', end=' ', flush=True, quiet=quiet)
except UnicodeEncodeError:
_log('txt updated', end=' ', flush=True, quiet=quiet)
except FileNotFoundError:
pass
try:
_log(pcaption, end=' ', flush=True, quiet=quiet)
except UnicodeEncodeError:
_log('txt', end=' ', flush=True, quiet=quiet)
os.makedirs(name.lower(), exist_ok=True)
with open(filename, 'w') as text_file:
text_file.write(caption)
with open(filename, 'wb') as text_file:
shutil.copyfileobj(BytesIO(caption), text_file)
os.utime(filename, (datetime.datetime.now().timestamp(), date_epoch))
def download_profilepic(name, url, quiet=False):
def save_location(name: str, location_json: Dict[str, str], date_epoch: Real, quiet: bool = False) -> None:
filename = name.lower() + '/' + _epoch_to_string(date_epoch) + '_location.txt'
location_string = location_json["name"]+"\n" + \
"https://maps.google.com/maps?q={0},{1}&ll={0},{1}\n" \
.format(location_json["lat"], location_json["lng"])
os.makedirs(name.lower(), exist_ok=True)
with open(filename, 'wb') as text_file:
shutil.copyfileobj(BytesIO(location_string.encode()), text_file)
os.utime(filename, (datetime.datetime.now().timestamp(), date_epoch))
_log('geo', end=' ', flush=True, quiet=quiet)
def download_profilepic(name: str, url: str, quiet: bool = False) -> None:
"""Downloads and saves profile pic with given url."""
date_object = datetime.datetime.strptime(requests.head(url).headers["Last-Modified"], \
'%a, %d %b %Y %H:%M:%S GMT')
filename = name.lower() + '/' + epoch_to_string(date_object.timestamp()) + \
filename = name.lower() + '/' + _epoch_to_string(date_object.timestamp()) + \
'_UTC_profile_pic.' + url[-3:]
if os.path.isfile(filename):
log(filename + ' already exists', quiet=quiet)
_log(filename + ' already exists', quiet=quiet)
return None
match = re.search('http.*://.*instagram.*[^/]*\.(com|net)/[^/]+/.', url)
match = re.search('http.*://.*instagram.*[^/]*\\.(com|net)/[^/]+/.', url)
if match is None:
raise ConnectionException("url \'" + url + "\' could not be processed")
raise ConnectionException("URL \'" + url + "\' could not be processed.")
index = len(match.group(0))-1
offset = 8 if match.group(0)[-1:] == 's' else 0
url = url[:index] + 's2048x2048' + ('/' if offset == 0 else str()) + url[index+offset:]
resp = get_anonymous_session().get(url, stream=True)
if resp.status_code == 200:
log(filename, quiet=quiet)
_log(filename, quiet=quiet)
os.makedirs(name.lower(), exist_ok=True)
with open(filename, 'wb') as file:
resp.raw.decode_content = True
shutil.copyfileobj(resp.raw, file)
os.utime(filename, (datetime.datetime.now().timestamp(), date_object.timestamp()))
else:
raise ConnectionException("file \'" + url + "\' could not be downloaded")
raise ConnectionException("File \'" + url + "\' could not be downloaded.")
def get_default_session_filename(username):
def get_default_session_filename(username: str) -> str:
"""Returns default session filename for given username."""
dirname = tempfile.gettempdir() + "/" + ".instaloader-" + getpass.getuser()
filename = dirname + "/" + "session-" + username
return filename
def save_session(session, username, filename=None, quiet=False):
def save_session(session: requests.Session, username: str, filename: Optional[str] = None, quiet: bool = False) -> None:
"""Saves requests.Session object."""
if filename is None:
filename = get_default_session_filename(username)
dirname = os.path.dirname(filename)
if not os.path.exists(dirname):
if dirname != '' and not os.path.exists(dirname):
os.makedirs(dirname)
os.chmod(dirname, 0o700)
with open(filename, 'wb') as sessionfile:
os.chmod(filename, 0o600)
pickle.dump(requests.utils.dict_from_cookiejar(session.cookies), sessionfile)
log("Saved session to %s." % filename, quiet=quiet)
_log("Saved session to %s." % filename, quiet=quiet)
def load_session(username, filename=None, quiet=False):
def load_session(username: str, filename: Optional[str] = None, quiet: bool = False) -> requests.Session:
"""Returns loaded requests.Session object, or None if not found."""
if filename is None:
filename = get_default_session_filename(username)
try:
@@ -225,19 +310,23 @@ def load_session(username, filename=None, quiet=False):
session.cookies = requests.utils.cookiejar_from_dict(pickle.load(sessionfile))
session.headers.update(default_http_header())
session.headers.update({'X-CSRFToken':session.cookies.get_dict()['csrftoken']})
log("Loaded session from %s." % filename, quiet=quiet)
_log("Loaded session from %s." % filename, quiet=quiet)
return session
except FileNotFoundError:
pass
def copy_session(session):
def copy_session(session: requests.Session) -> requests.Session:
"""Duplicates a requests.Session."""
new = requests.Session()
new.cookies = \
requests.utils.cookiejar_from_dict(requests.utils.dict_from_cookiejar(session.cookies))
new.headers = session.headers
return new
def test_login(session):
def test_login(session: requests.Session) -> Optional[str]:
"""Returns the Instagram username to which given requests.Session object belongs, or None."""
if session is None:
return
data = get_json(str(), session)
@@ -246,7 +335,9 @@ def test_login(session):
time.sleep(4 * random.random() + 1)
return data['config']['viewer']['username']
def default_http_header(empty_session_only=False):
def default_http_header(empty_session_only: bool = False) -> Dict[str, str]:
"""Returns default HTTP header we use for requests."""
user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' \
'(KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36'
header = { 'Accept-Encoding' : 'gzip, deflate', \
@@ -267,7 +358,9 @@ def default_http_header(empty_session_only=False):
del header['X-Requested-With']
return header
def get_anonymous_session():
def get_anonymous_session() -> requests.Session:
"""Returns our default anonymous requests.Session object."""
session = requests.Session()
session.cookies.update({'sessionid' : '', 'mid' : '', 'ig_pr' : '1', \
'ig_vw' : '1920', 'csrftoken' : '', \
@@ -275,7 +368,8 @@ def get_anonymous_session():
session.headers.update(default_http_header(empty_session_only=True))
return session
def get_session(user, passwd):
def get_session(user: str, passwd: str) -> requests.Session:
"""Log in to instagram with given username and password and return session object"""
session = requests.Session()
session.cookies.update({'sessionid' : '', 'mid' : '', 'ig_pr' : '1', \
@@ -297,18 +391,287 @@ def get_session(user, passwd):
else:
raise ConnectionException('Login error! Connection error!')
def download(name, session, profile_pic_only=False, download_videos=True,
fast_update=False, sleep_min_max=[0.25,2], quiet=False):
def get_feed_json(session: requests.Session, end_cursor: str = None, sleep: bool = True) -> Dict[str, Any]:
"""
Get JSON of the user's feed.
:param session: Session belonging to a user, i.e. not an anonymous session
:param end_cursor: The end cursor, as from json["feed"]["media"]["page_info"]["end_cursor"]
:param sleep: Sleep between requests to instagram server
:return: JSON
"""
if end_cursor is None:
return get_json(str(), session, sleep=sleep)["entry_data"]["FeedPage"][0]
tmpsession = copy_session(session)
query = "q=ig_me()+%7B%0A++feed+%7B%0A++++media.after(" + end_cursor + "%2C+12)+%7B%0A"+\
"++++++nodes+%7B%0A++++++++id%2C%0A++++++++caption%2C%0A++++++++code%2C%0A++++++++"+\
"comments.last(4)+%7B%0A++++++++++count%2C%0A++++++++++nodes+%7B%0A++++++++++++"+\
"id%2C%0A++++++++++++created_at%2C%0A++++++++++++text%2C%0A++++++++++++"+\
"user+%7B%0A++++++++++++++id%2C%0A++++++++++++++profile_pic_url%2C%0A++++++++++++++"+\
"username%0A++++++++++++%7D%0A++++++++++%7D%2C%0A++++++++++"+\
"page_info%0A++++++++%7D%2C%0A++++++++comments_disabled%2C%0A++++++++"+\
"date%2C%0A++++++++dimensions+%7B%0A++++++++++height%2C%0A++++++++++"+\
"width%0A++++++++%7D%2C%0A++++++++display_src%2C%0A++++++++is_video%2C%0A++++++++"+\
"likes+%7B%0A++++++++++count%2C%0A++++++++++nodes+%7B%0A++++++++++++"+\
"user+%7B%0A++++++++++++++id%2C%0A++++++++++++++profile_pic_url%2C%0A++++++++++++++"+\
"username%0A++++++++++++%7D%0A++++++++++%7D%2C%0A++++++++++"+\
"viewer_has_liked%0A++++++++%7D%2C%0A++++++++location+%7B%0A++++++++++"+\
"id%2C%0A++++++++++has_public_page%2C%0A++++++++++name%0A++++++++%7D%2C%0A++++++++"+\
"owner+%7B%0A++++++++++id%2C%0A++++++++++blocked_by_viewer%2C%0A++++++++++"+\
"followed_by_viewer%2C%0A++++++++++full_name%2C%0A++++++++++"+\
"has_blocked_viewer%2C%0A++++++++++is_private%2C%0A++++++++++"+\
"profile_pic_url%2C%0A++++++++++requested_by_viewer%2C%0A++++++++++"+\
"username%0A++++++++%7D%2C%0A++++++++usertags+%7B%0A++++++++++"+\
"nodes+%7B%0A++++++++++++user+%7B%0A++++++++++++++"+\
"username%0A++++++++++++%7D%2C%0A++++++++++++x%2C%0A++++++++++++y%0A++++++++++"+\
"%7D%0A++++++++%7D%2C%0A++++++++video_url%2C%0A++++++++"+\
"video_views%0A++++++%7D%2C%0A++++++page_info%0A++++%7D%0A++%7D%2C%0A++id%2C%0A++"+\
"profile_pic_url%2C%0A++username%0A%7D%0A&ref=feed::show"
tmpsession.headers.update(default_http_header())
tmpsession.headers.update({'Referer' : 'https://www.instagram.com/'})
tmpsession.headers.update({'Content-Type' : 'application/x-www-form-urlencoded'})
resp = tmpsession.post('https://www.instagram.com/query/', data=query)
if sleep:
time.sleep(4 * random.random() + 1)
return json.loads(resp.text)
def get_location(session: requests.Session, node_code: str, sleep: bool = True) -> Dict[str, str]:
pic_json = get_json("p/" + node_code, session, sleep=sleep)
media = pic_json["entry_data"]["PostPage"][0]["graphql"]["shortcode_media"] \
if "graphql" in pic_json["entry_data"]["PostPage"][0] \
else pic_json["entry_data"]["PostPage"][0]["media"]
if media["location"] is not None:
location_json = get_json("explore/locations/" +
media["location"]["id"],
session, sleep=sleep)
return location_json["entry_data"]["LocationsPage"][0]["location"]
def download_node(node: Dict[str, Any], session: requests.Session, name: str,
download_videos: bool = True, geotags: bool = False,
sleep: bool = True, shorter_output: bool = False, quiet: bool = False) -> bool:
"""
Download everything associated with one instagram node, i.e. picture, caption and video.
:param node: Node, as from media->nodes list in instagram's JSONs
:param session: Session
:param name: Name of profile to which this node belongs
:param download_videos: True, if videos should be downloaded
:param geotags: Download geotags
:param sleep: Sleep between requests to instagram server
:param shorter_output: Shorten log output by not printing captions
:param quiet: Suppress output
:return: True if something was downloaded, False otherwise, i.e. file was already there
"""
# pylint:disable=too-many-branches,too-many-locals
date = node["date"] if "date" in node else node["taken_at_timestamp"]
if '__typename' in node:
if node['__typename'] == 'GraphSidecar':
sidecar_data = session.get('https://www.instagram.com/p/' + node['code'] + '/', params={'__a': 1}).json()
edge_number = 1
downloaded = True
media = sidecar_data["graphql"]["shortcode_media"] if "graphql" in sidecar_data else sidecar_data["media"]
for edge in media['edge_sidecar_to_children']['edges']:
edge_downloaded = download_pic(name, edge['node']['display_url'],date,
filename_suffix=str(edge_number), quiet=quiet,
outputlabel=(str(edge_number) if edge_number != 1 else None))
downloaded = downloaded and edge_downloaded
edge_number += 1
if sleep:
time.sleep(1.75 * random.random() + 0.25)
elif node['__typename'] in ['GraphImage', 'GraphVideo']:
downloaded = download_pic(name, node["display_url"] if "display_url" in node else node["display_src"],
date, quiet=quiet)
if sleep:
time.sleep(1.75 * random.random() + 0.25)
else:
_log("Warning: Unknown typename discovered:" + node['__typename'])
downloaded = False
else:
# Node is an old image or video.
downloaded = download_pic(name, node["display_src"], date, quiet=quiet)
if sleep:
time.sleep(1.75 * random.random() + 0.25)
if "edge_media_to_caption" in node and node["edge_media_to_caption"]["edges"]:
save_caption(name, date, node["edge_media_to_caption"]["edges"][0]["node"]["text"], shorter_output, quiet)
elif "caption" in node:
save_caption(name, date, node["caption"], shorter_output, quiet)
else:
_log("<no caption>", end=' ', flush=True, quiet=quiet)
node_code = node['shortcode'] if 'shortcode' in node else node['code']
if node["is_video"] and download_videos:
video_data = get_json('p/' + node_code, session, sleep=sleep)
download_pic(name,
video_data['entry_data']['PostPage'][0]['graphql']['shortcode_media']['video_url'],
date, 'mp4', quiet=quiet)
if geotags:
location = get_location(session, node_code, sleep)
if location:
save_location(name, location, date, quiet=quiet)
_log(quiet=quiet)
return downloaded
def download_feed_pics(session: requests.Session, max_count: int = None, fast_update: bool = False,
filter_func: Optional[Callable[[Dict[str, Dict[str, Any]]], bool]] = None,
download_videos: bool = True, geotags: bool = False,
shorter_output: bool = False, sleep: bool = True, quiet: bool = False) -> None:
"""
Download pictures from the user's feed.
Example to download up to the 20 pics the user last liked:
>>> download_feed_pics(load_session('USER'), max_count=20, fast_update=True,
>>> filter_func=lambda node:
>>> not node["likes"]["viewer_has_liked"] if "likes" in node else not node["viewer_has_liked"])
:param session: Session belonging to a user, i.e. not an anonymous session
:param max_count: Maximum count of pictures to download
:param fast_update: If true, abort when first already-downloaded picture is encountered
:param filter_func: function(node), which returns True if given picture should not be downloaded
:param download_videos: True, if videos should be downloaded
:param geotags: Download geotags
:param shorter_output: Shorten log output by not printing captions
:param sleep: Sleep between requests to instagram server
:param quiet: Suppress output
"""
# pylint:disable=too-many-locals
data = get_feed_json(session, sleep=sleep)
count = 1
while True:
if "graphql" in data:
is_edge = True
feed = data["graphql"]["user"]["edge_web_feed_timeline"]
else:
is_edge = False
feed = data["feed"]["media"]
for edge_or_node in feed["edges"] if is_edge else feed["nodes"]:
if max_count is not None and count > max_count:
return
node = edge_or_node["node"] if is_edge else edge_or_node
name = node["owner"]["username"]
if filter_func is not None and filter_func(node):
_log("<pic by %s skipped>" % name, flush=True, quiet=quiet)
continue
_log("[%3i] %s " % (count, name), end="", flush=True, quiet=quiet)
count += 1
downloaded = download_node(node, session, name,
download_videos=download_videos, geotags=geotags,
sleep=sleep, shorter_output=shorter_output, quiet=quiet)
if fast_update and not downloaded:
return
if not feed["page_info"]["has_next_page"]:
break
data = get_feed_json(session, end_cursor=feed["page_info"]["end_cursor"], sleep=sleep)
def get_hashtag_json(hashtag: str, session: requests.Session,
max_id: Optional[str] = None, sleep: bool = True) -> Optional[Dict[str, Any]]:
"""Return JSON of a #hashtag"""
return get_json(name='explore/tags/{0}/'.format(hashtag), session=session, max_id=max_id, sleep=sleep)
def download_hashtag(hashtag: str, session: requests.Session,
max_count: Optional[int] = None,
filter_func: Optional[Callable[[Dict[str, Dict[str, Any]]], bool]] = None,
fast_update: bool = False, download_videos: bool = True, geotags: bool = False,
shorter_output: bool = False, sleep: bool = True, quiet: bool = False) -> None:
"""Download pictures of one hashtag.
To download the last 30 pictures with hashtag #cat, do
>>> download_hashtag('cat', session=get_anonymous_session(), max_count=30)
:param hashtag: Hashtag to download, without leading '#'
:param session: Session belonging to a user, i.e. not an anonymous session
:param max_count: Maximum count of pictures to download
:param filter_func: function(node), which returns True if given picture should not be downloaded
:param fast_update: If true, abort when first already-downloaded picture is encountered
:param download_videos: True, if videos should be downloaded
:param geotags: Download geotags
:param shorter_output: Shorten log output by not printing captions
:param sleep: Sleep between requests to instagram server
:param quiet: Suppress output
"""
data = get_hashtag_json(hashtag, session, sleep=sleep)
count = 1
while data:
for node in data['entry_data']['TagPage'][0]['tag']['media']['nodes']:
if max_count is not None and count > max_count:
return
_log('[{0:3d}] #{1} '.format(count, hashtag), end='', flush=True, quiet=quiet)
if filter_func is not None and filter_func(node):
_log('<skipped>', quiet=quiet)
continue
count += 1
downloaded = download_node(node, session, '#{0}'.format(hashtag),
download_videos=download_videos, geotags=geotags, sleep=sleep,
shorter_output=shorter_output, quiet=quiet)
if fast_update and not downloaded:
return
if data['entry_data']['TagPage'][0]['tag']['media']['page_info']['has_next_page']:
data = get_hashtag_json(hashtag, session, sleep=sleep,
max_id=data['entry_data']['TagPage'][0]['tag']['media']['page_info']['end_cursor'])
else:
break
def check_id(profile: str, session: requests.Session, json_data: Dict[str, Any], quiet: bool = False) -> str:
"""
Consult locally stored ID of profile with given name, check whether ID matches and whether name
has changed and return current name of the profile, and store ID of profile.
"""
profile_exists = len(json_data["entry_data"]) > 0 and "ProfilePage" in json_data["entry_data"]
is_logged_in = json_data["config"]["viewer"] is not None
try:
with open(profile + "/id", 'rb') as id_file:
profile_id = int(id_file.read())
if (not profile_exists) or \
(profile_id != int(json_data['entry_data']['ProfilePage'][0]['user']['id'])):
if is_logged_in:
newname = get_username_by_id(session, profile_id)
_log("Profile {0} has changed its name to {1}.".format(profile, newname),
quiet=quiet)
os.rename(profile, newname)
return newname
if profile_exists:
raise ProfileNotExistsException("Profile {0} does not match the stored "
"unique ID {1}.".format(profile, profile_id))
raise ProfileNotExistsException("Profile {0} does not exist. Please login to "
"update profile name. Unique ID: {1}."
.format(profile, profile_id))
return profile
except FileNotFoundError:
pass
if profile_exists:
os.makedirs(profile.lower(), exist_ok=True)
with open(profile + "/id", 'w') as text_file:
profile_id = json_data['entry_data']['ProfilePage'][0]['user']['id']
text_file.write(profile_id+"\n")
_log("Stored ID {0} for profile {1}.".format(profile_id, profile), quiet=quiet)
return profile
raise ProfileNotExistsException("Profile {0} does not exist.".format(profile))
def download(name: str, session: requests.Session,
profile_pic_only: bool = False, download_videos: bool = True, geotags: bool = False,
fast_update: bool = False, shorter_output: bool = False, sleep: bool = True,
quiet: bool = False) -> None:
"""Download one profile"""
# pylint:disable=too-many-arguments
# pylint:disable=too-many-branches,too-many-locals
# Get profile main page json
data = get_json(name, session)
if len(data["entry_data"]) == 0 or "ProfilePage" not in data["entry_data"]:
raise ProfileNotExistsException("profile %s does not exist" % name)
data = get_json(name, session, sleep=sleep)
# check if profile does exist or name has changed since last download
# and update name and json data if necessary
name_updated = check_id(name, session, data, quiet=quiet)
if name_updated != name:
name = name_updated
data = get_json(name, session, sleep=sleep)
# Download profile picture
download_profilepic(name, data["entry_data"]["ProfilePage"][0]["user"]["profile_pic_url"],
quiet=quiet)
time.sleep(abs(sleep_min_max[1]-sleep_min_max[0])*random.random()+abs(sleep_min_max[0]))
if sleep:
time.sleep(1.75 * random.random() + 0.25)
if profile_pic_only:
return
# Catch some errors
@@ -316,38 +679,33 @@ def download(name, session, profile_pic_only=False, download_videos=True,
if data["config"]["viewer"] is None:
raise LoginRequiredException("profile %s requires login" % name)
if not data["entry_data"]["ProfilePage"][0]["user"]["followed_by_viewer"]:
raise PrivateProfileNotFollowedException("user %s: private but not followed" % name)
raise PrivateProfileNotFollowedException("Profile %s: private but not followed." % name)
else:
if data["config"]["viewer"] is not None:
log("profile %s could also be downloaded anonymously." % name, quiet=quiet)
_log("profile %s could also be downloaded anonymously." % name, quiet=quiet)
if ("nodes" not in data["entry_data"]["ProfilePage"][0]["user"]["media"] or
len(data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"]) == 0) \
not data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"]) \
and not profile_pic_only:
raise ProfileHasNoPicsException("profile %s: no pics found" % name)
raise ProfileHasNoPicsException("Profile %s: no pics found." % name)
# Iterate over pictures and download them
def get_last_id(data):
if data["entry_data"] and data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"]:
return data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"][-1]["id"]
totalcount = data["entry_data"]["ProfilePage"][0]["user"]["media"]["count"]
count = 1
while get_last_id(data) is not None:
for node in data["entry_data"]["ProfilePage"][0]["user"]["media"]["nodes"]:
log("[%3i/%3i] " % (count, totalcount), end="", flush=True, quiet=quiet)
count = count + 1
downloaded = download_pic(name, node["display_src"], node["date"], quiet=quiet)
time.sleep(abs(sleep_min_max[1]-sleep_min_max[0])*random.random() + \
abs(sleep_min_max[0]))
if "caption" in node:
save_caption(name, node["date"], node["caption"], quiet=quiet)
if node["is_video"] and download_videos:
video_data = get_json('p/' + node["code"], session)
download_pic(name, \
video_data['entry_data']['PostPage'][0]['media']['video_url'], \
node["date"], 'mp4', quiet=quiet)
log(quiet=quiet)
_log("[%3i/%3i] " % (count, totalcount), end="", flush=True, quiet=quiet)
count += 1
downloaded = download_node(node, session, name,
download_videos=download_videos, geotags=geotags,
sleep=sleep, shorter_output=shorter_output, quiet=quiet)
if fast_update and not downloaded:
return
data = get_json(name, session, max_id=get_last_id(data))
time.sleep(abs(sleep_min_max[1]-sleep_min_max[0])*random.random()+abs(sleep_min_max[0]))
data = get_json(name, session, max_id=get_last_id(data), sleep=sleep)
def get_logged_in_session(username, password=None, quiet=False):
def get_logged_in_session(username: str, password: Optional[str] = None, quiet: bool = False) -> requests.Session:
"""Logs in and returns session, asking user for password if needed"""
if password is not None:
return get_session(username, password)
@@ -362,48 +720,100 @@ def get_logged_in_session(username, password=None, quiet=False):
print(err, file=sys.stderr)
password = None
def download_profiles(targets, username=None, password=None, sessionfile=None,
profile_pic_only=False, download_videos=True, fast_update=False,
sleep_min_max=[0.25,2], quiet=False):
def download_profiles(profilelist: List[str], username: Optional[str] = None, password: Optional[str] = None,
sessionfile: Optional[str] = None, max_count: Optional[int] = None,
profile_pic_only: bool = False, download_videos: bool = True, geotags: bool = False,
fast_update: bool = False,
sleep: bool = True, shorter_output: bool = False, quiet: bool = False) -> None:
"""Download set of profiles and handle sessions"""
# pylint:disable=too-many-arguments
# pylint:disable=too-many-branches,too-many-locals
# Login, if desired
if username is not None:
session = load_session(username, sessionfile, quiet=quiet)
if username != test_login(session):
session = get_logged_in_session(username, password, quiet)
log("Logged in as %s." % username, quiet=quiet)
_log("Logged in as %s." % username, quiet=quiet)
else:
session = get_anonymous_session()
# Iterate through targets list and download them
# Try block for KeyboardInterrupt (save session on ^C)
failedtargets = []
targets = set()
try:
# Generate set of targets
for pentry in profilelist:
if pentry[0] == '#':
_log("Retrieving pictures with hashtag {0}".format(pentry), quiet=quiet)
download_hashtag(hashtag=pentry[1:], session=session, max_count=max_count, fast_update=fast_update,
download_videos=download_videos, geotags=geotags, shorter_output=shorter_output,
sleep=sleep, quiet=quiet)
elif pentry[0] == '@' and username is not None:
_log("Retrieving followees of %s..." % pentry[1:], quiet=quiet)
followees = get_followees(pentry[1:], session)
targets.update([followee['username'] for followee in followees])
elif pentry == ":feed-all" and username is not None:
_log("Retrieving pictures from your feed...", quiet=quiet)
download_feed_pics(session, fast_update=fast_update, max_count=max_count,
download_videos=download_videos, geotags=geotags,
shorter_output=shorter_output, sleep=sleep, quiet=quiet)
elif pentry == ":feed-liked" and username is not None:
_log("Retrieving pictures you liked from your feed...", quiet=quiet)
download_feed_pics(session, fast_update=fast_update, max_count=max_count,
filter_func=lambda node:
not node["likes"]["viewer_has_liked"]
if "likes" in node
else not node["viewer_has_liked"],
download_videos=download_videos, geotags=geotags,
shorter_output=shorter_output, sleep=sleep, quiet=quiet)
else:
targets.add(pentry)
if len(targets) > 1:
_log("Downloading %i profiles..." % len(targets), quiet=quiet)
# Iterate through targets list and download them
for target in targets:
try:
download(target, session, profile_pic_only, download_videos,
fast_update, sleep_min_max, quiet)
try:
download(target, session, profile_pic_only, download_videos,
geotags, fast_update, shorter_output, sleep, quiet)
except ProfileNotExistsException as err:
if username is not None:
_log("\"Profile not exists\" - Trying again anonymously, helps in case you are just blocked")
download(target, get_anonymous_session(), profile_pic_only, download_videos,
geotags, fast_update, shorter_output, sleep, quiet)
else:
raise err
except NonfatalException as err:
failedtargets.append(target)
print(err, file=sys.stderr)
except KeyboardInterrupt:
print("\nInterrupted by user.", file=sys.stderr)
if len(targets) > 1 and len(failedtargets) > 0:
print("Errors occured (see above) while downloading profiles: %s" %
if len(targets) > 1 and failedtargets:
print("Errors occured (see above) while downloading profiles: %s." %
", ".join(failedtargets), file=sys.stderr)
# Save session if it is useful
if username is not None:
save_session(session, username, sessionfile, quiet=quiet)
def main():
parser = ArgumentParser(description='Simple downloader to fetch all Instagram pics and '\
'captions from a given profile')
parser.add_argument('targets', nargs='+', help='Names of profiles to download')
parser = ArgumentParser(description=__doc__,
epilog="Report issues at https://github.com/Thammus/instaloader/issues.")
parser.add_argument('profile', nargs='*', metavar='profile|#hashtag',
help='Name of profile or #hashtag to download. '
'Alternatively, if --login is given: @<profile> to download all followees of '
'<profile>; or the special targets :feed-all or :feed-liked to '
'download pictures from your feed (using '
'--fast-update is recommended).')
parser.add_argument('--version', action='version',
version='1.0')
parser.add_argument('-l', '--login', metavar='login_name',
help='Provide login name for your Instagram account')
parser.add_argument('-p', '--password',
help='Provide password for your Instagram account')
version=__version__)
parser.add_argument('-l', '--login', metavar='YOUR-USERNAME',
help='Login name for your Instagram account. Not needed to download public '\
'profiles, but if you want to download private profiles or all followees of '\
'some profile, you have to specify a username used to login.')
parser.add_argument('-p', '--password', metavar='YOUR-PASSWORD',
help='Password for your Instagram account. If --login is given and there is '\
'not yet a valid session file, you\'ll be prompted for your password if '\
'--password is not given. Specifying this option without --login has no '\
'effect.')
parser.add_argument('-f', '--sessionfile',
help='File to store session key, defaults to '+ \
get_default_session_filename("<login_name>"))
@@ -411,18 +821,26 @@ def main():
help='Only download profile picture')
parser.add_argument('-V', '--skip-videos', action='store_true',
help='Do not download videos')
parser.add_argument('-G', '--geotags', action='store_true',
help='Store geotags when available')
parser.add_argument('-F', '--fast-update', action='store_true',
help='Abort at encounter of first already-downloaded picture')
parser.add_argument('-c', '--count',
help='Do not attempt to download more than COUNT posts. '
'Applies only to #hashtag, :feed-all and :feed-liked.')
parser.add_argument('-S', '--no-sleep', action='store_true',
help='Do not sleep between actual downloads of pictures')
parser.add_argument('-O', '--shorter-output', action='store_true',
help='Do not display captions while downloading')
parser.add_argument('-q', '--quiet', action='store_true',
help='Disable user interaction, i.e. do not print messages (except errors) and fail ' \
'if login credentials are needed but not given.')
args = parser.parse_args()
try:
download_profiles(args.targets, args.login, args.password, args.sessionfile,
args.profile_pic_only, not args.skip_videos, args.fast_update,
[0,0] if args.no_sleep else [0.25,2], args.quiet)
download_profiles(args.profile, args.login, args.password, args.sessionfile,
int(args.count) if args.count is not None else None,
args.profile_pic_only, not args.skip_videos, args.geotags, args.fast_update,
not args.no_sleep, args.shorter_output, args.quiet)
except InstaloaderException as err:
raise SystemExit("Fatal error: %s" % err)
Executable
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env python3
import sys
from setuptools import setup
if sys.version_info < (3, 5):
sys.exit('Instaloader requires Python >= 3.5.')
setup(
name='instaloader',
version='1.3',
py_modules=['instaloader'],
url='https://github.com/Thammus/instaloader',
license='MIT',
author='Alexander Graf, André Koch-Kramer',
author_email='mail@agraf.me, koch-kramer@web.de',
description='Tool to download Instagram pictures (or videos) and captions from a given set of '
'profiles (even if private), from your feed or from all followees of a given '
'profile.',
long_description=open('README.rst').read(),
install_requires=['requests>=2.4'],
python_requires='>=3.5',
entry_points={'console_scripts': ['instaloader=instaloader:main']},
zip_safe=True,
keywords='instagram downloader',
classifiers=[
'Development Status :: 5 - Production/Stable',
'Environment :: Console',
'Intended Audience :: End Users/Desktop',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3 :: Only',
'Topic :: Internet',
'Topic :: Multimedia :: Graphics'
]
)