First Release Candidate for Version 4.4.5

* Bypass IP-based redirects to /accounts/login

There are two main changes made:

For users, we request /{username}/feed/ instead of /{username}/. For some
reason, this completely bypasses the login redirect. This page doesn't
work in browser while blocked, but fortunately all the data we need is
just present in the HTML page.

For posts, we change from using the /p/ page to using the graphql
endpoint for the same data, which is still subject to graphql rate
limits, but is not subject to login redirects. The data is identical
between the two pages, apart from the object keys being sorted
differently and rhx_gis being missing on graphql.

Yes, this now unblocks access from VPNs, Tor, cloud servers, etc.

* Apply requested patch to fix comments

* Remove rhx_gis from Post and Profile

Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>

.github/ISSUE_TEMPLATE/bug_report.md

@@ -24,4 +24,4 @@ If applicable, add error messages and tracebacks to help explain your problem.
 **Additional context**
 Add any other context about the problem here.
 
-(please also see https://instaloader.github.io/contributing.html for how to report a bug)
+<!-- please also see https://instaloader.github.io/contributing.html for how to report a bug -->

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,11 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Contributing to Instaloader
+    url: https://instaloader.github.io/contributing.html
+    about: Guidelines for how to report an issue.
+  - name: Instaloader Troubleshooting
+    url: https://instaloader.github.io/troubleshooting.html
+    about: Many common problems and their workarounds are described here.
+  - name: Instaloader Documentation
+    url: https://instaloader.github.io/
+    about: Many questions might be already answered here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -15,10 +15,10 @@ A clear and concise description of what you want to happen.
 A clear and concise description of any alternative solutions or features you've considered.
 
 **If the feature request is accepted, would you be willing to submit a PR?**
-
-Yes / No _(Help can be provided if you need assistance submitting a PR)_
+Yes / No
+<!-- Help can be provided if you need assistance submitting a PR -->
 
 **Additional context**
 Add any other context about the feature request here.
 
-(please also see https://instaloader.github.io/contributing.html for how to suggest a feature)
+<!-- please also see https://instaloader.github.io/contributing.html for how to suggest a feature -->

.github/ISSUE_TEMPLATE/question.md

@@ -6,3 +6,7 @@ labels: question
 ---
 
 Your question here...
+
+<!-- Please see our documentation: https://instaloader.github.io/ -->
+
+<!-- Basic questions regarding Instaloader's usage might be more appropriate to ask on Stack Overflow -->

.github/pull_request_template.md

@@ -0,0 +1,17 @@
+
+
+<!--
+Please describe:
+
+- A motivation for this change, e.g.
+  - Fixes # .
+  - More general: What problem does the pull request solve?
+
+- The changes proposed in this pull request
+
+- The completeness of this change
+  - Is it just a proof of concept?
+  - Is the documentation updated (if appropriate)?
+  - Do you consider it ready to be merged or is it a draft?
+  - Can we help you at some point?
+-->

.github/workflows/lint.yml

@@ -0,0 +1,30 @@
+name: PyLint and MyPy
+
+on: [push, pull_request]
+
+jobs:
+  lint:
+    name: PyLint and MyPy
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.8]
+    steps:
+      - name: Checkout Instaloader Repository
+        uses: actions/checkout@v2
+        with:
+          fetch-depth: 0  # needed for building docs
+      - name: Setup Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install Dependencies
+        run: |
+          python -m pip install pipenv==2020.6.2
+          pipenv sync --dev
+      - name: PyLint
+        run: pipenv run pylint instaloader
+      - name: MyPy
+        run: pipenv run mypy -m instaloader
+      - name: Build Documentation
+        run: pipenv run make -C docs html SPHINXOPTS="-W -n"

.github/workflows/stale.yml

@@ -7,6 +7,7 @@ on:
 jobs:
   stale:
 
+    name: Mark stale issues and pull requests
     runs-on: ubuntu-latest
     
     steps:

.github/workflows/windows_exe.yml

@@ -4,6 +4,7 @@ on: [push, pull_request]
 
 jobs:
   build:
+    name: Windows EXE build
     runs-on: windows-latest
     steps:
       - name: Checkout Instaloader repository

.travis.yml

@@ -27,7 +27,7 @@ deploy:
     repo: instaloader/instaloader.github.io
     target-branch: master
     on:
-      tags: true
+      branch: master
       python: 3.6
   - provider: script
     script: deploy/arch/deploy.sh $TRAVIS_TAG

docs/_static/instaloader.css

@@ -17,7 +17,9 @@ code {
   .highlight pre {
     padding: 0.7em;
     color: #fff; }
-  .highlight .c1, .highlight .k, .highlight .kn, .highlight .ow {
+  .highlight .c1 {
+    color: #666; }
+  .highlight .k, .highlight .kn, .highlight .ow {
     color: #008d06; }
   .highlight .nb, .highlight .ne, .highlight .nf, .highlight .vm {
     color: #f48400; }

docs/_static/instaloader.scss

@@ -36,7 +36,11 @@ code {
     color: #fff;
   }
 
-  .c1, .k, .kn, .ow {
+  .c1 {
+    color: #666;
+  }
+
+  .k, .kn, .ow {
     color: $color_instaloader_main
   }
 

docs/basic-usage.rst

@@ -111,8 +111,12 @@ Instaloader supports the following targets:
    ``profile`` (requires :option:`--login`).
 
 - ``-post``
-   The single **post** with the given shortcode. Must be preceeded by ``--`` in
-   the argument list to not be mistaken as an option flag.
+   Replace **post** with the post's shortcode to download single post. Must be preceeded by ``--`` in
+   the argument list to not be mistaken as an option flag::
+
+    instaloader -- -B_K4CykAOtf
+
+
 
    .. versionadded:: 4.1
 

docs/codesnippets.rst

@@ -28,8 +28,9 @@ Download Posts in a Specific Period
 -----------------------------------
 
 To only download Instagram pictures (and metadata) that are within a specific
-period, you can play around with :func:`~itertools.dropwhile` and
-:func:`~itertools.takewhile` from :mod:`itertools` like in this snippet.
+period, you can simply use :func:`~itertools.dropwhile` and
+:func:`~itertools.takewhile` from :mod:`itertools` on a generator that returns
+Posts in **exact chronological order**, such as :meth:`Profile.get_posts`.
 
 .. literalinclude:: codesnippets/121_since_until.py
 
@@ -37,6 +38,14 @@ See also :class:`Post`, :meth:`Instaloader.download_post`.
 
 Discussed in :issue:`121`.
 
+The code example with :func:`~itertools.dropwhile` and
+:func:`~itertools.takewhile` makes the assumption that the post iterator returns
+posts in exact chronological order.  As discussed in :issue:`666`, the following
+approach fits for an **almost chronological order**, where up to *k* older posts
+are inserted into an otherwise chronological order, such as an Hashtag feed.
+
+.. literalinclude:: codesnippets/666_historical_hashtag_data.py
+
 Likes of a Profile / Ghost Followers
 ------------------------------------
 

docs/codesnippets/121_since_until.py

@@ -5,13 +5,11 @@ import instaloader
 
 L = instaloader.Instaloader()
 
-posts = instaloader.Hashtag.from_name(L.context, 'urbanphotography').get_posts()
-# or
-# posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts()
+posts = instaloader.Profile.from_username(L.context, "instagram").get_posts()
 
 SINCE = datetime(2015, 5, 1)
 UNTIL = datetime(2015, 3, 1)
 
 for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
     print(post.date)
-    L.download_post(post, '#urbanphotography')
+    L.download_post(post, "instagram")

docs/codesnippets/666_historical_hashtag_data.py

@@ -0,0 +1,30 @@
+from datetime import datetime
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = instaloader.Hashtag.from_name(L.context, "urbanphotography").get_posts()
+
+SINCE = datetime(2020, 5, 10)  # further from today, inclusive
+UNTIL = datetime(2020, 5, 11)  # closer to today, not inclusive
+
+k = 0  # initiate k
+k_list = []  # uncomment this to tune k
+
+for post in posts:
+    postdate = post.date
+
+    if postdate > UNTIL:
+        continue
+    elif postdate <= SINCE:
+        k += 1
+        if k == 50:
+            break
+        else:
+            continue
+    else:
+        L.download_post(post, "#urbanphotography")
+        k = 0  # set k to 0
+        # if you want to tune k, uncomment below to get your k max
+        #k_list.append(k)
+#max(k_list)

docs/conf.py

@@ -18,6 +18,7 @@
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 import os
+import platform
 import subprocess
 import sys
 import typing
@@ -50,8 +51,9 @@ intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
                        'requests': ('https://requests.kennethreitz.org/en/master/', None)}
 
 current_release = subprocess.check_output(["git", "describe", "--abbrev=0"]).decode("ascii")[1:-1]
+date_format = "%e %b %Y" if platform.system() != "Windows" else "%d %b %Y"
 current_release_date = subprocess.check_output(
-    ["git", "log", "-1", "--tags", "--format=%ad", "--date=format:%e %b %Y"]).decode("ascii")[:-1]
+    ["git", "log", "-1", "--tags", "--format=%ad", "--date=format:"+date_format]).decode("ascii")[:-1]
 
 html_context = {'current_release': current_release, 'current_release_date': current_release_date}
 

docs/contributing.rst

@@ -35,8 +35,8 @@ reporting a problem, please keep the following in mind:
 - If not obvious, describe **which behavior you expected**
   instead of what actually happened.
 
-- If you think an issue has been closed accidentally or inappropriately, feel
-  free to reopen it.
+- If we have closed an issue apparently inadvertently or inappropriately, please
+  let us know.
 
 Writing Code or Improving the Documentation
 -------------------------------------------
@@ -45,28 +45,25 @@ Changes of the Instaloader source can be proposed as a
 `Pull Request <https://github.com/instaloader/instaloader/pulls>`__. There are only
 few things to consider:
 
-- Sometimes, the most current code is not in the ``master`` branch. Check that
-  you forked from the most recent branch.
+- Base your Pull Request on the ``master`` branch if it fixes a bug,
+  or the ``upcoming/v4.X`` branch (if it exists at the moment of submitting the PR)
+  otherwise.
 
 - We use `Pylint <https://www.pylint.org/>`__ for error and syntax checking of
-  the source. The file ``.travis.yml`` in the project's root directory
-  shows how it is invoked. Note that sometimes it might be better to disable a
-  warning rather than adapting the code to Pylint's desires.
+  the source and `MyPy <https://github.com/python/mypy>`__ for type checking.
+  Beware that sometimes it might be better to disable a warning rather than
+  adapting the code to a tool's desires.
 
 - The documentation source is located in the ``docs`` folder. The file
   ``cli-options.rst`` is merely an RST-formatted copy of ``instaloader --help``
   output, of which the source is in ``instaloader/__main__.py``.
 
-- Feel free to contact us, even if you "only" have Proof-of-Concepts or
-  not-fully integrated changes. They already might be an advance for the
-  project.
-
-Suggesting Features
--------------------
+Proposing Features
+------------------
 
 .. goal-start
 
-Instaloader's goal is to mimick the browser's behavior to access the data that
+Instaloader's goal is to mimic the browser's behavior to access the data that
 is available through the Instagram web interface, pack this data into complete
 and easily-(re)usable python objects, and provide a user interface for the most
 common downloading and metadata collection tasks, without changing any of the
@@ -74,10 +71,9 @@ online data.
 
 .. goal-end
 
-If you have an idea of how Instaloader should be enhanced, but do not want to
-implement the feature yourself, feel free to open a ticket in our 
-`Issue Tracker <https://github.com/instaloader/instaloader/issues>`__.
-Please consider the following:
+Prior spending effort on implementing a new feature, it might be appropriate to
+clarify how it could fit into the project's scope or discuss implementation
+details. If you feel the need to do so, please create a "feature suggestion".
 
 - Instaloader already has plenty of features. **Check the documentation**
   beforehand to ensure your desired suggestion is not already implemented.
@@ -86,16 +82,21 @@ Please consider the following:
   an issue suggesting the same or a similar feature, share your thoughts in a
   comment there, instead of opening a new issue.
 
-- If possible, provide us a **use case of the feature**: How could the user
-  invoke the new function? Which problem would it solve? If new information is
-  obtained, how would it be further processed?
+- **Motivate the feature**, i.e.
 
-- If not obvious, briefly motivate how your suggested feature **conforms with
-  Instaloader's project goal**.
+  - Provide us a **use case of the feature**: How could the user
+    invoke the new function? Which problem would it solve? If new information is
+    obtained, how would it be further processed?
 
-- **Be patient**. Naturally, bugs and pull requests have a higher priority than
-  feature suggestions. Keep in mind that this is a free software project, and
-  unfortunately we only have limited time to work on it.
+  - Describe already-working **alternatives of the feature** and how they
+    compare to your proposed feature.
+
+  - Briefly describe how your suggested feature **conforms with Instaloader's
+    project goal**.
+
+- Explain your **solution ideas**. Describe your ideas on how the feature could
+  be implemented and the underlying problem could be solved. Also **describe
+  alternatives** that you have considered.
 
 Donations
 ---------

docs/troubleshooting.rst

@@ -27,11 +27,25 @@ expired. In case the retry continuously fails for some reason, which should not
 happen in normal conditions, consider adjusting the
 :option:`--max-connection-attempts` option.
 
+There have been observations that services, that in their nature offer
+promiscious IP addresses, such as cloud, VPN and public proxy services, might be
+subject to significantly stricter limits for anonymous access. However,
+logged-in accesses (see :option:`--login`) do not seem to be affected.
+
+Too many queries in the last time
+---------------------------------
+
 **"Too many queries in the last time"** is not an error. It is a notice that the
 rate limit has almost been reached, according to Instaloader's own rate
 accounting mechanism. We regularly adjust this mechanism to match Instagram's
 current rate limiting.
 
+Private but not followed
+------------------------
+
+You have to follow a private account to access most of its associated
+information.
+
 Login Error
 -----------
 

instaloader/__init__.py

@@ -1,7 +1,7 @@
 """Download pictures (or videos) along with their captions and other metadata from Instagram."""
 
 
-__version__ = '4.4.4rc1'
+__version__ = '4.4.5rc1'
 
 
 try:

instaloader/__main__.py

@@ -3,7 +3,6 @@
 import ast
 import datetime
 import os
-import platform
 import re
 import sys
 from argparse import ArgumentParser, SUPPRESS
@@ -16,20 +15,10 @@ from .instaloader import get_default_session_filename
 from .instaloadercontext import default_user_agent
 
 
-def get_simple_argv0() -> str:
-    if platform.system() != "Windows":
-        argv0 = os.path.basename(sys.argv[0])
-    else:
-        dirname = os.path.relpath(os.path.dirname(sys.argv[0]))
-        basename = os.path.basename(sys.argv[0])
-        argv0 = os.path.join(dirname, basename)
-    argv0 = "instaloader" if argv0 == "__main__.py" else argv0
-    return argv0
-
-
 def usage_string():
     # NOTE: duplicated in README.rst and docs/index.rst
-    argv0 = get_simple_argv0()
+    argv0 = os.path.basename(sys.argv[0])
+    argv0 = "instaloader" if argv0 == "__main__.py" else argv0
     return """
 {0} [--comments] [--geotags]
 {2:{1}} [--stories] [--highlights] [--tagged] [--igtv]
@@ -93,6 +82,8 @@ def _main(instaloader: Instaloader, targetlist: List[str],
         instaloader.context.log('Only download storyitems with property "{}".'.format(storyitem_filter_str))
     # Login, if desired
     if username is not None:
+        if not re.match(r"^[A-Za-z0-9._]+$", username):
+            instaloader.context.error("Warning: Parameter \"{}\" for --login is not a valid username.".format(username))
         try:
             instaloader.load_session_from_file(username, sessionfile)
         except FileNotFoundError as err:
@@ -207,7 +198,7 @@ def _main(instaloader: Instaloader, targetlist: List[str],
         if len(profiles) > 1:
             instaloader.context.log("Downloading {} profiles: {}".format(len(profiles),
                                                                          ' '.join([p.username for p in profiles])))
-        if ((profiles and download_profile_pic) or download_posts) and not instaloader.context.is_logged_in:
+        if profiles and (download_profile_pic or download_posts) and not instaloader.context.is_logged_in:
             instaloader.context.error("Warning: Use --login to download higher-quality versions of pictures.")
         instaloader.download_profiles(profiles,
                                       download_profile_pic, download_posts, download_tagged, download_igtv,

instaloader/instaloader.py

@@ -393,12 +393,12 @@ class Instaloader:
         if ((format_string_contains_key(self.dirname_pattern, 'profile') or
              format_string_contains_key(self.dirname_pattern, 'target'))):
             profile_str = owner_profile.username.lower() if owner_profile is not None else target
-            filename = '{0}/{1}_{2}.{3}'.format(self.dirname_pattern.format(profile=profile_str,
-                                                                            target=target),
-                                                pic_identifier, name_suffix, pic_extension)
+            filename = os.path.join(self.dirname_pattern.format(profile=profile_str,
+                                                                target=target),
+                                    '{0}_{1}.{2}'.format(pic_identifier, name_suffix, pic_extension))
         else:
-            filename = '{0}/{1}_{2}_{3}.{4}'.format(self.dirname_pattern.format(), target,
-                                                    pic_identifier, name_suffix, pic_extension)
+            filename = os.path.join(self.dirname_pattern.format(),
+                                    '{0}_{1}_{2}.{3}'.format(target, pic_identifier, name_suffix, pic_extension))
         content_length = http_response.headers.get('Content-Length', None)
         if os.path.isfile(filename) and (not self.context.is_logged_in or
                                          (content_length is not None and
@@ -501,7 +501,7 @@ class Instaloader:
         """
 
         dirname = _PostPathFormatter(post).format(self.dirname_pattern, target=target)
-        filename = dirname + '/' + self.format_filename(post, target=target)
+        filename = os.path.join(dirname, self.format_filename(post, target=target))
         os.makedirs(os.path.dirname(filename), exist_ok=True)
 
         # Download the image(s) / video thumbnail and videos within sidecars if desired
@@ -630,7 +630,7 @@ class Instaloader:
 
         date_local = item.date_local
         dirname = _PostPathFormatter(item).format(self.dirname_pattern, target=target)
-        filename = dirname + '/' + self.format_filename(item, target=target)
+        filename = os.path.join(dirname, self.format_filename(item, target=target))
         os.makedirs(os.path.dirname(filename), exist_ok=True)
         downloaded = False
         if not item.is_video or self.download_video_thumbnails is True:
@@ -974,10 +974,12 @@ class Instaloader:
     def _get_id_filename(self, profile_name: str) -> str:
         if ((format_string_contains_key(self.dirname_pattern, 'profile') or
              format_string_contains_key(self.dirname_pattern, 'target'))):
-            return '{0}/id'.format(self.dirname_pattern.format(profile=profile_name.lower(),
-                                                               target=profile_name.lower()))
+            return os.path.join(self.dirname_pattern.format(profile=profile_name.lower(),
+                                                            target=profile_name.lower()),
+                                'id')
         else:
-            return '{0}/{1}_id'.format(self.dirname_pattern.format(), profile_name.lower())
+            return os.path.join(self.dirname_pattern.format(),
+                                '{0}_id'.format(profile_name.lower()))
 
     def save_profile_id(self, profile: Profile):
         """
@@ -1091,9 +1093,9 @@ class Instaloader:
 
                 # Save metadata as JSON if desired.
                 if self.save_metadata:
-                    json_filename = '{0}/{1}_{2}'.format(self.dirname_pattern.format(profile=profile_name,
-                                                                                     target=profile_name),
-                                                         profile_name, profile.userid)
+                    json_filename = os.path.join(self.dirname_pattern.format(profile=profile_name,
+                                                                             target=profile_name),
+                                                 '{0}_{1}'.format(profile_name, profile.userid))
                     self.save_metadata_json(json_filename, profile)
 
                 # Catch some errors

instaloader/structures.py

@@ -69,7 +69,6 @@ class Post:
         self._node = node
         self._owner_profile = owner_profile
         self._full_metadata_dict = None  # type: Optional[Dict[str, Any]]
-        self._rhx_gis_str = None         # type: Optional[str]
         self._location = None            # type: Optional[PostLocation]
         self._iphone_struct_ = None
         if 'iphone_struct' in node:
@@ -142,9 +141,11 @@ class Post:
 
     def _obtain_metadata(self):
         if not self._full_metadata_dict:
-            pic_json = self._context.get_json("p/{0}/".format(self.shortcode), params={})
-            self._full_metadata_dict = pic_json['entry_data']['PostPage'][0]['graphql']['shortcode_media']
-            self._rhx_gis_str = pic_json.get('rhx_gis')
+            pic_json = self._context.graphql_query(
+                '2b0673e0dc4580674a88d426fe00ea90',
+                {'shortcode': self.shortcode}
+            )
+            self._full_metadata_dict = pic_json['data']['shortcode_media']
             if self._full_metadata_dict is None:
                 # issue #449
                 self._context.error("Fetching Post metadata failed (issue #449). "
@@ -161,11 +162,6 @@ class Post:
         assert self._full_metadata_dict is not None
         return self._full_metadata_dict
 
-    @property
-    def _rhx_gis(self) -> Optional[str]:
-        self._obtain_metadata()
-        return self._rhx_gis_str
-
     @property
     def _iphone_struct(self) -> Dict[str, Any]:
         if not self._context.is_logged_in:
@@ -242,7 +238,9 @@ class Post:
         """URL of the picture / video thumbnail of the post"""
         if self.typename == "GraphImage" and self._context.is_logged_in:
             try:
-                return self._iphone_struct['image_versions2']['candidates'][0]['url']
+                orig_url = self._iphone_struct['image_versions2']['candidates'][0]['url']
+                url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
+                return url
             except (InstaloaderException, KeyError, IndexError) as err:
                 self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
         return self._node["display_url"] if "display_url" in self._node else self._node["display_src"]
@@ -266,7 +264,8 @@ class Post:
                 if not is_video and self._context.is_logged_in:
                     try:
                         carousel_media = self._iphone_struct['carousel_media']
-                        display_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
+                        orig_url = carousel_media[idx]['image_versions2']['candidates'][0]['url']
+                        display_url = re.sub(r'&se=\d+(&?)', r'\1', orig_url)
                     except (InstaloaderException, KeyError, IndexError) as err:
                         self._context.error('{} Unable to fetch high quality image version of {}.'.format(err, self))
                 yield PostSidecarNode(is_video=is_video, display_url=display_url,
@@ -389,7 +388,7 @@ class Post:
                                      created_at_utc=datetime.utcfromtimestamp(node['created_at']),
                                      text=node['text'],
                                      owner=Profile(self._context, node['owner']),
-                                     likes_count=node['edge_liked_by']['count'])
+                                     likes_count=node.get('edge_liked_by', {}).get('count', 0))
 
         def _postcommentanswers(node):
             if 'edge_threaded_comments' not in node:
@@ -415,14 +414,9 @@ class Post:
         if self.comments == 0:
             # Avoid doing additional requests if there are no comments
             return
-        try:
-            comment_edges = self._field('edge_media_to_parent_comment', 'edges')
-            answers_count = sum([edge['node']['edge_threaded_comments']['count'] for edge in comment_edges])
-            threaded_comments_available = True
-        except KeyError:
-            comment_edges = self._field('edge_media_to_comment', 'edges')
-            answers_count = 0
-            threaded_comments_available = False
+
+        comment_edges = self._field('edge_media_to_comment', 'edges')
+        answers_count = sum([edge['node'].get('edge_threaded_comments', {}).get('count', 0) for edge in comment_edges])
 
         if self.comments == len(comment_edges) + answers_count:
             # If the Post's metadata already contains all parent comments, don't do GraphQL requests to obtain them
@@ -430,14 +424,10 @@ class Post:
             return
         yield from (_postcomment(node) for node in
                     self._context.graphql_node_list(
-                        "97b41c52301f77ce508f55e66d17620e" if threaded_comments_available
-                        else "f0986789a5c5d17c2400faebf16efd0d",
+                        "97b41c52301f77ce508f55e66d17620e",
                         {'shortcode': self.shortcode},
                         'https://www.instagram.com/p/' + self.shortcode + '/',
-                        lambda d:
-                        d['data']['shortcode_media'][
-                            'edge_media_to_parent_comment' if threaded_comments_available else 'edge_media_to_comment'],
-                        self._rhx_gis))
+                        lambda d: d['data']['shortcode_media']['edge_media_to_parent_comment']))
 
     def get_likes(self) -> Iterator['Profile']:
         """Iterate over all likes of the post. A :class:`Profile` instance of each likee is yielded."""
@@ -452,8 +442,7 @@ class Post:
         yield from (Profile(self._context, node) for node in
                     self._context.graphql_node_list("1cb6ec562846122743b61e492c85999f", {'shortcode': self.shortcode},
                                                     'https://www.instagram.com/p/' + self.shortcode + '/',
-                                                    lambda d: d['data']['shortcode_media']['edge_liked_by'],
-                                                    self._rhx_gis))
+                                                    lambda d: d['data']['shortcode_media']['edge_liked_by']))
 
     @property
     def is_sponsored(self) -> bool:
@@ -534,7 +523,6 @@ class Profile:
         self._has_public_story = None  # type: Optional[bool]
         self._node = node
         self._has_full_metadata = False
-        self._rhx_gis = None
         self._iphone_struct_ = None
         if 'iphone_struct' in node:
             # if loaded from JSON with load_structure_from_file()
@@ -596,10 +584,9 @@ class Profile:
     def _obtain_metadata(self):
         try:
             if not self._has_full_metadata:
-                metadata = self._context.get_json('{}/'.format(self.username), params={})
+                metadata = self._context.get_json('{}/feed/'.format(self.username), params={})
                 self._node = metadata['entry_data']['ProfilePage'][0]['graphql']['user']
                 self._has_full_metadata = True
-                self._rhx_gis = metadata.get('rhx_gis')
         except (QueryReturnedNotFoundException, KeyError) as err:
             top_search_results = TopSearchResults(self._context, self.username)
             similar_profiles = [profile.username for profile in top_search_results.get_profiles()]
@@ -732,8 +719,7 @@ class Profile:
                                                         'include_reel': False, 'include_suggested_users': False,
                                                         'include_logged_out_extras': True,
                                                         'include_highlight_reels': False},
-                                                       'https://www.instagram.com/{}/'.format(self.username),
-                                                       self._rhx_gis)
+                                                       'https://www.instagram.com/{}/'.format(self.username))
             self._has_public_story = data['data']['user']['has_public_story']
         assert self._has_public_story is not None
         return self._has_public_story
@@ -792,8 +778,7 @@ class Profile:
                                                     {'id': self.userid},
                                                     'https://www.instagram.com/{0}/'.format(self.username),
                                                     lambda d: d['data']['user']['edge_owner_to_timeline_media'],
-                                                    self._rhx_gis,
-                                                    self._metadata('edge_owner_to_timeline_media')))
+                                                    first_data=self._metadata('edge_owner_to_timeline_media')))
 
     def get_saved_posts(self) -> Iterator[Post]:
         """Get Posts that are marked as saved by the user."""
@@ -807,8 +792,7 @@ class Profile:
                                                     {'id': self.userid},
                                                     'https://www.instagram.com/{0}/'.format(self.username),
                                                     lambda d: d['data']['user']['edge_saved_media'],
-                                                    self._rhx_gis,
-                                                    self._metadata('edge_saved_media')))
+                                                    first_data=self._metadata('edge_saved_media')))
 
     def get_tagged_posts(self) -> Iterator[Post]:
         """Retrieve all posts where a profile is tagged.
@@ -819,8 +803,7 @@ class Profile:
                     self._context.graphql_node_list("e31a871f7301132ceaab56507a66bbb7",
                                                     {'id': self.userid},
                                                     'https://www.instagram.com/{0}/'.format(self.username),
-                                                    lambda d: d['data']['user']['edge_user_to_photos_of_you'],
-                                                    self._rhx_gis))
+                                                    lambda d: d['data']['user']['edge_user_to_photos_of_you']))
 
     def get_igtv_posts(self) -> Iterator[Post]:
         """Retrieve all IGTV posts.
@@ -832,8 +815,7 @@ class Profile:
                                                     {'id': self.userid},
                                                     'https://www.instagram.com/{0}/channel/'.format(self.username),
                                                     lambda d: d['data']['user']['edge_felix_video_timeline'],
-                                                    self._rhx_gis,
-                                                    self._metadata('edge_felix_video_timeline')))
+                                                    first_data=self._metadata('edge_felix_video_timeline')))
 
     def get_followers(self) -> Iterator['Profile']:
         """
@@ -847,8 +829,7 @@ class Profile:
                     self._context.graphql_node_list("37479f2b8209594dde7facb0d904896a",
                                                     {'id': str(self.userid)},
                                                     'https://www.instagram.com/' + self.username + '/',
-                                                    lambda d: d['data']['user']['edge_followed_by'],
-                                                    self._rhx_gis))
+                                                    lambda d: d['data']['user']['edge_followed_by']))
 
     def get_followees(self) -> Iterator['Profile']:
         """
@@ -862,8 +843,7 @@ class Profile:
                     self._context.graphql_node_list("58712303d941c6855d4e888c5f0cd22f",
                                                     {'id': str(self.userid)},
                                                     'https://www.instagram.com/' + self.username + '/',
-                                                    lambda d: d['data']['user']['edge_follow'],
-                                                    self._rhx_gis))
+                                                    lambda d: d['data']['user']['edge_follow']))
 
     def get_similar_accounts(self) -> Iterator['Profile']:
         """
@@ -878,8 +858,8 @@ class Profile:
         yield from (Profile(self._context, edge["node"]) for edge in
                     self._context.graphql_query("ad99dd9d3646cc3c0dda65debcd266a7",
                                                 {"user_id": str(self.userid), "include_chaining": True},
-                                                "https://www.instagram.com/{0}/".format(self.username),
-                                                self._rhx_gis)["data"]["user"]["edge_chaining"]["edges"])
+                                                "https://www.instagram.com/{0}/"
+                                                .format(self.username))["data"]["user"]["edge_chaining"]["edges"])
 
 
 class StoryItem:

test/instaloader_unittests.py

@@ -94,6 +94,10 @@ class TestInstaloaderAnonymously(unittest.TestCase):
         self.assertEqual(EMPTY_PROFILE.lower(),
                          instaloader.Profile.from_id(self.L.context, EMPTY_PROFILE_ID).username)
 
+    def test_get_username_by_name_empty(self):
+        self.assertEqual(EMPTY_PROFILE_ID,
+                         instaloader.Profile.from_username(self.L.context, EMPTY_PROFILE).userid)
+
     def test_post_from_mediaid(self):
         for post in instaloader.Profile.from_username(self.L.context, PUBLIC_PROFILE).get_posts():
             post2 = instaloader.Post.from_mediaid(self.L.context, post.mediaid)