The GraphQL query used so far now returns edge_web_feed_timeline: null, which
raised a TypeError. Page through api/v1/feed/timeline/ instead and convert its
media items (iPhone format).
Fixes#2556
- `InstaloaderContext.get_page_data()` loads a page as a logged-out visitor and returns the query results that the page embeds. Instagram only embeds them when the request looks like a page navigation, so the request sends the `Sec-Fetch-*` headers of one.
- `Profile.from_username()` reads the profile from its public page. Anonymously, `_normalize_profile_data()` completes the node.
- Logged in, `Profile.get_posts()` uses query `28975909992013618` (`PolarisProfilePostsTabContentQuery_connection`) with the variables that the web app sends. The response has the same shape as before.
When logged in, `from_username()` does not normalize the page's node. The page shows the profile to a logged-out visitor, so normalizing would set `followed_by_viewer` to `False`, and `download_profiles()` would refuse private profiles that the session follows. Instead, the GraphQL profile query in `_obtain_metadata()` fills in the missing fields when they are first read.
Use doc_id 27128499623469141 (PolarisPostRootQuery /
xdt_api__v1__media__shortcode__web_info) which returns v1/iPhone API format. Converts the new format to the legacy GraphQL field names that the rest of the codebase expects.
Fix Profile.from_id(), Profile.from_username() and Profile._obtain_metadata() by using `users/web_profile_info/` and `users/{profile_id}/info` queries.
_mention_regex required the character before '@' to be a non-word
character other than '\n', so a mention at the start of a new line was
never matched. The '\n' exclusion was introduced in 8784ac7 when the
prefix went from '\W' to '[^\w\n]'; restoring '\W' matches mentions
after a newline while still keeping email addresses out of the results,
since the character before '@' in an address is a word character.
Also affects Profile.biography_mentions and StoryItem.caption_mentions,
which share the same regex.
Fixes#2709
When Instagram returns {"data": null} for a shortcode metadata query (e.g. under
rate limiting, or for an unavailable post), response["data"]["xdt_shortcode_media"]
raised a raw TypeError before the existing `if pic_json is None` check could run.
Guard the intermediate "data" so the clean BadResponseException is raised instead,
which the download loop already handles.
Use doc_id 26347858941511777 (search query) in from_username() to get user_id
Use doc_id 25980296051578533 (PolarisProfilePageContentQuery) in _obtain_metadata()
Add _normalize_profile_data() to map new API fields to legacy format
Handle Instagram feedback_required/checkpoint_required/challenge_required responses with a AbortDownloadException, causing the resumable iteration to be interrupted.
Co-authored-by: Alexander Graf <17130992+aandergr@users.noreply.github.com>
Alexander Graf