Instagram scraper in Python: profiles, posts, comments and search

You can scrape public Instagram profiles, posts, comments and keyword search with one Python file and no login: toolzerhub/instagram-scraper on GitHub. It is 345 lines, depends on one package (curl_cffi) and printed live data from all four commands when we ran it on 2026-10-13. It does not return follower lists or a user's posts, and Instagram refused it from our datacenter server. We make ToolzerHub: if you need followers, following or a user's posts, our Instagram API returns them, and the API section below shows what one call returns.

Install the script and run the four commands

Clone the repo, install curl_cffi and run profile on any public username. We ran everything below on Python 3.12.13 with curl_cffi 0.16.3.

git clone https://github.com/toolzerhub/instagram-scraper.git
cd instagram-scraper
pip install -r requirements.txt
python instagram_scraper.py profile natgeo

The output, trimmed (2026-10-13; the trailing emoji in the biography is left out):

{
  "id": "787132",
  "username": "natgeo",
  "full_name": "National Geographic",
  "biography": "Step into wonder and find your inner explorer with National Geographic",
  "external_url": "http://visitstore.bio/natgeo",
  "bio_links": ["http://visitstore.bio/natgeo", "https://ngmdomsubs.nationalgeographic.com/..."],
  "category": null,
  "is_business": false,
  "is_private": false,
  "is_verified": true,
  "followers_count": 268396518,
  "following_count": 194,
  "profile_pic_url": "https://scontent.cdninstagram.com/..."
}

The other three commands take a post URL or shortcode (the part after /p/ or /reel/) or a keyword. Every command accepts -o file.json to save the result and --proxy URL to route through a proxy.

python instagram_scraper.py post https://www.instagram.com/reel/Dd6-VBbhtY6/
python instagram_scraper.py comments Dd6-VBbhtY6 --limit 100 -o comments.json
python instagram_scraper.py search "national park" --limit 24 -o search.json

What each command returned

All four commands returned live data in 2.6 to 9.0 seconds each, except that comments stopped at 185 of the 258 the post reports. We ran each once on 2026-10-13 through a residential proxy and timed it, including the session setup.

• profile natgeo took 7.3 seconds and returned the fields above. The follower count moved between our runs (268,396,513 to 268,396,520), so it is live data.
• post Dd6-VBbhtY6 took 2.6 seconds. It returned a reel about national parks with 5,985 likes, 258 comments, the caption, a timestamp and signed image and video URLs.
• comments Dd6-VBbhtY6 --limit 100 took 8.7 seconds and returned 100 comments, two pages of 50. With --limit 1000 we got 185 comments in 8.8 seconds, then paging stopped, although the post reports 258. Every one of the 185 had reply_count 0, so the file holds top-level comments only. We don't know why the other 73 never came back.
• search "national park" --limit 24 took 9.0 seconds and returned 24 posts from 24 different accounts, all videos, with play_count from 1,686 to 8,268,768 (median about 115,600).
• profile for 10 accounts in one command (natgeo, nasa, smithsonian, noaa and six more) returned 10 of 10 in 34.6 seconds.

The signed image_url, video_url and profile_pic_url values expire, so download media soon after you scrape it.

How the script gets the data

The script sends requests modelled on the ones Instagram's own website sends to a visitor who isn't signed in.

1. It opens instagram.com once and keeps the cookies and page tokens (lsd, jazoest) that the site's JavaScript sends back with every query. It sends requests with curl_cffi, an HTTP client built on curl.
2. profile fetches the profile page that Instagram serves to iPhone Safari. That page embeds the profile as JSON, and the script walks every <script type="application/json"> blob until it finds a dict with username, pk and follower_count. It then asks the site's PolarisProfilePageContentQuery by numeric id for bio links and category. If that second call fails, you still get the basic fields.
3. post sends PolarisPostRootQuery, comments sends PolarisLoggedOutDesktopWWWPostCommentsPaginationQuery and search sends PolarisLoggedOutPopularSearchPageQuery.

Comments need the numeric media id, not the shortcode. The shortcode is that id written in a 64-character alphabet, so the script decodes it:

def media_id_of(shortcode):
    alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"
    n = 0
    for ch in shortcode:
        n = n * 64 + alphabet.index(ch)
    return str(n)

media_id_of("Dd6-VBbhtY6") returns 3997781742772999738, the same id the post command printed.

Export a list of profiles to CSV

Instagram is a class you can import, so a list of usernames becomes a CSV in a few lines. This script sits next to instagram_scraper.py, reuses one session for the whole list, waits 2 seconds between profiles and reports the ones that fail. It is the version we ran.

import csv
import os
import sys
import time

from instagram_scraper import Instagram, ScrapeError

FIELDS = ["username", "full_name", "followers_count", "following_count", "is_verified", "category", "external_url"]


def main(path, usernames):
    ig = Instagram(os.environ.get("HTTPS_PROXY"))  # one session for the whole list
    done = 0
    with open(path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
        writer.writeheader()
        for name in usernames:
            try:
                writer.writerow(ig.profile(name))
                done += 1
            except ScrapeError as exc:
                print(f"{name}: {exc}", file=sys.stderr)
            time.sleep(2)  # pause between profiles
    print(f"wrote {done} of {len(usernames)} profiles to {path}")


if __name__ == "__main__":
    main(sys.argv[1], sys.argv[2:])

python profiles_to_csv.py profiles.csv natgeo nasa nationalparkservice noaa smithsonian discovery zz_no_such_user_88123

Output on 2026-10-13, after 36 seconds:

zz_no_such_user_88123: No profile data on the page for zz_no_such_user_88123. The user may not exist.
wrote 6 of 7 profiles to profiles.csv

username,full_name,followers_count,following_count,is_verified,category,external_url
natgeo,National Geographic,268396513,194,True,,http://visitstore.bio/natgeo
nasa,NASA,104341563,90,True,,https://www.nasa.gov
nationalparkservice,National Park Service,7218966,602,True,Government organization,http://nps.gov
noaa,NOAA,917172,53,True,Government organization,http://www.noaa.gov
smithsonian,Smithsonian,1202952,223,True,,https://sprout.link/smithsonian
discovery,Discovery,19741243,936,True,,http://discovery.com/

The failed name is skipped and the rest are kept. From a server, set HTTPS_PROXY first (see the next section).

Where the script gets blocked

The repo says Instagram blocks most datacenter and cloud IPs, so run the script from a home connection or through a residential proxy. Our test was one server: on 2026-10-13 we ran profile natgeo on our server with no proxy and got, after 1.6 seconds:

Error: Instagram answered HTTP 302 for natgeo: blocked, or the user does not exist.

Through a residential proxy the same server returned all 10 profiles in the sweep above. The proxy costs are yours. We later ran this script, Instaloader, Scout and our API on the same 10 profiles; see the best Instagram scrapers we tested.

Two more limits to plan for. The four DOC_* ids at the top of the script name the queries the site sends, and the repo's README says Instagram changes them every few months. We have not watched one change; if a command that used to work starts failing, update the id first. The script also has no retries and no concurrency on purpose, so a network timeout surfaces as an error and you rerun that name.

Check the rules before you scrape

Instagram's Terms of Use say you can't attempt to "access or collect information in unauthorized ways", which "includes creating accounts or accessing or collecting information in an automated way without our express permission, regardless of whether such automated access or collection is undertaken while logged-in to an Instagram account". The script reads only what any visitor can see, and the terms still apply to it. Profiles and comments are personal data (usernames, free text), so privacy law where you or the people in the data live may apply: collect only what you need and delete it when you are done.

If the data is yours, Meta's own API is the route Meta supports. Its comments endpoint, for example, needs "An access token from a User who created the IG Media object", so it works for your posts and not for anyone else's. Personal accounts have no official API at all since Meta shut the Basic Display API in December 2024; see what replaced the Instagram Basic Display API.

What the script can't do and the API calls that do

The script has no follower list, no following list and no list of a user's posts. Our Instagram API returns them as JSON from any server with one API key, and you run no proxy. On 2026-10-13 we called these endpoints on public accounts:

Each of these is 1 credit per call. For a user's posts, reels and comments we ran 10 live calls each on 2026-10-13, and for following, similar accounts and hashtags 10 each the same day: all 60 returned a success response.

A user's posts, as a request and a trimmed response:

curl -H "x-api-key: $TOOLZERHUB_API_KEY" \
  "https://api.toolzerhub.com/v1/instagram/user/posts?username_or_url=nasa"

{
  "data": {
    "num_results": 12,
    "more_available": true,
    "next_max_id": "...",
    "items": [
      { "code": "DeJ28eTxo4s", "taken_at": 1791292211, "product_type": "feed", "like_count": 237615, "comment_count": 731 },
      { "code": "DeCPVhERA8e", "taken_at": 1791036618, "product_type": "clips", "like_count": 225058, "comment_count": 2337 }
    ]
  }
}

That call took 6.0 seconds and returned 12 posts; the real items carry many more fields, including the caption. Two limits from the same day: very large accounts get a short followers list (nasa returned 48 users with no next page), and the comments endpoint returned 15 comments a call, so a post with thousands of comments takes hundreds of calls. For a complete comments export, see export Instagram comments to CSV, and for the follower paging loop and its limits on big accounts, see the Instagram API followers list.

What the API costs

A free key gives you 100 credits a month with no card: get one here. Paid plans are $29 for 20,000 credits ($1.45 per 1,000), $79 for 60,000 ($1.32), $199 for 165,000 ($1.21) and $495 for 500,000 ($0.99), so a 1-credit call costs $0.99 to $1.45 per 1,000 calls. Exporting 1,000 followers at about 20 per page is roughly 50 calls: 50 credits, or 5 to 7 cents. Comments at about 15 per call are roughly 67 calls per 1,000 comments, 7 to 10 cents. Some endpoints cost 3 credits; each docs page shows its exact cost before you call it. Check the pricing page for current rates.

Followers succeeded on 98.7% of 1,239 calls over the 7 days to 2026-10-13, counting a call as failed only on a 5xx error. The posts, reels and comments endpoints see little production traffic yet, which is why we ran the 10-call checks above.

We are not affiliated with Instagram or Meta Platforms, Inc.

Sources

• toolzerhub/instagram-scraper, source and README (commit b6b953e), https://github.com/toolzerhub/instagram-scraper, run 2026-10-13
• Instagram Terms of Use, https://help.instagram.com/581066165581870, read 2026-10-13
• Meta, Instagram Platform IG Media comments reference (page updated 2025-01-21), https://developers.facebook.com/documentation/instagram-platform/instagram-graph-api/reference/ig-media/comments, read 2026-10-12
• ToolzerHub pricing, https://toolzerhub.com/pricing, checked 2026-10-13
• ToolzerHub API OpenAPI spec (credits per endpoint), https://api.toolzerhub.com/openapi.json, read 2026-10-13