ipapi.gr

Bot signals — Search & SEO crawler

CCBot IP addresses

Common Crawl CCBot

CCBot

risk.is_crawler Ranges Reviewed
Official documentation (opens in a new tab) All search & SEO crawlers

Check an IP

Is this IP really CCBot?

Checks the address against what Common Crawl CCBot publishes, the same evidence behind risk.is_crawler.

01 — Verification

How IPAPI.GR verifies CCBot.

Only from what Common Crawl CCBot publishes about itself. No third-party blocklists, no behavioural guesses. Live address counts change with every publication, so check the official file for the current list.

02 — In the response

What a match looks like.

A lookup of a CCBot address sets risk.is_crawler. The flags come with geolocation and network data in the same response, for Pro API keys.

The response says which kind of bot an address belongs to, not which operator. Pair it with the user agent: a request that claims to be CCBot but doesn't match is unverified.

GET /api/lookup/{ip}?key=PRO_KEY
{
  // … location and network fields …
  "risk": {
    "is_ai": false,
    "is_crawler": true,
    "is_scanner": false
  }
}

03 — Configuration

Allow, block or verify it yourself.

robots.txt
User-agent: CCBot
Disallow: /
Common Crawl's CCBot follows robots.txt.
nginx + jq
# 1. Build a geo include from the official file (cron, e.g. hourly).
#    Keep the old file if the download fails or comes back empty.
curl -fsS https://index.commoncrawl.org/ccbot.json \
  | jq -r '.prefixes[] | (.ipv4Prefix // .ipv6Prefix // empty) | "\(.) 1;"' \
  > /etc/nginx/ccbot-ranges.conf.new && [ -s /etc/nginx/ccbot-ranges.conf.new ] && mv /etc/nginx/ccbot-ranges.conf.new /etc/nginx/ccbot-ranges.conf \
  && nginx -t -q && nginx -s reload

# 2. In the http block: 1 when the client IP is in the published ranges.
geo $ccbot_ip {
    default 0;
    include /etc/nginx/ccbot-ranges.conf;
}

# 3. A request that claims to be CCBot from any other address is an impostor.
map "$http_user_agent:$ccbot_ip" $fake_ccbot {
    default 0;
    "~*(CCBot).*:0$" 1;
}

# In the server block:
if ($fake_ccbot) { return 403; }
Built from index.commoncrawl.org/ccbot.json, a JSON file with a prefixes array.

04 — Questions

About CCBot.

Where does CCBot publish its IP addresses?

Common Crawl CCBot publishes its addresses at https://index.commoncrawl.org/ccbot.json. IPAPI.GR reads that publication directly, not a third-party copy.

How often is the CCBot list refreshed?

IPAPI.GR re-fetches it every hour. If a refresh fails, the previous entries expire after at most 24 hours instead of being trusted forever.

Which risk flag does CCBot set?

A matching address sets risk.is_crawler in the lookup response. The risk flags are returned for Pro API keys.

From the blog

Go deeper.

Related operators

Every operator on the coverage page →

Verify CCBot on every request.

Pro: 40,000,000 requests a day with is_ai, is_crawler and is_scanner in every lookup.

See Pro Start free