Google special crawlers
risk.is_crawler
- Method
- Ranges Published IP ranges
- Official file
- Format
- JSON range file
- Refresh
- Every hour. If a refresh fails, entries expire after 24 hours instead of being trusted forever.
Bot signals — Search & SEO crawler
Google special crawlers IP addresses
Google-operated special-case crawlers, including advertising validation.
01 — Verification
How IPAPI.GR verifies Google special crawlers.
Only from what Google special crawlers publishes about itself. No third-party blocklists, no behavioural guesses. Live address counts change with every publication, so check the official file for the current list.
02 — In the response
What a match looks like.
A lookup of a Google special crawlers address sets risk.is_crawler. The flags come with geolocation and network data in the same response, for Pro API keys.
The response says which kind of bot an address belongs to, not which operator. Pair it with the user agent: a request that claims to be Google special crawlers but doesn't match is unverified.
{
// … location and network fields …
"risk": {
"is_ai": false,
"is_crawler": true,
"is_scanner": false
}
}
03 — Configuration
Allow, block or verify it yourself.
# 1. Build a geo include from the official file (cron, e.g. hourly).
# Keep the old file if the download fails or comes back empty.
curl -fsS https://developers.google.com/static/crawling/ipranges/special-crawlers.json \
| jq -r '.prefixes[] | (.ipv4Prefix // .ipv6Prefix // empty) | "\(.) 1;"' \
> /etc/nginx/google-special-crawlers-ranges.conf.new && [ -s /etc/nginx/google-special-crawlers-ranges.conf.new ] && mv /etc/nginx/google-special-crawlers-ranges.conf.new /etc/nginx/google-special-crawlers-ranges.conf \
&& nginx -t -q && nginx -s reload
# 2. In the http block: 1 when the client IP is in the published ranges.
geo $google_special_crawlers_ip {
default 0;
include /etc/nginx/google-special-crawlers-ranges.conf;
}
prefixes array.04 — Questions
About Google special crawlers.
Where does Google special crawlers publish its IP addresses?
Google special crawlers publishes its addresses at https://developers.google.com/static/crawling/ipranges/special-crawlers.json. IPAPI.GR reads that publication directly, not a third-party copy.
How often is the Google special crawlers list refreshed?
IPAPI.GR re-fetches it every hour. If a refresh fails, the previous entries expire after at most 24 hours instead of being trusted forever.
Which risk flag does Google special crawlers set?
A matching address sets risk.is_crawler in the lookup response. The risk flags are returned for Pro API keys.
Related operators
Verify Google special crawlers on every request.
Pro: 40,000,000 requests a day with is_ai, is_crawler and is_scanner in every lookup.