Bot signals — Search & SEO crawler
Google-operated user-triggered fetchers only. Excludes the separate customer-controlled Google Cloud fetcher feed.
01 — Verification
Only from what Google-operated user-triggered fetchers publishes about itself. No third-party blocklists, no behavioural guesses. Live address counts change with every publication, so check the official file for the current list.
risk.is_crawler
02 — In the response
A lookup of a Google-operated user-triggered fetchers address sets risk.is_crawler. The flags come with geolocation and network data in the same response, for Pro API keys.
The response says which kind of bot an address belongs to, not which operator. Pair it with the user agent: a request that claims to be Google-operated user-triggered fetchers but doesn't match is unverified.
{
// … location and network fields …
"risk": {
"is_ai": false,
"is_crawler": true,
"is_scanner": false
}
}
03 — Configuration
# 1. Build a geo include from the official file (cron, e.g. hourly).
# Keep the old file if the download fails or comes back empty.
curl -fsS https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json \
| jq -r '.prefixes[] | (.ipv4Prefix // .ipv6Prefix // empty) | "\(.) 1;"' \
> /etc/nginx/google-user-triggered-fetchers-ranges.conf.new && [ -s /etc/nginx/google-user-triggered-fetchers-ranges.conf.new ] && mv /etc/nginx/google-user-triggered-fetchers-ranges.conf.new /etc/nginx/google-user-triggered-fetchers-ranges.conf \
&& nginx -t -q && nginx -s reload
# 2. In the http block: 1 when the client IP is in the published ranges.
geo $google_user_triggered_fetchers_ip {
default 0;
include /etc/nginx/google-user-triggered-fetchers-ranges.conf;
}
prefixes array.04 — Questions
Google-operated user-triggered fetchers publishes its addresses at https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json. IPAPI.GR reads that publication directly, not a third-party copy.
IPAPI.GR re-fetches it every hour. If a refresh fails, the previous entries expire after at most 24 hours instead of being trusted forever.
A matching address sets risk.is_crawler in the lookup response. The risk flags are returned for Pro API keys.
Pro: 40,000,000 requests a day with is_ai, is_crawler and is_scanner in every lookup.