Bot signals — Search & SEO crawler
Only the crawler-specific IPv4-encoded hostname convention demonstrated in Baidu's official forward/reverse DNS example. Does not classify all baidu.com or Baidu ERNIE.
01 — Verification
Only from what Baiduspider publishes about itself. No third-party blocklists, no behavioural guesses.
risk.is_crawler
baiduspider-a-b-c-d.crawl.baidu.com, where a-b-c-d is the IPv4 address itself.
The hostname must then resolve forward to the same address.
02 — In the response
A lookup of a Baiduspider address sets risk.is_crawler. The flags come with geolocation and network data in the same response, for Pro API keys.
The response says which kind of bot an address belongs to, not which operator. Pair it with the user agent: a request that claims to be Baiduspider but doesn't match is unverified.
{
// … location and network fields …
"risk": {
"is_ai": false,
"is_crawler": true,
"is_scanner": false
}
}
03 — Questions
Look up the PTR record for the IP address. The hostname must end in .crawl.baidu.com, and a forward lookup of that hostname must return the same IP address. A PTR record alone can be set by anyone who controls the address, so the forward confirmation is the part that matters.
A matching address sets risk.is_crawler in the lookup response. The risk flags are returned for Pro API keys.
Pro: 40,000,000 requests a day with is_ai, is_crawler and is_scanner in every lookup.