Bot signals · Internet scanner
What is the Google Cloud Web Security scanner?
Google Cloud Web Security Scanner
Web Security Scanner, part of Google Cloud’s Security Command Center, identifies security vulnerabilities and misconfigurations in App Engine, Google Kubernetes Engine and Compute Engine web applications. It crawls an application, following links within the scope of its starting URLs, and tries to exercise as many user inputs and event handlers as possible. Its managed scans run automatically once a week and send only GET requests, so they do not submit forms.
01 · Profile
Google Cloud Web Security at a glance.
What to look for in your logs, and what the lookup API returns for its addresses.
- Type
- Internet scanner
- robots.txt
- Does not control scanning.
- API flag
risk.is_scanner
A user agent alone never sets a flag. Anyone can claim to be Google Cloud Web Security; the flag follows the address the request comes from.
02 · In the response
What a match looks like.
A lookup of a Google Cloud Web Security address sets risk.is_scanner. The flags come with geolocation and network data in the same response, for API keys on every paid plan.
The response says which kind of bot an address belongs to, not which operator. A request that claims to be Google Cloud Web Security while risk.is_scanner is false should not be trusted as Google Cloud Web Security.
{
// … location and network fields …
"risk": {
"is_ai": false,
"is_crawler": false,
"is_scanner": true
}
}
03 · Control
Block Google Cloud Web Security.
At the edge
- robots.txt won’t help. It does not control port or vulnerability scanning, so a block has to happen before the request reaches your app.
- Firewall, WAF or CDN. Many offer bot rules you can allow or deny by category.
- In your app. Look up the client address and allow or deny on
risk.is_scanner, alongside the user agent. - Never on the user agent alone. Blocking the name stops only honest clients; allowing it lets anyone in who borrows it.
04 · Questions
About Google Cloud Web Security.
What is Google Cloud Web Security?
Web Security Scanner, part of Google Cloud’s Security Command Center, identifies security vulnerabilities and misconfigurations in App Engine, Google Kubernetes Engine and Compute Engine web applications. It crawls an application, following links within the scope of its starting URLs, and tries to exercise as many user inputs and event handlers as possible. Its managed scans run automatically once a week and send only GET requests, so they do not submit forms.
How do I block Google Cloud Web Security?
robots.txt does not control port or vulnerability scanning. Block Google Cloud Web Security at your firewall, WAF or CDN, or look up each client address and deny requests where risk.is_scanner is true.
Can I trust a request that says it is Google Cloud Web Security?
Not on the user agent alone: any client can send a bot’s name in its user agent. A user agent alone never sets a flag. Check the address with the free bot checker or the lookup API.
Which risk flag does Google Cloud Web Security set?
Its addresses set risk.is_scanner in the lookup response. The risk flags come with every paid plan.
From the blog
Go deeper.
Related bots
Recognise Google Cloud Web Security on every request.
is_ai, is_crawler and is_scanner in every lookup, on every paid plan, from 9,00 € a month (VAT included).