Arquivo.pt Web Crawler
risk.is_crawler
- Method
- Ranges Published IP ranges
- Official file
- Format
- JSON range file
- Refresh
- Every hour. If a refresh fails, entries expire after 24 hours instead of being trusted forever.
Bot signals — Search & SEO crawler
Arquivo.pt Web Crawler IP addresses
Arquivo.pt published web archiving crawler ranges.
01 — Verification
How IPAPI.GR verifies Arquivo.pt Web Crawler.
Only from what Arquivo.pt Web Crawler publishes about itself. No third-party blocklists, no behavioural guesses. Live address counts change with every publication, so check the official file for the current list.
02 — In the response
What a match looks like.
A lookup of a Arquivo.pt Web Crawler address sets risk.is_crawler. The flags come with geolocation and network data in the same response, for Pro API keys.
The response says which kind of bot an address belongs to, not which operator. Pair it with the user agent: a request that claims to be Arquivo.pt Web Crawler but doesn't match is unverified.
{
// … location and network fields …
"risk": {
"is_ai": false,
"is_crawler": true,
"is_scanner": false
}
}
03 — Configuration
Allow, block or verify it yourself.
# 1. Build a geo include from the official file (cron, e.g. hourly).
# Keep the old file if the download fails or comes back empty.
curl -fsS https://arquivo.pt/crawlerips.json \
| jq -r '.prefixes[] | (.ipv4Prefix // .ipv6Prefix // empty) | "\(.) 1;"' \
> /etc/nginx/arquivo-ranges.conf.new && [ -s /etc/nginx/arquivo-ranges.conf.new ] && mv /etc/nginx/arquivo-ranges.conf.new /etc/nginx/arquivo-ranges.conf \
&& nginx -t -q && nginx -s reload
# 2. In the http block: 1 when the client IP is in the published ranges.
geo $arquivo_ip {
default 0;
include /etc/nginx/arquivo-ranges.conf;
}
prefixes array.04 — Questions
About Arquivo.pt Web Crawler.
Where does Arquivo.pt Web Crawler publish its IP addresses?
Arquivo.pt Web Crawler publishes its addresses at https://arquivo.pt/crawlerips.json. IPAPI.GR reads that publication directly, not a third-party copy.
How often is the Arquivo.pt Web Crawler list refreshed?
IPAPI.GR re-fetches it every hour. If a refresh fails, the previous entries expire after at most 24 hours instead of being trusted forever.
Which risk flag does Arquivo.pt Web Crawler set?
A matching address sets risk.is_crawler in the lookup response. The risk flags are returned for Pro API keys.
From the blog
Go deeper.
Related operators
Verify Arquivo.pt Web Crawler on every request.
Pro: 40,000,000 requests a day with is_ai, is_crawler and is_scanner in every lookup.