HAProxy

1 article tagged “HAProxy”.

9 min read

Block AI crawlers in Apache and HAProxy by verified IP (tested configs)

The usual .htaccess recipe for GPTBot is a user-agent regex. It can't tell a real crawler from a scraper using its name, and it can't safely let AI search bots in. These Apache and HAProxy configs check the operators' official IP lists, and we ran them against real addresses.

  • AI crawlers
  • Apache
  • HAProxy

Try it

One GET request: location, network and bot signals

Free keys include 1.000 lookups a day. No card needed.