# How to Verify a Real AI Crawler vs a Spoofed Bot

https://citaeotion.ai/verify-ai-crawler-vs-spoofed-bot/

You verify a real AI crawler by checking its source IP, not its user-agent. Any scraper can type "GPTBot" into a header; only the vendor controls the IP.
To verify a real AI crawler versus a spoofed bot, match the request source IP against the vendor published IP range and run forward-confirmed reverse DNS; never trust the user-agent string alone because it is trivially faked.
Quick answer: A user-agent header is plain text that any client can set, so a bot claiming to be GPTBot may be a scraper in disguise. Real verification works two ways. First, the source IP has to appear in the address ranges the vendor publishes for its crawler; OpenAI, Anthropic, Perplexity, and Google all publish these ranges. Second, forward-confirmed reverse DNS has to line up: the IP traces back to a hostname in the vendor domain, and that hostname resolves forward to the same IP. A request that clears both is genuine. One that fails is spoofed, and it can be rate-limited or blocked. Doing this once by hand is simple; doing it for every hit, every day, is not, which is where continuous tracking earns its place.
Why the user-agent alone means nothing
The User-Agent is a request header the client fills in, which means any scraper can send a request that lands in your log looking exactly like Anthropic or OpenAI. Scrapers do this to ride the goodwill sites extend to trusted crawlers, dodging the rate limits and blocks that would otherwise stop them. So a matching user-agent is a claim, not proof. The real evidence is the network address the request actually came from, which the sender cannot forge over an established connection. Verification is simply the work of trusting that address instead of the label attached to it. Once you detect AI crawlers in your access logs, confirming each one is genuine is the next step.
The two checks that actually prove a crawler
Published IP ranges. The major AI vendors publish the address blocks their crawlers use. A genuine hit comes from inside those blocks; a request that claims to be GPTBot from an address the vendor never published is not GPTBot. OpenAI publishes ranges for GPTBot, OAI-SearchBot, and ChatGPT-User, and Anthropic, Perplexity, and Google publish theirs as well.
Forward-confirmed reverse DNS. For crawlers that support it, this holds up even when a range shifts. The address has to trace back to a hostname that sits under the vendor domain, and that hostname has to resolve forward to the same address. A spoofer controls neither the vendor address space nor the vendor DNS, so a faked request cannot satisfy both at once. Together the two checks separate the crawlers you can trust from the ones only wearing their name.
This is not hypothetical. Perplexity has been the subject of independent reports about undeclared crawling, which is exactly why the source address matters more than a friendly-looking header.
Do this at scale, automatically
Verifying one hit is simple. Verifying thousands of daily hits is not, and that is the real problem. citAEOtion reads your actual server traffic and sorts genuine AI crawlers from impostors by measured hits, not by the header they claim. It shows the bot, the page, the timestamp, the HTTP status, and the category on one dashboard, and citAEOtion runs the verification continuously so you never have to check a single address by hand. Prompt-based tools guess whether a model saw your page; citAEOtion measures which verified bot actually fetched it.
Verify my AI crawler traffic
Frequently Asked Questions
Can a user-agent string be faked?
Yes, easily. It is a plain-text header the client sets, so a scraper can impersonate GPTBot or ClaudeBot with a single request. The string is never proof on its own.
What is forward-confirmed reverse DNS?
It is a two-way DNS check: reverse-resolve the IP to a hostname, confirm the hostname belongs to the vendor, then forward-resolve that hostname back to an IP and confirm it equals the original. Spoofers cannot control the vendor DNS, so they fail the check.
How do I verify GPTBot specifically?
GPTBot is confirmed by its source address, not its name: a genuine hit comes from the ranges OpenAI publishes for GPTBot. A request carrying the GPTBot name from any other address is spoofed. citAEOtion runs that match for every hit automatically.
What should I do with a spoofed bot?
Rate-limit or block it at your firewall or CDN. A request that forges a trusted crawler name while failing IP verification is scraping, not a legitimate AI crawler.
Do all AI crawlers support reverse DNS?
Not all. Some, like GPTBot, are verified primarily by published IP ranges, while Googlebot and PerplexityBot support reverse DNS. Use whichever method the vendor documents, and prefer IP-range matching when reverse DNS is unavailable.
Authoritative method reference: Google Search Central, verifying crawlers with reverse DNS.

---
Site: citAEOtion - https://citaeotion.ai/
LLM index: https://citaeotion.ai/llms.txt | Full text: https://citaeotion.ai/llms-full.txt