Amazonbot Explained: What Amazon's Crawler Takes and Whether to Allow It

Amazonbot is Amazon's web crawler, used to improve Amazon's products and services, and Amazon says the content it collects may be used to train Amazon AI models. You control it through robots.txt with the Amazonbot token. Amazon also runs two separate agents, Amzn-SearchBot and Amzn-User, that it says are not used for model training.

Quick answer: Amazonbot crawls public pages for Amazon and may feed Amazon AI model training, according to Amazon's developer documentation. It follows robots.txt allow and disallow rules, but it does not support crawl-delay, and changes take about 24 hours to apply. If you want to stay out of training but remain eligible for Alexa search results, block Amazonbot and allow Amzn-SearchBot, which Amazon says does not crawl for generative AI training. Verify any hit against Amazon's published IP list before trusting the user agent.

What Is Amazonbot and Who Runs It

Most site owners know GPTBot and ClaudeBot by name. Amazonbot tends to stay under the radar until someone opens a server log and finds a crawler they never thought about working through thousands of pages.

Amazonbot is Amazon's own crawler. Amazon's developer documentation says it is "used to improve our products and services. This helps us provide more accurate information to customers and may be used to train Amazon AI models." That last clause is the one to read twice. A crawl is not only about answering a question later. It can also feed model training.

Amazon does not publish a page-level breakdown of how any single URL gets used. Treat any confident claim about the exact destination of your pages as unverified.

Amazon's Three Agents

Amazon now documents three separate user agents, and they do different jobs. Mixing them up is the most common mistake in Amazon crawler policies.

This matters for the old advice that Amazonbot exists to help Alexa answer questions. Amazon's current documentation ties Alexa to Amzn-SearchBot and Amzn-User, not to Amazonbot. If Alexa visibility is what you care about, the agent to keep allowed is Amzn-SearchBot.

How to Identify Amazonbot in Your Server Logs

Amazon publishes the user agent as:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36

Amzn-SearchBot and Amzn-User follow the same pattern with their own tokens. Match on the token, since the Chrome version part changes.

User agents are easy to fake, so treat the string as a lead, not proof. The general practice is to take the request's IP address and check it against the operator's published list. Amazon publishes IP addresses for Amazonbot at developer.amazon.com/amazonbot/ip-addresses/ and a separate list for Amzn-SearchBot. A reverse DNS lookup followed by a forward lookup is another common check. If the IP does not belong to Amazon, you are looking at an impersonator. Our guide to fake AI crawlers covers why spoofing a big-name bot is so common.

Why Some Site Owners Block Amazonbot

The first complaint is load. Any large crawler can eat server capacity, and Amazonbot does not support the crawl-delay directive, according to Amazon. So you cannot slow it down with robots.txt. Your choices are allow, disallow, or rate limiting at the server or CDN level.

The second complaint is reciprocity. Amazonbot does not send referral traffic the way a search engine does, and Amazon says its content may go into model training. For a publisher who sells content, crawl budget spent on a bot with no visible return looks like a cost.

There is a fair counterargument. If voice assistants are a discovery surface for your audience, you want Amzn-SearchBot to reach you. Blocking every Amazon agent removes you from that surface entirely. That trade is a business decision, not an SEO rule.

How to Allow or Block Amazonbot

Blocking happens through your robots.txt file. To keep content out of Amazonbot while staying eligible for Alexa search results:

User-agent: Amazonbot Disallow: / User-agent: Amzn-SearchBot Allow: /

A few practical notes:

• Amazon says robots.txt changes take about 24 hours to reflect in its systems. Do not judge the rule on day one.
• For finer control, Amazon says its bots honor the page-level robots meta tag noarchive, which it describes as "do not use the page for model training," along with noindex and none.
• Amzn-User may not follow every robots.txt directive, because a person can trigger it. A firewall rule is the only firm block there.
• If you run a CDN or WAF with bot controls, make sure those settings and robots.txt do not contradict each other.
• Confirm the block worked by watching server traffic afterward, not by assuming it took effect.

Allowing Amazonbot is the default, so most sites have already decided by not deciding. Even if you are fine with the crawl, you still want to know how much capacity it uses. For a wider view of the trade-offs, see our piece on allowing vs blocking AI bots.

Tracking Amazonbot Alongside Other AI Crawlers

One crawler is manageable. A dozen is where it gets messy. GPTBot, ClaudeBot, PerplexityBot, Amazon's three agents and others can all hit the same WordPress site in the same week. Our list of the top AI crawlers shows who the usual visitors are and what each operator says they do.

The only way to know what actually happened on your site is to read real server traffic, not to infer activity from prompts. You want to see which bot hit which page, when, and what status code it received. For agencies with several client sites, that record is also what makes a white-label crawler report hold up in a client meeting.

Deciding Whether to Allow Amazonbot

Treat it as a cost and benefit question. Start with how much traffic verified Amazonbot requests actually add, using real hits instead of a blended bot percentage. Then weigh that against what you gain from Amazon's use of the content.

If the crawl is heavy and you do not want your pages in Amazon's training data, blocking Amazonbot is reasonable, and you can keep Amzn-SearchBot allowed for Alexa. If your content answers the kind of consumer questions people put to voice assistants, you have a case for leaving more of the door open. Either way, write the decision down, check that it worked, and revisit it when crawl patterns shift.

citAEOtion shows you, page by page, whether Amazonbot and every other AI crawler actually visits, verified from real server traffic rather than guesses. citAEOtion is a $99 Lifetime Deal; if you want to see it first, book a 20-minute walkthrough.

Frequently Asked Questions

What is Amazonbot used for?

Amazon says Amazonbot is used to improve its products and services, to provide more accurate information to customers, and that it may be used to train Amazon AI models. Alexa search experiences are tied to a separate agent, Amzn-SearchBot, in Amazon's current documentation.

Should I block Amazonbot?

It depends on what the crawl costs you and whether you mind your content being used for Amazon AI training. A common middle ground is to disallow Amazonbot and allow Amzn-SearchBot, which Amazon says does not crawl for generative AI training. That keeps you eligible for Alexa search while opting out of training crawls.

How do I verify that a crawler is really Amazonbot?

Do not trust the user agent string alone. Check the request's IP address against the list Amazon publishes at developer.amazon.com/amazonbot/ip-addresses/. If the IP is not on Amazon's list, treat the traffic as spoofed.

Does Amazonbot respect crawl-delay?

No. Amazon's documentation says its bots do not support the crawl-delay directive. They do follow user-agent allow and disallow rules, and changes take about 24 hours to take effect. To slow Amazonbot down, use rate limiting at the server or CDN.

Can I stop Amazon from training on a single page?

Yes. Amazon says its bots honor the page-level robots meta tag noarchive, which it defines as do not use the page for model training. Add it to the pages you want excluded and leave the rest of the site crawlable.