Applebot-Extended Explained: AI Training, Siri, and Your Content

Applebot-Extended is a robots.txt user agent that lets publishers opt out of having their content used to train Apple's generative AI foundation models. It does not crawl web pages itself. Apple's regular crawler, Applebot, still does the crawling for Siri, Spotlight and Safari search.

Quick answer: Apple runs one crawler, Applebot, and one control token, Applebot-Extended. According to Apple, disallowing Applebot-Extended in robots.txt stops your content from being used to train Apple's foundation models, while your pages can still appear in Apple search features. Blocking Applebot instead removes you from Spotlight, Siri and Safari search. You will never see Applebot-Extended in your server logs, because Apple says it does not crawl; you will see Applebot.

What Applebot-Extended Actually Does

Apple's Applebot documentation describes Applebot-Extended as "a secondary user agent" that "gives web publishers additional controls over how their website content can be used by Apple." Disallowing it opts your site out of having content "used to train Apple's general purpose foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools."

The detail most write-ups get wrong is the next line. Apple states plainly: "Applebot-Extended does not crawl webpages." It is a usage switch, not a bot. Applebot fetches the page. Applebot-Extended tells Apple what it may do with what Applebot fetched.

If that model sounds familiar, it is the same design Google uses. Our explainer on Google-Extended covers the Google version, where a separate token controls AI use while Googlebot keeps crawling for search.

What Applebot Data Is Used For

Apple lists three uses for data crawled by Applebot:

• Search. It powers "the search technology integrated into many user experiences in Apple's ecosystem including Spotlight, Siri, and Safari."
• Model training. It "may also be used to help train Apple foundation models." This is the use Applebot-Extended lets you refuse.
• Answer context. It "may be used to provide additional context and up-to-date content when AI models are used to generate output" in Apple products.

That third use has its own control. Apple says it will not use content tagged with the nosnippet meta tag as additional context for AI-generated output. So there are two separate levers: Applebot-Extended for training, and nosnippet for use in generated answers.

Applebot vs Applebot-Extended at a Glance

Read the right column carefully. Allow Applebot and disallow Applebot-Extended, and you stay eligible for Apple search while opting out of training. Block Applebot and you disappear from the search side too.

How Many Sites Block Applebot-Extended

A June 2026 robots.txt snapshot by US Tech Automations found that 31 of 107 prominent sites with readable robots.txt files disallowed Applebot-Extended, a refusal rate of 29 percent. Named blockers in that report include the BBC, Bloomberg, The New York Times and CNN. Those publishers are not blocking Apple's search crawler. They are declining the training use specifically.

How to Control Applebot-Extended in robots.txt

Each name needs its own user agent line. A rule for Applebot does not apply to Applebot-Extended, so publishers who want a clean split write two blocks. A typical setup for "search yes, training no" looks like this:

User-agent: Applebot Allow: / User-agent: Applebot-Extended Disallow: /

A few things to check before you trust the rule:

1. Partial opt-outs work. Apple's own example disallows a single folder, Disallow: /private/, so you can keep some sections available for training and exclude others.
2. Watch the Googlebot fallback. Apple says that if robots.txt does not mention Applebot but does mention Googlebot, Applebot will follow the Googlebot instructions. A Googlebot rule written years ago may already be shaping what Apple sees.
3. Watch wildcards. A single User-agent: * block with Disallow: / hits every crawler and removes the per-bot control these names are meant to give you.
4. Use nosnippet for answer context. If you also want to keep content out of AI-generated answers in Apple products, Apple points to the nosnippet meta tag rather than robots.txt.

Our guides to robots.txt for AI crawlers and how to block AI crawlers cover the same syntax for the other major operators.

What Apple Looks Like in Your Logs

Here is where the "does not crawl" detail matters in practice. If you search your access logs for Applebot-Extended, you should find nothing. The requests come from Applebot. Apple documents the user agent as ending in (Applebot/Applebot_version; +http://www.apple.com/go/applebot), with the device, OS and Safari version details in front.

That also means you cannot confirm an Applebot-Extended opt-out by watching crawl volume drop. Applebot keeps crawling for search either way. The opt-out changes how Apple uses the content, not whether Apple fetches it. What server data can confirm is that Applebot still reaches your important pages with a 200, so your search eligibility is intact after the change.

Verification is the other half. User agent strings can be spoofed, so check traffic that claims to be Applebot before you act on it. Apple says genuine Applebot traffic resolves to a hostname in the applebot.apple.com domain on reverse DNS, and it publishes its IP ranges at search.developer.apple.com/applebot.json.

Should You Opt Out of Applebot-Extended?

It depends on what your content is worth as training material versus what you gain from Apple's AI features using it. A news publisher or a paid research site has a clear reason to refuse: its content is the product. A local business or a software company that wants its name and facts to show up correctly in Apple Intelligence features may prefer to stay in.

Either way, the choice is cheap to reverse. It is one robots.txt block. Write the decision down, note the date, and review it when Apple updates its documentation or when your content strategy changes.

Tracking Apple Next to Other AI Crawlers

Applebot is one of a growing set of crawlers that hit WordPress sites, next to GPTBot, ClaudeBot, PerplexityBot, Bingbot and others. Some are pure crawlers, some are control tokens, and some are user-triggered fetchers. Our guide to the types of AI crawlers sorts them out.

Prompt sampling guesses at how AI systems might describe your brand. Server data shows what actually happened: which bot requested which URL, how often, and whether it got a 200, a 404 or a redirect. For agencies, that is what turns a robots.txt debate into a specific fix for a specific client site.

citAEOtion shows you, page by page, whether Applebot and every other AI crawler actually reaches your WordPress site, verified from real server traffic. citAEOtion is a $99 Lifetime Deal, or you can book a 20-minute walkthrough and see your own crawler mix first.

Frequently Asked Questions

Is Applebot-Extended the same crawler as Applebot?

No, and it is not a crawler at all. Apple states that Applebot-Extended does not crawl web pages. Applebot does the crawling, and Applebot-Extended is a robots.txt token that controls whether that crawled content can be used to train Apple's foundation models.

How do I block Applebot-Extended in robots.txt?

Add a block with User-agent: Applebot-Extended followed by Disallow: /, or disallow only the folders you want excluded. Keep it separate from your Applebot rules, since a rule for one name does not cover the other.

Does blocking Applebot-Extended remove my pages from Siri or Spotlight?

No. Apple says pages that disallow Applebot-Extended can still be included in search results. Search features such as Spotlight, Siri and Safari rely on Applebot, so blocking Applebot is what removes you from those surfaces.

Why don't I see Applebot-Extended in my server logs?

Because it never visits. Apple states that Applebot-Extended does not crawl web pages. All of Apple's crawl requests arrive as Applebot, so that is the user agent to look for and verify.

Can I stop Apple from using my content in AI-generated answers?

Apple treats that separately from training. It says it will not use content tagged with the nosnippet meta tag as additional context when its AI models generate output. Use Applebot-Extended for training and nosnippet for answer context.