Google-Extended is a robots.txt control token that decides whether your Google-crawled content can be used for Gemini AI, and it is not a separate crawler with its own user agent.
Google-Extended is a robots.txt token, not a crawler. It has no user agent and fetches nothing. It only tells Google whether content already crawled by Googlebot may be used to train Gemini and to ground answers in Gemini Apps and Vertex AI. GoogleOther is a real crawler that sends the token GoogleOther and handles Google's non-search jobs like research and product development. Googlebot is the crawler that builds Google Search. The three are separate, and blocking Google-Extended does not affect your Google Search ranking.
Quick answer: These are three different things. Googlebot is Google's search crawler and it builds the index that powers Google Search. Google-Extended is a robots.txt product token only. It has no separate HTTP user agent and does not fetch pages itself. It governs whether Google may use content, already crawled by Googlebot, to train future Gemini models and to ground answers in Gemini Apps and Google Search on Vertex AI. Disallowing Google-Extended does not remove you from Google Search or change your ranking. GoogleOther is a genuine crawler that identifies itself with the token GoogleOther and runs Google's non-search fetches, such as research and internal product work, with its own IP ranges published in special-crawlers.json rather than googlebot.json. None of these bots run JavaScript, so browser analytics never see them and your server log is the record.
Google-Extended is a token, not a crawler
The single most misunderstood fact here: Google-Extended never visits your site. It is a control you place in robots.txt, and Google reads it as a policy signal. There is no Google-Extended IP address, no Google-Extended user agent in your logs, and no Google-Extended request line to find. Google applies the token to content that Googlebot already fetched, then decides whether that content is eligible for generative AI uses.
What Google-Extended controls, per Google's documentation:
- Training future Gemini models - Grounding answers in Gemini Apps - Grounding answers in Google Search on Vertex AIWhat it does not control: your presence or ranking in Google Search. Those are decided by Googlebot and the search index. This separation is the whole point of the token. It lets a publisher stay fully indexed in Search while opting out of Gemini training, a choice that was not possible before Google introduced Google-Extended. Your eligibility to rank in Google AI Overviews is decided the same way, through Google Search rather than this token.
How to use Google-Extended in robots.txt
Because it is a robots.txt token, you set it exactly like any other user agent block. To opt all of your content out of Gemini training and grounding while staying in Google Search:
User-agent: Google-Extended Disallow: /To allow it, either say nothing (allowing is the default) or write:
User-agent: Google-Extended Allow: /To opt only one section out:
User-agent: Google-Extended Disallow: /premium/Since Google-Extended sends no requests, you cannot confirm it by watching logs. You confirm it is set correctly by testing your robots.txt file. What you can watch in logs is Googlebot and GoogleOther, which do send real requests.
What GoogleOther is
GoogleOther is a real crawler, and it does show up in your logs with the product token GoogleOther. Google introduced it to take non-search fetches off Googlebot. Its jobs include research and development, internal analysis, and other product uses that are not about building the search index. Google has been open that some of that work relates to its AI efforts, without itemizing each use publicly.
The distinction Google draws is clean: Googlebot for the search index, GoogleOther for everything else. That matters for how you read your traffic. A spike in GoogleOther is not a ranking signal and not an alert. It is Google pulling your content for its own internal needs. If you lump all Google-labeled traffic together, you inflate your SEO numbers with a crawler that never touches the index.
GoogleOther also has two media variants:
GoogleOther-Image -> image fetches for non-search uses GoogleOther-Video -> video fetches for non-search usesImportantly, GoogleOther does not share Googlebot's IP ranges. Its ranges are published separately in Google's special-crawlers.json file, while Googlebot's live in googlebot.json. If you verify Google traffic by IP, check the right file for the right token.
Googlebot vs Google-Extended vs GoogleOther, side by side
Googlebot crawler, builds Google Search index, IPs in googlebot.json Google-Extended robots.txt token only, no user agent, controls Gemini use GoogleOther crawler, non-search jobs, IPs in special-crawlers.jsonReading this table stops the two most common mistakes. First, blocking Google-Extended to try to slow crawling does nothing, because it is not a crawler. Second, treating GoogleOther traffic as SEO activity misreads it, because GoogleOther never feeds Search. Keep the search decision (Googlebot), the Gemini decision (Google-Extended), and the everything-else crawler (GoogleOther) as three separate things.
How to see Google's AI crawling on your site
GoogleOther, like the other AI-related crawlers such as Anthropic's ClaudeBot, reads the HTML your server returns and does not run JavaScript. Google Analytics only records visitors whose browsers execute its tracking script, so GoogleOther and its media variants never appear in GA. Your reports show your human search traffic, but the machine fetches Google runs for its AI and research work are missing entirely.
The record of those fetches lives in your web server access log, which is what citAEOtion reads to track AI crawlers on WordPress. It reads real traffic, classifies each request by its actual user agent, and separates Googlebot from GoogleOther, GoogleOther-Image, and GoogleOther-Video so you are not double-counting. You can see exactly which Google agent pulled which page, when, and how often, which is the only way to tell search crawling apart from AI and research crawling. For the wider field of bots reading you, our roundup of the top AI crawlers maps who else to watch for.
A single site costs $34.99 a month, with agency plans and a Founders Club discount. Compare tiers on the citAEOtion pricing page, or book a live demo to see Google's crawlers split apart on a live WordPress dashboard.
The definitive list of Google's crawlers, tokens, and fetchers is maintained in Google Search Central's crawler overview.
Frequently Asked Questions
Does blocking Google-Extended hurt my Google Search ranking?
No. Google-Extended only controls Gemini training and grounding. Google states it does not affect inclusion or ranking in Google Search, which is governed by Googlebot.
Why can't I find Google-Extended in my server logs?
Because it is a robots.txt token, not a crawler. It has no user agent and sends no requests. You will never see a Google-Extended line in a log. Check that it is applied by testing your robots.txt file.
What is the difference between GoogleOther and Googlebot?
Googlebot builds the Google Search index. GoogleOther handles non-search fetches such as research, internal analysis, and AI-related work. They use different IP ranges, and GoogleOther traffic is not a ranking signal.
Does Google-Extended stop GoogleOther from crawling?
They are independent. Google-Extended governs how Googlebot-crawled content may be used for Gemini. To limit the GoogleOther crawler itself, add a robots.txt rule for the GoogleOther token.
How do I verify GoogleOther by IP?
Match the source IP against Google's special-crawlers.json file, not googlebot.json. GoogleOther has its own published ranges separate from Googlebot.
Tell Googlebot and GoogleOther apart on your site. Book a free demo