Glossary
Google-Extended
A robots.txt token that controls whether your content trains and grounds Google's Gemini models — without affecting normal Search indexing.
By Teeming Chew, Founder Last updated
Google-Extended is a standalone product token Google introduced in 2023. It is not a separate crawler: it is a control you set in robots.txt to opt your content in or out of being used for training and grounding Gemini and Vertex AI generative features. Allowing it keeps you eligible to be referenced by Google's AI; disallowing it does not change how Googlebot crawls or ranks you in classic Search.
Should I allow Google-Extended?
For most sites pursuing AI visibility, yes. Blocking Google-Extended can reduce your eligibility to be surfaced and grounded inside Gemini-powered experiences. Block it only when you have a specific reason to withhold content from generative use.
How do I set Google-Extended in robots.txt?
Add a dedicated block: User-agent: Google-Extended followed by Allow: / (or Disallow: / to opt out). It obeys the standard robots.txt syntax, separate from the Googlebot and * blocks.
Does blocking Google-Extended hurt my rankings?
No. It only governs generative AI training and grounding, not crawling or ranking in traditional Google Search. Your blue-link rankings are unaffected either way.
Part of the Cite Hustle GEO glossary — definitions for generative engine optimization and AI search. See how it fits the bigger picture in the GEO methodology.