OpenAI · Training crawler
GPTBot
GPTBot is OpenAI's web crawler that collects public content used to train and improve its generative AI models, including the models behind ChatGPT. It is a training crawler, not a live search bot — blocking GPTBot keeps your content out of future OpenAI training datasets but does not remove you from ChatGPT search results. GPTBot obeys robots.txt and is controlled independently of OpenAI's other crawlers.
Last updated
- User-agent token
- GPTBot
- Operator
- OpenAI
- Feeds
- ChatGPT model training
- robots.txt
- Unverified
How to control GPTBot in robots.txt
Edit the robots.txt file at the root of your domain (for example https://example.com/robots.txt), add one of the groups below, then save and re-deploy.
Remember: a named User-agent: GPTBot group overrides your global User-agent: * rules, so repeat any private Disallow paths inside it.
Allow GPTBot (recommended for AI visibility)
# Welcome GPTBot, but keep private areas blocked.
# A named user-agent group overrides "User-agent: *", so repeat
# your own private Disallow rules inside this group.
User-agent: GPTBot
Allow: /
Disallow: /admin/
Disallow: /account/
Disallow: /cart/
Disallow: /checkout/Block GPTBot
# Block GPTBot from the entire site.
User-agent: GPTBot
Disallow: /FAQ
Does blocking GPTBot remove me from ChatGPT?
No. GPTBot only gathers training data. To appear (or not) in ChatGPT's live search citations you control OAI-SearchBot separately; to allow a one-off fetch when a user pastes your URL you control ChatGPT-User.
Does GPTBot respect robots.txt?
Yes. OpenAI documents that GPTBot honors robots.txt, and you can allow or disallow it independently of OAI-SearchBot and ChatGPT-User.
Is your site visible to AI crawlers?
Run a free AI-visibility audit to see which AI crawlers can reach your content and how often you get cited by ChatGPT, Perplexity, Claude, and Google AI Overviews.
Run a free auditPart of the Cite Hustle AI crawler directory. For the full framework on AI search visibility, read the GEO methodology.