GPTBot
GPTBot is OpenAI's web crawler that collects public content used to train and improve its generative AI models, including the models behind ChatGPT. It is a training crawler, not a live search bot. Blocking GPTBot keeps your content out of future OpenAI training datasets but does not remove you from ChatGPT search results. GPTBot obeys robots.txt and is controlled independently of OpenAI's other crawlers.
Last updated
What does GPTBot do?
- User-agent token
- GPTBot
- Operator
- OpenAI
- Feeds
- ChatGPT model training
- robots.txt
- Unverified
How do you control GPTBot in robots.txt?
Edit the robots.txt file at the root of your domain (for example https://example.com/robots.txt), add one of the groups below, then save and re-deploy.
Remember: a named User-agent: GPTBot group overrides your global User-agent: * rules, so repeat any private Disallow paths inside it.
Allow GPTBot (recommended for AI visibility)
# Welcome GPTBot, but keep private areas blocked.
# A named user-agent group overrides "User-agent: *", so repeat
# your own private Disallow rules inside this group.
User-agent: GPTBot
Allow: /
Disallow: /admin/
Disallow: /account/
Disallow: /cart/
Disallow: /checkout/Block GPTBot
# Block GPTBot from the entire site.
User-agent: GPTBot
Disallow: /FAQ
Does blocking GPTBot remove me from ChatGPT?
No. GPTBot only gathers training data. To appear (or not) in ChatGPT's live search citations you control OAI-SearchBot separately; to allow a one-off fetch when a user pastes your URL you control ChatGPT-User.
Does GPTBot respect robots.txt?
Yes. OpenAI documents that GPTBot honors robots.txt, and you can allow or disallow it independently of OAI-SearchBot and ChatGPT-User.
Is your site visible to AI crawlers?
Run a free technical access audit to inspect published crawler controls and raw HTML delivery. It does not measure whether an engine indexed or cited the site.
Run a free auditPart of the Cite Hustle AI crawler directory. For the full framework on AI search visibility, read the GEO methodology.