Glossary
GPTBot
OpenAI's web crawler, which fetches public content for use in ChatGPT training and retrieval.
By Teeming Chew, Founder Last updated
GPTBot is the user-agent OpenAI uses for general web crawling. It respects robots.txt and can be allowed or blocked at the site level.
How do you allow GPTBot?
Add to robots.txt: User-agent: GPTBot followed by Allow: / (or remove any existing Disallow rules for that agent).
Is GPTBot the same as ChatGPT-User?
No. GPTBot is the broad training/retrieval crawler. ChatGPT-User is a separate user-agent used when a ChatGPT user explicitly browses to a URL. OAI-SearchBot is a third agent used for ChatGPT Search. All three should be allowed for full visibility.
Part of the Cite Hustle GEO glossary: definitions for generative engine optimization and AI search. See how it fits the bigger picture in the GEO methodology.