OpenAI Publisher Controls: Crawl Access, Citations and Referrals Are Different Outcomes
Separate OAI-SearchBot access, GPTBot training controls, ChatGPT links or citations, and tagged referrals so each observation stays within its evidence.
Published August 9, 2026: This guide reflects OpenAI’s current publishers and developers FAQ.
OpenAI publisher controls cover different outcomes. OAI-SearchBot concerns search discovery. GPTBot concerns potential model training. A user-directed fetch is another event. Links or citations in ChatGPT are appearances, while utm_source=chatgpt.com can identify some referral visits.
Do not report one of these signals as if it proved all the others. A crawler request does not prove a citation, and a citation does not prove a visit.
Four separate outcomes
| Signal | What it can show | What it cannot prove |
|---|---|---|
| OAI-SearchBot access | Declared permission for search discovery requests | Indexing, citation, referral, or ranking |
| GPTBot access | Declared permission relating to model training collection | Search discovery or a ChatGPT citation |
| Link or citation | The page was referenced in a particular output | A stable rank, click, conversion, or complete retrieval history |
| Tagged referral | A visit with the documented ChatGPT source parameter | Every ChatGPT visit or the answer that produced it |
Search and training are separate
OpenAI says publishers should allow OAI-SearchBot if they want their sites included in ChatGPT search summaries and snippets. GPTBot is the separately named crawler for potential training use. A publisher can therefore document different preferences for search discovery and training.
OpenAI also notes that ChatGPT Atlas may discover a link and title through other search or discovery providers even when OAI-SearchBot is blocked. Publishers that do not want a page summarized should use the documented noindex control. That is another reason to avoid treating one robots rule as a universal appearance switch.
Keep private material behind authentication. As the crawler policy guide explains, robots.txt is not access authorization.
Measure access with logs
- Save the robots.txt version and deployment timestamp.
- Verify the client using OpenAI’s current documentation rather than trusting the user-agent string alone.
- Record requested URL, timestamp, response code, bytes, and cache status.
- Separate OAI-SearchBot, GPTBot, and user-directed activity where the evidence permits.
- Watch crawl volume and errors after a policy change.
- Keep the result at the access layer: “verified requests occurred” or “no verified requests were observed.”
Absence from a limited log window is not proof that a page can never appear. Cached results, other discovery providers, filters, retention, and low request volume can all affect the observation.
Measure referrals with boundaries
OpenAI says ChatGPT referral URLs include utm_source=chatgpt.com. Create a documented analytics segment for that parameter and preserve the raw landing page, timestamp, campaign fields, consent state, and on-site outcome. Keep an additional referrer-based segment for diagnosis, but do not assume it captures every visit.
Report sessions or visits according to the analytics system’s definition. Do not rename them citations. A visitor may follow a link without the publisher being able to reconstruct the exact answer, prompt, or retrieval path.
Use the five-layer measurement crosswalk to keep access, appearance, referral, and outcome rows separate.
A publisher control card
- Desired outcome: search discovery, no training collection, public user access.
- Controls: OAI-SearchBot allowed, GPTBot disallowed, public pages indexable, private pages authenticated.
- Evidence: dated robots file, verified logs, index directives, analytics segment.
- Limit: no claim that access guarantees a citation or that tagged referrals represent every ChatGPT visit.
- Review trigger: OpenAI documentation change, crawler incident, site migration, or measurement change.
This card makes the policy auditable without promising control over selection. It also gives legal, editorial, analytics, and infrastructure owners a shared record of what the site intends.
Ask a question or join the discussion