OpenAI Publisher Controls: Crawl Access, Citations and Referrals Are Different Outcomes

Choose OpenAI publisher controls by outcome: search discovery, training collection, title-and-link suppression, referrals, or authenticated access.

Sonar and a publisher route search, training, and user controls toward separate link, citation, referral, and fetch outcomes.

OpenAI publisher controls operate at several different layers. OAI-SearchBot manages automatic search discovery requests, GPTBot concerns potential training collection, and ChatGPT-User is associated with certain user-initiated actions. A citation and a measured referral visit require separate observations.

Choose the outcome before changing a rule. The current OpenAI crawler reference is the source for crawler roles and published verification ranges. A familiar user-agent string on its own is insufficient evidence of the operator.

Search discovery and potential training have separate controls

OpenAI says OAI-SearchBot is used for ChatGPT search results, while GPTBot collects content that may be used to train the model. It also says sites opted out of OAI-SearchBot can still appear as navigational links. A search opt-out, therefore, requires a surface-specific reading; it is not a promise that the URL will never be mentioned anywhere.

ChatGPT-User is different again. OpenAI describes certain user-requested page visits and says robots.txt rules may not apply to those actions. Keep private information behind authentication and enforce permissions in the application. The search, agent, and training policy guide explains how to document separate access intentions.

Retire the old Atlas instructions

During the October 15 source check, the former publishers-and-developers FAQ redirected to ChatGPT Atlas retirement guidance. OpenAI states that Atlas has retired. The page does not establish an absolute retirement event date, so this article does not assign one.

The earlier version used that FAQ for Atlas-specific title-and-link and noindex instructions. Those instructions have been removed from the current workflow. Review saved runbooks that still name Atlas and verify their present product surface before applying an old control. A redirect to a retirement article cannot substantiate a live search-removal contract.

Keep an evidence card for each intended outcome

Control, observation and outcome require different proof
IntentionEvidence to retainBoundary
Permit search discoveryDated OAI-SearchBot rule, verified client and responsePermission does not guarantee selection or a citation.
Decline potential training collectionDated GPTBot preference and verified request observationsThe rule is not an audit of a model’s training history.
Exclude private materialAuthentication policy and unauthorized-response checkRobots declarations do not authorize or deny an account.
Measure a visitObserved landing URL, referrer, analytics source/medium and timeA visit cannot count citations that produced no click.

For an illustrative public documentation site, the owner might permit search discovery, decline GPTBot, and require sign-in for internal drafts. Record those choices separately, then check each at its enforcement point. Do not use a successful public page fetch as evidence that private drafts are protected.

Verify access with a bounded log review

  1. Save the deployed rule and its effective time.
  2. Verify the requesting client against current operator documentation.
  3. Record URL scope, timestamp, response status, and cache/origin context.
  4. Separate search, training, and user-directed observations where the available fields permit.
  5. State the observation window and missing evidence before drawing a conclusion.

No verified requests during a short window can reflect low demand, caching, log retention, or filtering. It cannot establish permanent exclusion from every discovery source. Avoid retaining private prompts, raw personal identifiers, or session secrets in a public audit.

Treat a referral tag as an observation to check

The previous version described utm_source=chatgpt.com as a current universal convention. The former supporting FAQ no longer supplies that contract. Preserve tagged arrivals when they occur, and test the current citation-to-arrival routes before relying on the parameter. This correction does not establish that OpenAI removed tags from every link.

Use the GA4 referral workflow to record what reached the site and the measurement crosswalk to keep access, citation, and referral evidence apart. No controlled ChatGPT click experiment was performed for this update.

Correction note: October 15 source verification replaced the redirected publisher FAQ, removed retired Atlas instructions, and narrowed the referral-tag claim. Existing crawler-role distinctions remain grounded in OpenAI’s current reference.

For discovery signals beyond a crawler request, the Cloudflare and OpenAI pilot review separates newly indexed pages from citations, referral arrivals, and reader outcomes.

Keep learning

Continue this topic

Community discussion

Discuss: OpenAI Publisher Controls: Crawl Access, Citations and Referrals Are Different Outcomes

Have a question, a useful example, or a different perspective? Join the discussion, share evidence, and help other readers reach a better answer.

0 replies Moderated
No replies yet.

Be the first to ask a focused question, share a practical example, or add useful evidence.

Ask a question or join the discussion

Share evidence, a useful example, or a clear question. Be specific, stay on topic, and challenge ideas without attacking people. First-time replies may be held for moderation.