Cloudflare Redirects for AI Training: Test Canonicals Before Turning 301s On
Cloudflare can redirect verified AI training crawlers to a same-origin canonical URL. Test the first 256 KB, redirect order, logs, and rollback before enabling it.
Tagged collection
Latest articles tagged “Technical SEO”
Open an article for the complete method, evidence, limitations, and a focused reader discussion.
-
-
Google-InspectionTool User-Agent Correction: Audit Exact-Match WAF Rules
Google corrected an erroneous semicolon in its Google-InspectionTool user-agent documentation. Audit copied exact-match WAF and parser rules.
-
Schema.org 30.0 Is Not a Google Rich Result Update
Schema.org 30.0 expanded the vocabulary, but a vocabulary release does not automatically create Google rich-result support. Audit the two layers separately.
-
Google-GeminiNotebook Replaced Google-NotebookLM: Update Log Parsers Without Losing History
Google renamed the NotebookLM crawler token. Update exact-match rules and analytics without splitting one crawler into two histories.
-
How to Test Whether a Public ChatGPT Site Is Indexable
OpenAI allows eligible users to publish ChatGPT Sites publicly, but public access is not proof of indexability. Use this controlled technical audit before reporting results.
-
Cloudflare’s September 15 AI Crawler Defaults: What Publishers Must Check
Cloudflare’s September 15 defaults can block multipurpose crawlers when Training is blocked. Audit ad pages, verified bots, and final responses before the change.
-
Googlebot’s 2 MB Fetch Limit: Audit What Appears Before the Cutoff
Googlebot fetches only the first 2 MB of supported non-PDF files. Audit the byte order of titles, canonicals, main content, links, and structured data.
-
Back Button Hijacking: Audit Third-Party Scripts Before Google Treats It as Spam
Reproduce browser-history manipulation across real entry, consent, ad, and device states, then isolate the first- or third-party script that causes it.
-
A URL Can Return HTTP 200 and Still Fail Before the Client Reads HTML
A reproducible client matrix shows Node rejecting oversized response headers before HTML while curl still succeeds, even though the server returns HTTP 200.
