CiteShade Raised Wrong Answers From 1% to 68%

The citation can be genuine and still misassign responsibility

A normal citation check asks whether the linked page exists and contains related language. Citation laundering survives that shallow test. The trusted page can be real, relevant and correctly linked while a malicious source steers the model toward a false answer.

CiteShade studies a setting where an attacker controls one source among several sources supplied to a retrieval-augmented system. Correct evidence remains available. The attack attempts to change the answer while shifting visible attribution toward another, trusted document.

Four events create the laundering path

1. Mixed evidence enters context. The system receives correct sources plus one attacker-controlled source.
2. The attacker source changes the answer. Its text supplies or encourages an incorrect conclusion.
3. The answer cites a trusted neighbor. The visible reference points to a source that did not cause the wrong claim.
4. A reviewer sees credible attribution. A domain-level or link-presence check can miss the causal mismatch.

This is not the same as a fabricated URL. The harder problem is attribution: which input changed the answer, and does the cited passage support the exact claim attached to it?

The strongest reported results are large, but bounded

The abstract says perplexity filtering and citation-support checking were not sufficient on their own. A source can look linguistically ordinary, and a cited page can appear to support nearby language without being the input that caused the answer.

Source deletion asks the missing causal question

A counterfactual check removes one source and reruns the answer under controlled conditions. If the claim changes only when the uncited or differently cited source disappears, that source is evidence about causation even when the interface credits another page.

The method is more demanding than link inspection. Model decoding varies, retrieval may change, and removing a source can alter context length or ordering. A defensible audit therefore holds the remaining source set, prompt, model and settings constant and repeats each condition.

A practical audit for high-risk AI answers

1. Save the complete prompt, answer, visible citations and source contents.
2. Split the answer into checkable claims instead of grading the response as one block.
3. For each citation, locate the exact passage and record whether it entails, partly supports or contradicts the claim.
4. Create one controlled condition for each source removed individually.
5. Repeat the baseline and deletion conditions enough times to expose decoding noise.
6. Record which claim changes, which citation changes and whether the answer remains correct.
7. Flag any trusted citation attached to a claim driven by another source.

Download the counterfactual citation audit ledger. Its example rows are labeled EXAMPLE-REMOVE. Replace them with sanitized observations and keep private prompts, account data and confidential source text outside the public file.

Publishers should distinguish being cited from supplying the answer

AI-visibility reports often count a domain whenever its URL appears beside an answer. CiteShade adds a harder measurement question: did that source actually drive the claim?

For ordinary brand monitoring, citation presence remains useful. For medical, financial, legal, safety or reputation-sensitive claims, add passage support and counterfactual influence when access to the source set permits it. A trusted domain can receive credit for a statement it did not cause, while the source that shaped the answer remains hidden.

The citation-support audit helps test passage entailment. The citation-selection model separates retrieval, use and visible attribution. CiteShade shows why those layers should not be collapsed.

Use two different audits: the four-layer citation ledger traces retrieval, page selection, re-attribution and validation. CiteShade’s source-deletion test asks a different causal question: which source actually changed the answer? A citation can pass the first audit and still fail the second.

Treat attribution as a claim to test

The paper does not make every AI citation untrustworthy. It demonstrates a failure mode in which visible attribution can hide causal influence from another source.

The corrective action is concrete: verify the cited passage, preserve the full source set and use source-removal tests for consequential answers. A citation is evidence about what the interface credited. It is not automatically proof of what caused the answer.