A True Fact Can Still Be the Wrong Answer When an AI Cites the Wrong Source

Imagine a fictional support assistant that says, “Your replacement ships tomorrow,” and cites a customer record. The shipment date is real—but it belongs to another person whose case looks similar. The sentence may be true somewhere in the database and still be the wrong answer to this customer.

Evidence has an identity

Many answer systems retrieve snippets, summarize them and attach citations. A basic check can ask whether the answer’s words resemble retrieved text. That helps, but it can miss a more specific failure: the statement is supported by one record, while the answer attributes it to a different source.

This is not a narrow formatting defect. In a support queue, the identity of the record is part of the meaning. “A shipment is delayed” is different from “your shipment is delayed.” A claim about a policy document is different from a claim about a customer’s individual exception. Similarity can pull the answer toward the wrong neighbor.

Multiverse Computing describes ProvenanceGuard as checking claim support and named-source attribution in MCP traces. Its account reports a held-out test involving 361 claims from 40 answers, with source identification around 86% where sources were identifiable; a harder test with similar sources reports 50.3% exact-source identification. These are separate settings, and neither is a general measure of all hallucinations or an assurance for arbitrary systems. The ProvenanceGuard account

The harder result is especially instructive. Exact source attribution becomes difficult precisely when plausible records resemble one another. In many real workflows, the difficult case is the one worth testing: same product, similar date, overlapping names, or a policy with a general rule and a specific exception.

Separate factual support from attribution

Consider two fictional support records. Record A says a blue model ships on Friday. Record B says a green model ships on Tuesday. A user asks about the green model. An assistant might generate “Your order ships Friday” and cite Record B because the product name matches. The date is a fact in the corpus, but the cited record does not support it.

Pooling evidence can hide this boundary. If the model sees both records as one convenient context window, it may combine the right product from one with the right date from another. A verifier that asks only “does the answer appear in the retrieved context?” can pass the merged claim. The check must preserve source identity through retrieval, generation and answer review.

What an inspectable trace can show

A trace should record observable information: which tools ran, which source identifiers they returned, what claims the system produced, and which evidence spans were linked to each claim. It can also record the verifier’s disposition, such as supported, ambiguous or unsupported. This is enough to investigate provenance without exposing hidden internal reasoning.

Conceptually, an event might look like this:

claim: "The green model ships Tuesday"
source_id: "case-B"
evidence_span: "estimated ship date: Tuesday"
check: supported

This schematic describes the idea; it is not a required product schema. In a real system, the source identifier must be stable enough to resolve the correct document version and access policy. The evidence span should be specific enough that a reviewer can find the supporting text, not merely the whole database row.

The answer can then be repaired when the check fails. If another source supports the statement, cite that source and confirm it belongs to the current question. If the evidence is similar but not decisive, qualify the answer or ask a follow-up. If no source supports it, remove the claim. A system should not “fix” a citation by attaching whichever record happens to contain the same phrase.

A verifier needs permission boundaries too

Source-aware checking can involve sensitive records. A verification step should not gain broader access than the answer system itself needs. If the original workflow is allowed to retrieve only a user’s current case, a verifier should not search every customer record to find a convenient support for the generated claim.

This has a practical consequence for MCP tools. Each tool response should retain enough identity and provenance metadata for the verifier to evaluate attribution, while access control remains enforced at the source. Redacting or transforming content may be necessary; if that removes the evidence needed for an exact check, the output should say verification is limited rather than implying certainty.

The system also needs to handle multiple legitimate sources. A company policy can support a general rule while a customer record supports an exception. The answer should link each statement to the relevant source, and the verifier should not collapse them into a single undifferentiated citation list. Claim-level attribution lets readers see which document carries which part of the conclusion.

Evaluate the failure you care about

Aggregate scores make convenient headlines, but a team should assemble cases from its own retrieval risks. Include near-duplicate records, stale versions, policy-plus-exception pairs, missing evidence and claims that are true only for a different entity. Ask reviewers to label not just whether a statement is correct, but whether the cited source supports it for the intended subject.

Measure at least three outcomes separately: factual correctness, source identification and safe handling of ambiguity. A system may get the fact right while misattributing it; it may identify the correct source but overstate what the source says; or it may abstain too often. Those are different tradeoffs and call for different changes in retrieval, prompting or verification.

What attribution checks cannot settle

A source can be correctly cited and still be wrong, obsolete or incomplete. Provenance verification answers a narrower question: does this evidence support this claim under the tested conditions? It does not decide whether the source should be trusted, whether the data was entered accurately, or whether the answer is appropriate for a high-consequence action.

The identity of evidence matters beyond chat answers. The OCR layout explainer considers which page region supports an extracted field. The ultraviolet sky-map story considers how a visualization can distinguish observed data from inferred completion.

That narrower guarantee is still useful. An answer system should make it possible to distinguish “we found a matching phrase” from “this named source supports this claim.” When it cannot resolve the distinction, it should leave the uncertainty visible. A true fact with the wrong source may mislead more than an obvious fabrication because its citation gives the answer an undeserved appearance of care.