Detecting a Citation the Agent Never Used

The test has two independent gates. A citation passes only when both gates pass.

The order matters only for cost. Run provenance first because it is a string search and costs almost nothing. Run support only on citations that pass provenance. In a typical RAG pipeline, most failing citations are provenance failures, so this ordering saves a large share of judge calls.

Notice what the support gate deliberately does not receive: the model's reasoning, the other retrieved chunks, the original query, or any confidence language from the answer. It gets the claim and one passage. That isolation is the point. If you hand the judge the full context, it will agree with the answer for the same reason the original model did. Isolation forces the judge to ask a narrow question: does this text, on its own, license this sentence.

The support check is an entailment test. The cited passage is the premise, the claim is the hypothesis, and the judge answers whether the premise entails the hypothesis. Three levels are enough: yes (the passage states or directly implies the claim), partial (the passage supports part of the claim but not all of it), no (the passage is unrelated or contradicts the claim).

Treat partial as a failure for the purpose of the queue. Partial is where the interesting defects live: a passage that supports "revenue grew" being cited for "revenue grew 12% and margins expanded." The citation is not fabricated, which is why naive checks pass it, but it does not support the sentence as written.

Keep the judge prompt rigid and the output format constrained:

You are checking one citation. You have no other information.

PASSAGE:
{passage_text}

CLAIM:
{claim_sentence}

Does the PASSAGE alone entail the CLAIM?
Answer with exactly one word: yes, partial, or no.
Do not explain. Do not use outside knowledge.
If the PASSAGE needs other context to support the CLAIM, answer no.

Two practical notes. First, use a different model for the judge than the one that wrote the answer where you can, so a systematic bias in one model does not confirm itself. Second, calibrate before you enforce. Hand label around 100 citation pairs, run the judge on the same set, and check how often the judge says yes on pairs you labeled no. That number is your false pass rate, and it determines whether the gate is real or decorative. A judge with a high false pass rate will happily approve stretched citations.

There is a warning about judge-based evaluation worth carrying into this design. A chunking evaluation on academic texts found that RAGAs-based faithfulness showed limited reliability in their setup, which means off-the-shelf faithfulness scorers can drift away from human judgment on your corpus. Calibration is not optional.

The support check is also where chunking choices come home to roost. If a chunk is the unit of citation, the citation inherits the chunk's boundaries. Splitting a document badly can separate a claim from its qualification, so a passage that reads "the discount applies" is cited for a claim that applies only to "orders over 500 units" because the qualifier landed in the next chunk. Chroma's evaluation of chunking strategies shows that the choice of chunking strategy has a significant impact on retrieval accuracy and that default settings for some popular strategies perform poorly. A support failure in your queue can be a chunking bug wearing a citation bug's clothes.

The provenance check answers a blunt question: was this passage in the prompt the model actually received, exactly as the model received it? Not in the retrieval results. Not in the vector store. In the serialized prompt.

The implementation is a string search against the prompt you logged, run at the moment of the call:

def provenance_ok(prompt_text, citation):
    # citation carries the chunk id and a verbatim span from the chunk
    if citation.chunk_id not in prompt_text:
        return "absent"
    span = normalize_whitespace(citation.verbatim_span)
    if span not in normalize_whitespace(prompt_text):
        return "absent"
    return "present"

Three failure paths deserve their own reason codes. The chunk identifier is missing entirely, which means the citation points at something that never made it into context. The identifier is present but the verbatim span is not, which usually means the prompt was truncated mid chunk or the passage was rewritten in transit. The span is present but appears only outside the context block, for example in the user's own question, which is a citation to the question rather than to evidence.

Truncation is the one teams miss. Long context windows make it tempting to concatenate many chunks and let the tail fall off when the budget is hit. The model then cites a chunk whose identifier made it into the prompt header while most of its body did not. The identifier check passes; the span check fails. This is why you need both, and why you should log the prompt exactly as sent, including any templating the serving layer applied.

Prompt templates make this harder than it looks. If your template injects chunk text after whitespace normalization, line ending conversion, or XML escaping, a naive substring search on the raw chunk will fail on citations that are actually fine. Normalize both sides identically before searching, and store the normalized prompt alongside the raw one so your audit trail is reproducible.

There is a way to remove most of this check by construction. APIs that support native document citations attach citation locations computed by the API rather than generated by the model. As an explanation of agent citation architecture puts it, the API-parsed locations cannot point at a document you did not provide and cannot point at a passage that does not say what the claim says. Where you cannot use native citations, the prompt search is the substitute.

Most teams treat citation failures as one bucket. Split them into three, because each has a different cause and a different fix.

Fabricated source. The cited document or chunk does not exist, was never retrieved, or the identifier was invented. Provenance fails on the identifier. This is a generation problem: the model was not constrained to a closed set of acceptable citation ids. Fix it with constrained decoding or structured output where the citation field must match one of the ids in the prompt. A comparison of citation generation approaches walks through naive prompting versus structured outputs versus grammar-constrained generation, and the constrained approaches exist precisely to make invented ids impossible rather than unlikely.

Source never in context. The id is real and retrievable, but it was not in the prompt, or its text was truncated away. Provenance fails on the span or the id. This is a plumbing problem, and it is the one that most often gets misdiagnosed as hallucination. Fix it in the assembly step: assert that every cited id was present before serialization, and fail loudly on truncation instead of letting the tail fall off silently. Logging retrieval results is not logging the prompt, and the gap between them is exactly where this failure lives.

In context but does not support the claim. Provenance passes, support returns partial or no. This is a generation or retrieval problem, and you need one more piece of information to tell which: was the passage the best available evidence for that claim in the retrieved set? If yes, the model misread it or overreached. If no, the model cited the wrong chunk and the real culprit is ranking or chunk construction. The work on position bias is a useful reminder that what ranks high gets used, and a passage that sits in position one but only half supports the claim will be cited anyway.

Every citation that fails either gate lands in one queue with its reason code, and the queue is grouped by reason, not by document. Grouping by reason is what turns the queue into a fix list.

Each queue entry should carry the claim sentence, the cited chunk id, the passage text, both gate verdicts with the judge's raw answer, the pointer to the logged prompt, and the reason code. A reviewer needs all of it to make a decision without re-running anything. The approach to automated source verification in RAG pipelines is instructive here: it packages the prompt state for each transaction so verification and audit happen offline against a complete record rather than a reconstruction. It also separates checks that fit different failure types, such as validating numbers in the response against the source materials, which is a cheap deterministic check to bolt onto the support gate for numeric claims.

Reviews fall into three outcomes: accept the citation as is (the judge was too strict, log it as a calibration sample), repair the answer (drop the claim, narrow it to what the passage supports, or attach a better passage), and escalate the underlying cause (chunking, truncation, ranking). Track the third outcome separately, because it is the one that reduces future queue volume rather than clearing today's.

Control the volume. Running both gates on every citation will produce a queue larger than any team can read. Use a tiered policy: deterministic checks on every citation, judge-based support checks on a sample for low-stakes surfaces, and full enforcement on high-stakes surfaces such as legal, medical or financial answers. Research on verifying citation prompts found that prompt design shapes how well a model can judge citation correctness and that a "find the closest fitting citation" style prompt outperformed correction-oriented prompts, with correction prompts even provoking corrective hallucinations. Keep your judge asking whether a passage fits, never asking it to invent a better source.

Track four numbers, weekly, per surface, and report them next to the definition so nobody argues about what they mean.

Two more are worth watching when volume permits. First, support rate broken down by chunking strategy, since chunk boundaries change what a passage can support. Second, support rate by retrieval rank, which tells you whether low ranked passages are being cited as filler. Keep the definitions stable and compare like with like. The guidance on selecting embedding models makes the general point well: you cannot improve what you do not measure, and preregistering what counts as a pass is the only defense against moving the goalposts after the fact.

One cost check before you enforce anything. If you re-encode or re-embed a large corpus, the storage and refresh bill dwarfs the judge cost. Work on embedding storage optimization reports that float8 quantization achieves a 4x storage reduction with minimal measured degradation, and that combining moderate PCA with float8 gets 8x total compression with less impact than int8 alone. Frequency of re-ingest and dimensionality, not the headline per-token price, drive the bill, as a vendor cost comparison lays out. Get the retrieval economics right so the citation gate has a corpus worth verifying.

When citation failures turn out to be model-dependent, running the same gates across several models is the cleanest way to isolate the cause. That is the kind of routing problem Quanta Inference is built for: text, image, video, audio and embedding models through one OpenAI-compatible key, so you can swap the answer model or the judge without rebuilding the harness.

Build the pieces in this order and you get a working gate before you get a pile of dashboards.

1. Log the serialized prompt exactly as sent, with a request id, and store it immutably. Everything else depends on this artifact.
2. Require structured citations with a chunk id and a verbatim span. Reject answers whose citation field does not parse.
3. Ship the provenance check first. It is deterministic, cheap and catches truncation and fabrication immediately.
4. Add the support judge on a sample, calibrate against 100 hand labels, and only then widen coverage.
5. Route failures into the reason-coded queue, and give each reason an owner before the first entry arrives.
6. Watch the four metrics, and treat a rising end-to-end pass rate that coincides with a falling queue repair rate as a warning rather than a win.

The test is not perfect. It cannot read the model's mind to prove the citation caused the answer, so the faithfulness problem stays partly out of reach. What it can do is make the two most common lies expensive: pointing at a source that was never there, and pointing at a source that says less than the sentence claims.