When the Agent's Summary Replaces the Source

The decision rule for the screen is simple: a reader should be able to tell, without reading a word, which pixels are the model and which are the source. The most reliable way to achieve that is three visual layers with distinct treatments.

Layer 1: the interpretation. The agent's summary lives here. It should look like a draft, not a verdict: a distinct surface (a tinted panel or a bordered card), a label that names the author ("Agent summary"), and sentence-level structure so each sentence can carry its own status. Do not give this layer the page's primary heading treatment. It is commentary on the record, not the record.

Layer 2: the quoted evidence. Direct quotations from the retrieved spans, with visible quotation marks, source attribution and offsets. This layer carries the authority. If a summary sentence expands on a quote, the reader sees the gap between the two in a single glance because the two are adjacent but visually different.

Layer 3: source metadata. Document title, section, retrieval timestamp, the model and prompt that produced the summary, and the character offsets of each cited span. This layer makes the whole thing auditable. A Dataiku survey of more than 800 global data leaders found that only 5% say AI output is traceable 100% of the time, a gap that leaves most organizations unable to produce the audit trail a regulator would require as reported in Dataiku's governance design patterns article.

The following table shows how the layers should differ, so the distinction survives a quick scan.

If a designer cannot fill in every cell distinctly, the layers will blur on screen and the reader will fall back to trusting the summary by default.

The layout separates claim from record. Anchoring connects them. Citation extraction, as described by ZeroEntropy, maps each claim in a generated answer back to the specific span in the retrieved context that supports it, producing structured output of document ID and character offsets per claim in their citation extraction explainer. That structured output is what the screen needs. Without offsets, "the source says so" is just another claim.

The same source explains why inline citation during generation is unreliable. Models pick a plausibly related span rather than the actually supporting one, invent citation tokens that do not correspond to a real span, and attribute a claim to a single span when support is spread across several. Even with strong prompting, inline citation accuracy hovers in the 85 to 95% range for top models, per that article, which is high enough to look right and low enough that roughly one in ten to one in twenty citations is misleading. The recommended pattern is post-hoc extraction: generate the answer first, then run a separate extractor over each claim plus a small candidate context, returning the supporting span or a null result.

For the review screen, the null result is the most important output. A sentence with no supporting span must not be silently dropped and must not be left unmarked. It should render with an explicit unverified state, in the summary layer, with the same prominence as a cited sentence. The EvalCommunity rule applies at the level of interface: if you cannot verify the source, do not let it borrow the authority of one from their verification workflow.

A minimal anchor record for the screen looks like this:

{
  "sentence_id": "s3",
  "summary_sentence": "The vendor's liability cap excludes indirect damages.",
  "anchor": {
    "doc_id": "contract-2024-11",
    "section": "7.2 Limitation of Liability",
    "start_char": 14820,
    "end_char": 14996,
    "quote": "In no event shall either party be liable for indirect, incidental, or consequential damages."
  },
  "verification": "supported",
  "model": "<generation model id>",
  "retrieved_at": "<ISO timestamp>"
}

The verification field should have exactly three values on screen: supported, partially supported, and unverified. Partially supported matters because real claims often combine a supported fact with an inference. The ZeroEntropy piece notes that a claim may rest on support spread across multiple spans, which the interface should show as a list of anchors rather than a single pointer from the same explainer.

A study at CHI on traceable text tested summaries linked to source documents at the phrase level and found that readers answered questions about the source more quickly and more accurately when summaries contained hallucinations. On questions about hallucinatory content, correctness rose from 12.5% to 70% with the traceable design, and participants used the links both to check claims and to navigate into the source as reported in the ACM paper on traceable texts. That is the strongest argument available for building the link into the screen rather than relying on reader diligence.

The interaction should be one tap, not a search. Tapping a summary sentence opens the raw passage in its original context, scrolls the relevant span into view, and highlights it. "Original context" means the full section around the span, not a cropped snippet, because cropping is how retrieval itself can mislead: a sentence that reads one way alone can reverse meaning beside the preceding clause. The traceable text study's example makes the point in miniature, linking a summary phrase about mildly swollen neck glands to the original clinical note reading mild cervical lymphadenopathy without thyromegaly from the same ACM paper.

Three implementation details decide whether this works in practice. First, preserve scroll position and return path, so the reader gets back to the same sentence; otherwise the tap becomes a one-way trip and people stop taking it. Second, highlight the span, not the whole document, because a highlighted span gives a target for comparison while a highlighted page gives homework. Third, show the quote and offsets inline in the summary layer as well, so short quotes can be verified without leaving the page; the tap is for context and for full documents.

Consider a research agent that summarizes 40 retrieved passages into a 30-sentence briefing. Assume post-hoc extraction with an inline-style accuracy of 90%, at the top of the 85 to 95% range reported for top models per ZeroEntropy. That implies about 27 sentences with a correct anchor and 3 with a wrong or missing one.

The traceable design's advantage is not that readers check everything. It is that checking is cheap enough to do on the sentences that matter, and the visual marking of unverified sentences tells them where to look first. The arithmetic of review effort is the whole argument: if each check costs one tap and two seconds of reading, a reviewer can check all 30 sentences in about a minute; if each costs a sidebar search, they will check three.

A second number worth putting on the screen is coverage. Show the count of anchored sentences over total sentences, for example "27 of 30 sentences anchored." That ratio tells the reader how much of the summary is checkable at all, and it gives the team building the agent a metric to improve. Braintrust's guidance on human-in-the-loop evaluation makes the related point that automated scorers handle high-volume checks while human reviewers focus on the cases automated checks handle less reliably, including factual disputes and edge cases in their eval workflow guide. Anchoring coverage is the automated check; the reader's tap is the human review, pointed at the right sentences.

Most review screens are designed around the happy path, and the unverified state gets whatever styling is left over. That is backwards. The unverified sentence is the one the reader most needs to see, so it should be the most visually distinct element on the page.

Three rules make the state legible. First, never render an unverified sentence inside the same citation chrome as supported ones: no badge, no link, no quote, but a different treatment entirely, for example a dashed border and the label "no supporting passage retrieved." Second, keep the sentence in place and do not hide unverified content behind a filter, because a reader who has to click "show unverified" will not. Third, make the absence explicit in the metadata layer too, so the record shows that no span was found rather than leaving a blank that looks like an omission. The EvalCommunity checklist's instruction to write "not provided" rather than inventing a detail is the same principle applied to a form field from their checklist.

The interface should also be honest about the limits of its own explainability. The NN/g piece notes that modern models are complex enough that even AI engineers cannot always accurately trace the reasons behind an output, and that explanations offered by chatbots are often inaccurate, hidden or confusing in the NN/g analysis. A span anchor does not claim to explain why the model wrote the sentence. It claims only that this passage was retrieved and this sentence paraphrases it. That narrower claim is the one the interface can actually keep.

The screen is the visible end of a small pipeline, and each stage has a decision that shows up in the UI.

Retrieval. Store the exact passages retrieved, with document IDs and character offsets, not just the document. The offsets are what make highlighting possible.

Generation. Produce the summary sentence by sentence, so each sentence has a stable identifier before anchoring runs.

Anchoring. Run a post-hoc extractor per sentence against the top few retrieved passages, returning document ID and span offsets or null, as described in the ZeroEntropy pattern from their explainer. Evaluate the extractor as binary span retrieval with precision, recall and F1, and use intersection-over-union thresholds for partial matches.

Recording. Log the anchor set, the extractor's decision, the reader's action and a timestamp. This is the audit trail that Dataiku's governance patterns call for and that the 5% traceability figure suggests most organizations lack per Dataiku.

Rendering. Three layers, sentence-level status, one-tap spans, preserved return path.

If your team runs many kinds of models behind these agents, text for summaries, embeddings for retrieval, and possibly image or audio inputs for multimodal sources, the extraction and generation steps can share one integration. Quanta Inference routes text, image, video, audio and embedding models through a single OpenAI-compatible API key with prepaid US-dollar pricing, which keeps the anchoring pipeline on one contract instead of several: https://quantaintelligence.ai/inference

The screen itself is the deliverable that matters. A summary without a path back to its evidence is a claim dressed as a record, and the review screen's only real job is to keep that distinction visible.