When the Reviewer Cannot Check the Source: A Fallback Screen

Design the fallback panel around five fields. Each answers a question the reviewer would otherwise have to guess at.

The discipline behind this list is human-in-the-loop design, which the enterprise UX literature defines as a structured approach where humans actively participate in producing, evaluating and refining AI-generated outcomes, with the interface making that collaboration explicit and continuous Aufait's HITL UX guide. The same source names the behavioral risk the screen exists to counter: automation-induced complacency, where vigilance fades because the system usually works, surfacing as automation bias, complacency, skill erosion and blind spots Aufait. A citation that renders identically whether or not anyone checked it feeds exactly that complacency.

Two rules keep the panel honest. First, never render an unverified citation in the same style as a verified one. Status must be visible at a glance, not buried in a tooltip. Second, state what is missing in the reviewer's language, not the system's. "The publisher returned a paywall on the fetch attempt and no passage was stored" is useful. "Status: partial" is not. The trust-signal guidance from AI product discovery work is pointed here: require that each use case carries a specification for how trust is signaled, because users abandon AI features they do not trust regardless of model quality reloadux on AI opportunity deliverables.

Not every uncheckable source is uncheckable in the same way. The fallback panel should branch on cause, because each cause has a different remedy and a different default severity.

The mapping matters because reviewers cannot triage what they cannot name. An internal document store that will not open is a different problem from a journal paywall, and both differ from a DOI that resolves to nothing. Give the reviewer one clear sentence per case, a suggested next step, and the option to attach a human-found copy. The escalation path for cases outside the standard flow should be a first-class part of the design, not a support ticket, because governance frameworks treat escalation paths for edge cases as one of the defining elements of governed human-AI collaboration Dataiku's governance design patterns.

One additional distinction earns its own branch: ambiguity. A source can exist, have been read, and still not support the claim. The EvalCommunity checklist treats this as a separate failure (wrong claim, wrong summary, missing limits) requiring its own check rather than a re-fetch EvalCommunity. Ambiguity is the only mode where the fix is a domain expert, not a librarian, so route it differently.

Most teams already agree that risky claims need review. What they lack is a written threshold that a build actually implements. Databricks' guidance is to make human review risk-based rather than universal: reserve it for high-impact, uncertain or regulated decisions, and use confidence thresholds and risk scoring so only a subset of decisions routes to a person Databricks on human-in-the-loop. The fallback screen is the exception handler inside that routing.

Write the policy as an ordered list of conditions, evaluated before the approve control is enabled:

BLOCK = {"number", "quotation", "high_stakes"}

def gate(claim, source):
    if source.status == "verified":
        return "approve"
    if claim.kind in BLOCK:
        return "block"          # human must supply reachable copy or alternative
    if source.status == "paywalled" and claim.kind == "background":
        return "warn"           # reviewer may approve with a note
    if source.retrieval_record is None:
        return "block"          # nothing was actually retrieved
    return "warn"

Three classes always block when the source is unverified: numbers, direct quotations and claims labeled high-stakes. The reasoning is asymmetric harm. A paraphrased background sentence that turns out to be slightly off is an editing problem. A wrong figure in a board paper or a misquoted sentence in a legal filing is an incident. The rigorous-mapping literature is direct about the cost of skipping this kind of grounding: roadmaps built on assumptions rather than observed reality produce features users abandon and months of misdirected effort reloadux. Substitute "citations" for "features" and the argument holds.

For the warn class, do not let approve be the only exit. Record who approved on what basis and what remains unconfirmed, so the decision survives an audit. Anaconda's annual data science report found that over half of organizations have no AI governance framework in place Dataiku; the lightweight version of a framework is a log of approvals and overrides written at the moment they happen. Governance that functions only as an extra approval hurdle gets bypassed or abandoned, which is why controls belong inside the workflow rather than in a separate gate in front of it Dataiku.

Also expect the block rules to touch published outputs, not just drafts. Dawgen's account of a broad legal duty to detect synthetic content, and the rising penalties behind it, means an organization that ships a citation it never opened is exposed long after the draft closed Dawgen.

Take a 40-claim draft and run it through the fallback screen before an editor sees it.

• Citations attempt to resolve 40 sources. 32 open directly. 5 return a paywall notice. 2 URLs return a not-found error. 1 resolves to a page with no matching text.
• Of the 40 claims, 11 are numbers, 6 are direct quotations and 14 are labeled background. The other 9 are internal references to a document store the model never touched.
• The screen blocks every number and quotation among the 8 unresolved sources. Say that accounts for 6 claims. It clears the background claims among the paywalled set for revision with a note. Say that accounts for 2 claims.
• The reviewer now has 32 verified claims, 6 blocked claims needing a human-supplied source, 2 warnings to annotate and 0 silent unknowns.

That arithmetic is the argument. The screen converts a hidden risk distribution into a short, numbered work list. It also gives you a ratio to track: the share of citations that fail to resolve. Dawgen reports that the gap between what regulators and courts expect and what practice delivers is the ordinary condition of fast-moving technology Dawgen. Measure your own failure rate so you know whether the gap is closing or widening.

The same appraisal logic appears in the process-mapping literature. Procedures that map tasks, decisions and handoffs across people and systems make weak spots visible before they become incidents Asana's guide to process mapping. Process mapping exists to communicate how work actually gets done, not how it is supposed to happen, and it is most useful precisely where a handoff is invisible Asana. The fallback screen is process mapping for one handoff: the moment a reviewer decides whether to trust a citation.

A second worked example sets the threshold. If 200 citations a week resolve in 190 cases, then 10 per week hit the fallback. If half of those are paywalls on background claims, the warn path absorbs 5. If 2 are numbers or quotations, the block path absorbs 2. Blocking 2 claims a week is a queue a named editor can clear; blocking 100 would stall publication. The ratio tells you where to invest: if the paywall failures cluster on one publisher, negotiate access; if the not-found errors cluster on one workflow, fix the prompt.

The screen has done half its job when it blocks a claim. The other half is the intake path for the human-found copy. Keep it short:

1. The reviewer uploads or links an accessible copy and records where it came from.
2. The reviewer highlights the passage that supports the claim, or states that no passage does.
3. The system records the linking decision with the reviewer's name and a timestamp.
4. The citation status flips to verified, and any downstream documents citing it are marked as affected.

Step four is the one teams forget. If a number appears in five places and one reviewer clears it, all five need the correction or the others stay unverified. Build that propagation in from day one. The verification guide's six-step workflow makes the same final demand: record the check, so a reader in six months can see what was confirmed and by whom EvalCommunity.

Human review is only trustworthy if it is itself documented. Anaconda's finding that most organizations lack a governance framework is, in practice, a finding that most approvals happen without a trail Dataiku. The intake path is where your organization decides to leave one.

Log every fallback event: the claim type, the failure mode, the reviewer's decision and the time to resolution. That log turns a safety feature into training data. With a monthly read, you can answer questions like which publishers produce the most paywall blocks, whether invented citations cluster around one workflow, and how long a blocked claim waits for a human. Databricks makes the operational point that human feedback must be captured, governed and used to improve agent behavior over time rather than left in disconnected review workflows Databricks.

Two thresholds are worth setting from the log. First, a target resolution time for blocked claims in high-stakes documents, so approval is not held hostage by an external publisher indefinitely. Second, a retire rule for citation patterns that repeatedly fail to resolve: retire the prompt fragment or add the source to an approved list. Both thresholds start as guesses and become evidence.

That closes the loop in the way rigorous discovery work describes: grounding a roadmap in observed behavior rather than assumptions, so that the gap between the plan and reality shrinks each cycle reloadux. A citation pipeline is a process, and the fallback screen is where the process is stress-tested. If you want that work handled end to end by a specialist who reads your context, Quanta Experts can take on the review workflow itself.