When a Downstream Agent Returns Confident Garbage

The hop is a component, not a prompt. Put it in the call path, not inside the producer's instructions, so the producer cannot talk its way past it.

caller --> producer --> verification hop --> caller
                         |
                         +-- reject (return to producer with reason)
                         +-- repair (apply deterministic fix, then re-check)
                         +-- escalate (hand to human or fallback path)

The hop does four things in order, and the order matters because each step is cheaper than the next:

1. Contract check. Does the output parse, carry every required field, and use the declared types? Structured output types are the right tool here: a framework with a typed output model forces the response to match a schema or call an output function, and validates the returned data before your code ever sees it Pydantic AI output. This step catches the loud failures that fault injection already covers. Keep it, then move on.
2. Provenance check. For every factual claim, does a retrieved document id support it? If the producer was supposed to cite three sources and cites none, the output fails regardless of how good the prose reads. Multi-agent trust research keeps returning to this point: a downstream agent may receive instructions it cannot independently verify, and the missing control is verifiable delegation, with provenance logged at each hop multi-agent trust boundaries.
3. Fixture check. Run the output against labelled fixtures using a judge that is deterministic where possible. Exact matching handles identifiers and numbers. For prose, use a narrow judge model with the label definitions in its instructions and a fixed rubric, not an open-ended "is this good".
4. Rule application. Reject, repair or escalate, using the rule below.

One design constraint that is easy to miss: the hop must read the same context the caller will. An output that is wrong relative to earlier constraints in a long session looks fine in isolation, which is exactly how context loss survives review Latitude. Pass the original request, the retrieved documents and the producer's output into the hop together.

A worked reject, repair, escalate rule

The rule needs three inputs, all of which the hop already has: the label with the highest severity found, the number of repair attempts already made, and the blast radius of the action the caller is about to take.

• Reject when the output failed the contract check or hit a fabricated_entity label. Return the failure to the producer with the specific reason and the supporting document ids it ignored. Cap retries at two. Retry loops that re-send full context burn tokens and rarely converge, which is why resilience guidance treats retry amplification as a failure in its own right Preventing Cascading Failures.
• Repair when the fix is deterministic and safe. Normalize a unit, correct a sign against the source data, trim a truncated field, or replace an unverified name with a null and a note. Repair must be code, not a second model call, otherwise you have replaced a check with another guess.
• Escalate when the output hit stale_fact or scope_creep, when repairs have already been applied twice, or when the caller's next action is irreversible (a payment, a deletion, an outbound message to a customer).

Worked example. A support agent drafts a refund confirmation. The producer returns: "Your refund of 4,500 USD has been approved." The hop runs:

1. Contract: parses, has amount, currency, order_id. Pass.
2. Provenance: order_id matches a retrieved order. The retrieved order row says 450.00 USD. Fail on the amount, label unit_or_sign.
3. Fixture: the closest fixture is the classic off-by-100 scale error, same label. Fail.
4. Rule: the caller is about to email the customer, so blast radius is high; the label is deterministic, so repair is allowed. The hop rewrites the amount from the order row, re-checks, and passes the corrected draft.

Total added cost: one lookup and a rule evaluation. A model call is only spent in the ambiguous branches.

The contrast is worth stating plainly. Fault injection would have reported this workflow as healthy, because every tool returned 200 and every step completed. The verification hop caught it because it compared meaning against a known-bad pattern.

A verification hop is a classifier with a cost curve. State the curve before you pick a judge, because the judge choice is the whole design.

Run the cheap judges first and reserve the model judge for outputs that survive them. Two details keep the judge trustworthy. First, freeze the rubric in the fixture repo and version it next to the labels, so that a change to what counts as scope_creep is reviewable. Second, measure the judge against your fixtures on a schedule: a judge that fails fixtures it used to pass is a regression, even if the producer never changed.

There is a second argument for cheap judges. Cascade failures in agent systems are expensive in a way that traditional systems are not, because every retry re-sends full conversation context and burns tokens at every step Preventing Cascading Failures. A verifier that costs a model call for every output, on top of a producer that retries twice, can double the bill of a workflow whose actual task is a single lookup. Deterministic checks first is a cost decision as much as a quality decision.

What the hop should log

The hop's logs are how you learn. Log the delegating producer, the receiving caller, the task correlation id, the label found, the rule applied and the outcome multi-agent trust boundaries. That set lets an investigator reconstruct why a downstream action was trusted.

Two derived metrics matter more than raw counts:

• Fixture hit rate by label. A rising fabricated_entity rate after a retrieval index change is a signal, not noise.
• Repair rate. If most outputs need repair, the producer's instructions or retrieval are broken, and the verifier is doing work the producer should be doing.

Set an alert on the second one. A verifier that repairs 40 percent of outputs is a tax on every call, and the right fix is upstream.

A verification hop is a trust boundary control, and it makes more sense when you see it next to its neighbours. The industry risk catalogs are explicit that compromise in one agent propagates to others through shared resources, communication channels and state corruption, with horizontal, vertical and chain-reaction patterns FINOS multi-agent trust boundaries. A verification hop is the read-side equivalent of that control: it limits what a corrupted upstream agent can do to a downstream one without being noticed.

Security guidance for agents also converges on the same three layers, which is a useful check that you have not over-built: least privilege, approval gates on high-impact actions, and monitoring behavior patterns rather than single events monday.com AI agent security. The verification hop belongs to the third layer, with a foot in the second when it escalates.

There is a plausible objection: if the verifier is a model, an attacker who can influence the producer's output can influence the verifier too. That is why the fixture check runs after the provenance check, and why deterministic judges carry the load wherever the label allows. Prompt injection remains the top risk on the OWASP list precisely because models do not reliably separate trusted instructions from untrusted content, and retrieval-augmented generation reduces but does not eliminate the exposure Prompt Injection Prevention Cheat Sheet. A judge that reads the producer's output unsupervised is a model reading untrusted content. Treat it as one.

The practical consequence: never let the verification hop call tools. It reads, compares and routes. Writing is the caller's job, after the hop returns.

Start narrow. Pick the one handoff where a wrong-but-confident answer costs the most: the draft that reaches a customer, the record that updates a ledger, the summary that feeds a decision. Put the hop there first, with at least three fixtures per label, exact-match checks for every identifier and number, and escalation as the default for anything unlabeled.

Roll out in three steps that each take a day, not a quarter:

1. Shadow mode. The hop logs its verdict and lets every output through. Compare the hop's reject rate against your incident log. If the hop flags nothing you have an incident for, the fixtures are too generic.
2. Repair-only mode. Enable deterministic repairs. Measure the repair rate and the producer's retry count. Both should fall as instructions and retrieval get fixed.
3. Enforce. Turn on reject and escalate for the labels you have fixtures for, and nothing else. Expand only when a new incident produces a new fixture.

Workforce readiness deserves a line because it is where these projects quietly die. Adoption of agents, not the technology, is the step that produces value, and change programs that skip it tend to leave agents without owners CIO change management. The verification hop needs a named owner, a fixture review on a fixed cadence, and one reviewer with authority to overrule the producer. That is a Friday afternoon recurring meeting, not a platform team.

Then review the whole chain once. Map every handoff, decide which ones get a hop, and record why the others do not. A written decision not to verify is a control; an unwritten one is a hope. If the chain crosses several vendors and model families, one API for text, image, audio and embeddings keeps the call path uniform, which is what makes a single verification hop practical to operate across every producer Quanta Inference.

The one thing to avoid is a verifier that only checks what is easy to check. Format and schema checks are already covered by typed output models and by fault injection. The value of the hop is entirely in the semantic checks, driven by fixtures from your own incidents. If the hops and the fixtures disagree, trust the fixtures and rewrite the hop.