Business

When the Gate Says No: The Agent's Failed-Verify Escalation Note

Antonio Evans10 min read

A split scene: on the left, an agent process stops at a gate marked with a red status light; on the right, a reviewer at a desk opens a single structured ticket that shows the gate check, the input digest.
Key points

Key points

  1. 1A failed verification should arrive in the human queue as a self-contained ticket, not as a link back to a run the reviewer cannot see.
  2. 2The minimum field set is run and gate identifiers, the exact check and threshold that tripped, an input digest, a failing output excerpt, the tool call trace, an environment fingerprint, a one-command reproduction recipe, and a proposed decision.
  3. 3Sparse failures cost more than they save, because the reviewer re-runs discovery that the agent already did, and agents that report success after a silent failure make that rework invisible until a customer complains.
  4. 4Severity, owner and a required resolution code turn the queue from a pile of open items into a dataset you can use to tune the gate itself.

The failure mode the gate was supposed to prevent

A verification gate exists to stop bad output before it reaches a person who will act on it. The gate does its job and raises a hand. Then the ticket arrives in the human queue and says something like "verification failed on run 8891, see logs." The reviewer opens the logs, finds a truncated transcript, and starts guessing. Within ten minutes the reviewer is doing exactly the work the gate was meant to front-load: reconstructing the input, hunting the tool call that returned the odd number, and asking whether the run even used the configuration anyone assumed.

The cost is not just the ten minutes. It is that the reviewer has no reliable way to distinguish a genuine defect from a stale artifact, so every ticket becomes an investigation. Agents fail in production in a small number of recognizable ways, and the ones that matter most here are tool misuse (right tool, wrong arguments), silent partial success (the agent reports done after a step failed), and stale inputs (confident answers built on data that stopped updating) as catalogued by Winder. Each of those leaves a fingerprint, but only if the escalation note records it.

There is a second, less obvious cost. Verification is only as good as the reviewer's ability to reach an independent judgment. Guidance on fact-checking AI output is consistent on this point: the reviewer must leave the AI output and consult sources outside it, because AI content is a composite of unattributable sources and you cannot assess the source, only the claims per the University of Maryland's lateral reading guide. A ticket that only says "the gate failed" forces the reviewer to trust the gate, which inverts the whole point of having a human in the loop. The ticket has to hand over enough raw material for the reviewer to disagree with the gate.

Why a pointer is not a ticket

The instinct to store everything in a trace store and link to it is reasonable, and it fails for three practical reasons.

First, retention and access diverge. Trace stores are pruned, sampled or access-controlled differently from the work queue, so the link that worked the day of the failure is dead three weeks later when someone audits the gate. Second, the reviewer's role is not the agent's role. Handoff design guidance is blunt about this: a handoff note must separate customer statements, verified system facts, AI inferences and completed actions, and a polished summary must not conceal a failed tool call or an unverified assumption per Salted's handoff guide. Third, the reviewer needs a decision, not a history. A pointer delivers history.

The same guidance makes a structural point worth stealing: a handoff ends with an accepted owner, not a transfer message, and the workflow should treat requested, queued, accepted and resolved as distinct observable states from the same handoff guide. Most failed-verification queues collapse all four into "open," which is why nothing ages correctly and nobody can tell whether a ticket was ever really looked at.

There is a numerical case too. If a reviewer needs 12 minutes to reconstruct context on a sparse ticket versus 4 minutes on a complete one, a queue of 40 tickets per week costs roughly 8 hours of pure rediscovery. That is a full working day spent re-deriving facts the agent already held in memory at failure time. The agent had the input, the arguments, the environment and the threshold. Not shipping them is a data loss decision, not a simplification.

The escalation schema

Here is a field set that fits in a JSON object and reads well as a rendered ticket. Treat it as a contract: if the gate cannot populate a field, it must write an explicit null with a reason rather than omitting the key, so the reviewer knows the difference between "not applicable" and "lost."

{
  "ticket_id": "esc-2026-04-1187",
  "run_id": "run_9f3a2c11-b7",
  "gate": {
    "name": "citations_resolvable",
    "version_ref": "gate-def@sha256:4e1b…",
    "check": "every cited URL returns 200 and matches registered domain",
    "observed": "3 of 41 citations returned 404",
    "threshold": "0 unresolved citations permitted",
    "trip_point": "citations[17], citations[28], citations[39]"
  },
  "input": {
    "digest_sha256": "b1c9…",
    "source_ref": "s3://kb/2026/04/inputs/b1c9.json",
    "byte_size": 184223,
    "redactions_applied": ["customer_email"]
  },
  "output_excerpt": {
    "span": "citations[15:20]",
    "text": "…as documented in the vendor guide (https://…/retired-page)…"
  },
  "tool_trace": [
    {"step": 14, "tool": "http_fetch", "args_digest": "77ae…", "status": 404},
    {"step": 15, "tool": "http_fetch", "args_digest": "91bd…", "status": 404}
  ],
  "environment": {
    "agent_image": "agent@sha256:0c2d…",
    "knowledge_snapshot": "kb-2026-04-01",
    "tool_registry_rev": "tr-8821",
    "region": "us-east-2"
  },
  "reproduce": "quantarepro run --ticket esc-2026-04-1187 --offline-cache",
  "proposed_decision": {
    "action": "reject",
    "reason": "citations point to pages retired after knowledge snapshot; refresh snapshot or drop claims",
    "confidence": "medium",
    "would_accept_if": "all cited URLs resolve against kb-2026-04-15"
  },
  "severity": "medium",
  "owner": "content-verification",
  "resolution_code": null
}

Three fields deserve explanation because they are the ones teams skip.

The input digest is a hash of the exact bytes the agent saw, plus a pointer to where those bytes live. Without it, two runs with the same prompt but different retrieved passages look identical in the queue, and the reviewer cannot tell whether a re-run will reproduce the failure. Hashing the input rather than storing it inline also keeps sensitive material out of the ticket body, which matters because handoff guidance warns against putting unrestricted copies of private records into a handoff note per Salted.

The reproduce field must be one command that a reviewer can paste. Idempotency guidance for long-running agents makes the same distinction that applies here: keep separate identifiers for the business operation, the execution attempt and the provider's reference, because they answer different questions and collapsing them into one ambiguous request ID makes a restart untraceable per PADISO's idempotency tutorial. The ticket ID, run ID and gate definition hash are exactly those three identities.

The proposed decision is the agent's own recommendation, stated as a decision rather than a summary, with a would_accept_if field that tells the reviewer what evidence would flip it. This is the cheapest way to make the gate auditable: the reviewer either accepts the proposal or records why not, and both outcomes are useful signal.

FieldWhy the reviewer needs itWhat breaks without it
run and gate identifiersTies the ticket to a reproducible execution and a specific gate definitionGate changes cannot be correlated with failure patterns
check and thresholdTells the reviewer which rule tripped and how far off it wasReviewer cannot distinguish a near miss from a systemic break
input digestConfirms the reviewer is judging the same bytes the agent sawRe-runs silently test different inputs
output excerptShows the failing text in context, not just a flagReviewer must fetch the whole output to understand one line
tool traceExposes wrong arguments and failed callsSilent partial success is invisible
environment fingerprintSeparates "the model failed" from "the environment changed"Configuration drift gets misattributed to the model
reproduce commandOne paste to observe the failureReviewer re-runs discovery by hand
proposed decisionGives the reviewer a default to accept or rejectEvery ticket starts from zero

Reading the tool trace for the three failures that hide

The tool trace is the field that earns its place most often, because it is where silent partial success stops being silent. The pattern to look for is a step whose status is not an error but whose result does not support the output. A fetch that returned a redirect page, a database query that returned zero rows, a search that returned results from the wrong index: none of these throw, and all of them produce confident prose downstream.

Before you can diagnose that, you need argument-level traces, not tool-name traces. The Winder taxonomy is precise about why: picking the wrong tool and calling the right tool with wrong arguments look identical in a log that records only tool names, and they have completely different fixes per Winder. Their reading of the MAST annotation study, which covered 1,642 execution traces from seven open-source multi-agent frameworks, puts reasoning-action mismatch at 13.2% of observed failures and disobeying the task specification at another 11.8%. Roughly a quarter of everything that went wrong is a version of "the call did not match the intent," which is exactly what a trace with args_digest captures and a trace without it does not.

The same source's fix list maps onto ticket fields neatly: verify outcomes externally rather than trusting the agent's self-report, which is what output_excerpt plus tool_trace enables; enforce hard step and cost ceilings in the loop, which is what makes a trip_point meaningful; and treat integrations as monitored dependencies with freshness checks, which is what knowledge_snapshot in the environment fingerprint records.

There is a security dimension to the trace too. An authorization layer that evaluates a tool call before execution is complementary to, not a substitute for, the audit record: the same infrastructure that enforces spending limits and capability scoping also enforces quality gates and produces a signed record of the decision per the pre-action authorization paper. If your gate runs after execution, the ticket is your audit record, and it should carry the same correlation discipline: which identity acted, on what resource, under which policy.

For teams running many agents, the escalation note is where those records converge. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API key, which means a run identifier can be resolved across model families without the reviewer juggling four provider consoles. That is a practical reduction in the number of places a reviewer has to look.

Severity, owner and the resolution code

Routing is the difference between a queue and a backlog. Two fields carry it: severity and owner. Severity should derive from the gate, not from the agent's mood, and it should encode reversibility. Handoff criteria guidance rates actions by risk based on reversibility and routes consequential actions through fixed approval gates that sentiment cannot override per Userflow's escalation criteria. A failed verification on an irreversible action (a payment instruction, a published page, a deleted record) is high severity regardless of how confident the agent felt.

The resolution code is the field teams forget and later wish they had. Require an enum, and require it at close:

Resolution codeMeaningFeeds back into
fixed_inputInput data was wrong or staleUpstream data quality, snapshot refresh cadence
gate_too_strictOutput was acceptable; threshold wrongGate tuning
gate_too_looseDefect passed a later check; gate missed itGate coverage
agent_errorTool misuse or wrong argumentsPrompt, schema, tool descriptions
env_driftEnvironment differed from expectationConfiguration management
accepted_riskReviewer knowingly shipped itPolicy review

Without these codes, you cannot answer the only question that matters for gate maintenance: is the gate producing mostly true positives or mostly noise? A queue where 80 percent of tickets close as gate_too_strict is a gate that is burning reviewer hours and training the team to ignore it. That is the same failure mode as an escalation policy that fires too early; Zendesk's troubleshooting guidance names overly permissive fallback logic and knowledge content too broad to trigger a clean handoff as the usual culprits when an agent escalates when it should not per Zendesk's AI agent troubleshooting article.

One more routing rule: the ticket should pause autonomous retries on that task until a human accepts ownership. Idempotency guidance is explicit that a timeout means the result is unknown until checked, not necessarily that nothing happened, and that retries must not create a second refund, replacement or ticket per Salted. Agents that retry a failed verification and resubmit often create two tickets for one intent, and the reviewer spends the first five minutes deciding whether they are looking at a duplicate.

Worked example: one ticket, one pass

Suppose an agent drafts a research summary and the citations_resolvable gate trips. Compare the two tickets a reviewer might receive.

The sparse ticket says: run 8891, gate citations_resolvable, failed. The reviewer opens the run, cannot find the output because the trace was sampled, asks the on-call engineer which knowledge snapshot was live, discovers it was two releases behind, and eventually decides the gate was right but for the wrong reason. Elapsed time: 25 minutes, one Slack thread, no durable record.

The complete ticket carries the three failing citation offsets, the 404 status on each fetch step, the knowledge snapshot identifier, the input digest and the one-line reproduce command. The reviewer runs the command with the offline cache, sees the same three 404s, and confirms the environment fingerprint shows a snapshot from before the vendor retired those pages. The agent's proposed decision was reject with would_accept_if naming a later snapshot. The reviewer agrees, closes with fixed_input, and files a note to refresh the snapshot cadence. Elapsed time: 4 minutes, and the resolution code now counts toward a data quality trend line.

Notice what changed: not the model, not the gate logic, not the reviewer's skill. The only change was that the failure carried its own context. That is the entire argument for treating the escalation note as a schema with a contract rather than as a log line with a link.

The same discipline applies when the reviewer has to judge a number rather than a citation. Verification guidance for AI-produced calculations insists on checking the inputs first, then whether the formula matches the quantity intended, then the operations, units and rounding, because reproducing a calculation from the same mistaken input only reproduces the error more neatly per Manuel Garcia's calculation verification guide. An input digest plus an output excerpt is precisely what lets a reviewer run that sequence without asking the agent what it did.

Making the gate produce good tickets

Three implementation notes, in order of payoff.

Populate the schema at failure time, in the exception path, not later. The agent holds the input, the arguments, the environment and the threshold at the moment the gate trips. Anything written a minute later is reconstruction, and reconstruction is where fields go missing.

Make missing fields loud. If tool_trace is empty, the ticket should say tool_trace: [] (agent made no tool calls) rather than omitting the key. The distinction between "no calls" and "trace dropped" is the difference between a fast review and a wrong one.

Instrument the tickets themselves. Track time-to-first-touch, time-to-close, and the distribution of resolution codes by gate. Gates with a high gate_too_strict rate get tuned; gates with a high agent_error rate get better tool schemas or argument validation; gates that never fire get audited for whether they are actually running. This is the same logic as reviewing contained and escalated conversations together, so that a threshold change fixes one problem without creating another per Userflow.

If you are standing this up across several agent workflows at once, and the reviewer's job is to decide rather than to hunt, a free 30-minute discovery call is a reasonable place to map your gates and handoffs to a ticket contract before you build one per team.

FAQ

FAQ

What is the single most important field in a failed-verify escalation note?

The exact check and threshold that tripped, because it is the only field that tells the reviewer whether the output is wrong or the gate is wrong. A ticket without it forces the reviewer to re-derive the rule from the run, which is the rework the note exists to prevent. Everything else in the schema supports that judgment.

Can I just link to the trace store instead of copying fields into the ticket?

You can, but retention, sampling and access controls differ between trace stores and queues, so the link will eventually break or be unreachable for the reviewer. Handoff guidance recommends a short structured note backed by the original record, with a stable reference rather than inline copies of sensitive data per Salted. Copy the small fields, reference the large ones, and never rely on a link alone.

How do I keep the agent's proposed decision from biasing the reviewer?

Separate it from the evidence and label it explicitly as a proposal, with a would_accept_if field that names the evidence that would flip it. The reviewer should read the excerpt and trace before reading the recommendation, and the resolution code should record agreement or disagreement. That structure turns the proposal into a checkable claim rather than an anchor.

What should happen to autonomous retries while a ticket is open?

Pause consequential actions for that task until a human accepts ownership, because a retry can duplicate an action whose outcome was never confirmed per Salted. Retries are fine for read-only steps. For anything that changes state, treat an unknown result as unknown until someone checks.

How do I know whether my gate is worth the reviewer hours it costs?

Track resolution codes by gate. A gate that mostly closes as gate_too_strict is costing review time without catching defects, while one that mostly closes as agent_error is pointing at a real tool or schema problem. Review that distribution on a schedule, the same way you would review escalation thresholds across contained and escalated cases together per Userflow.

Sources

Sources

  1. Why AI Agents Fail in Production, and the Observability ...
  2. Fact-checking AI with Lateral Reading - Research Guides
  3. AI-to-human handoff: escalation rules, context and metrics
  4. Retries Without Duplicate Actions: Idempotency for AI Agents
  5. Before the Tool Call: Deterministic Pre-Action ...
  6. AI Agent Human Escalation Criteria
  7. Troubleshooting common issues with AI agents
  8. How to Verify a Calculation Produced by AI - Manuel B. Garcia
About the author

About the author

Antonio Evans
Antonio Evans

Antonio Evans is the founder of Quanta Intelligence and a fractional CTO who builds HIPAA-ready AI systems for health and life-science companies. Before that he was technical co-founder of FriendsEAT for twenty years, CEO of Socialdraft, and a data engineer at IBM. Antonio lives with his dog Santino in Lisbon, Portugal.

All articles