When an Agent Stops and Escalates: A Decision Table

Four rows, four signals, four payloads. The threshold values are starting points; calibrate them against your own traces. What matters is that each row fires deterministically and that the human sees the same fields every time.

Treat this as a checklist you can implement row by row. A common mistake is to build only the low-confidence row because it is the most visible, then discover that the agent never gets to a low-confidence state before it has written something.

The signal is not "the agent is retrying." Retrying a transient 429 or a timeout is correct behavior. The signal is that the agent is repeating the same call with the same inputs and getting the same result, which is wasted work at best and a sign of a reasoning loop at worst.

The demo in this write-up on preventing reasoning loops makes the mechanism concrete: an ambiguous tool that said "more results may be available" drove 14 repeated calls, while a tool returning a clear terminal state stopped in 2. The same source cites a community observation of an agent that ran 847 reasoning steps at a reported $47 per minute without ever delivering an answer. Whether or not you trust the exact figures, the shape is familiar.

The implementation is a sliding window over a fingerprint of tool name plus normalized arguments. Normalizing matters: an agent that reformats whitespace or reorders JSON keys is making the same call.

from collections import deque

class FingerprintWindow:
    def __init__(self, size=5, limit=2):
        self.recent = deque(maxlen=size)
        self.limit = limit

    def check(self, tool_name, arguments):
        key = (tool_name, canonical_json(arguments))  # sort keys, strip whitespace
        repeats = self.recent.count(key)
        self.recent.append(key)
        if repeats >= self.limit:      # the third identical attempt
            return "repeat_failure"
        return None

Two design rules make this row useful. First, use the error's own structure rather than scraping the message string. Speakeasy's guide to API errors argues errors should carry both a human-readable message and an application-specific code precisely so software can decide what to do next, and shows Stripe-style codes like trip_too_short that can be branch conditions. If your tools return stable codes, the fingerprint includes the code, and the escalation knows whether this is "file not found" (fixable) or "permission denied" (needs a human to grant access).

Second, verify before you retry. Research on non-atomic tool failures found that agents typically assume tool calls are atomic, so a timeout after dispatch looks like a failure when the operation actually succeeded, and naive retries duplicate actions. A verify-before-retry wrapper that checks postcondition state before trying again reduced duplicate actions in the study. For your escalation, that means the payload should say whether the agent already confirmed the side effect happened.

What the human gets: the tool name, the exact arguments, every error code and message, and the first-failure timestamp. Add the model's stated reason for the retry if your traces capture it, because "I retried because the response said more results may be available" tells the reviewer that the tool contract, not the model, needs fixing.

Distinguish this from prompt-failure escalation, where an agent that hits a refusal starts probing for other paths and broader permissions. That is a security event, not a retry. Policy should make stopping a valid outcome and alert when an agent widens its scope after a block.

Cost is the trigger most teams add last and regret first. The signal should fire before the budget is gone, so the human has room to make a decision that costs something.

A three-tier design works. At 50 percent of budget, log a warning with per-step attribution. At 80 percent, escalate with the breakdown and pause if the next planned step is expensive. At 100 percent, hard stop regardless of what the model wants. The 80 percent figure is a starting point you should justify from your own distribution of run costs: if your median run uses 40 percent of budget, an 80 percent trigger will rarely fire and is safe; if your median is 75 percent, you are effectively escalating every run and should raise the budget instead.

Attribution is the hard part and the part that makes the escalation actionable. Per-step cost only becomes meaningful when every model call and tool call is tagged with the step that caused it, which is exactly what distributed tracing gives you. OpenTelemetry-style spans per agent step let you query "which step spent the money" instead of reconstructing it from logs after the fact.

What the human gets: a table like this, not a single number.

Now the reviewer can see that step 7 ate two thirds of the budget while re-parsing an ambiguous document, and the right decision is obvious: fix the input, not the budget. Without per-step attribution the reviewer sees "cost exceeded 80 percent" and extends the budget, which teaches the agent nothing.

The escalation type here is an approval gate with a budget decision attached. Promptessor's HITL guide describes the useful decision set as approve, modify, reject and escalate, and argues approve/reject alone is too narrow. A cost overrun is the clearest case where "modify" (run a cheaper plan) beats a binary yes/no.

Confidence is the trigger everyone builds and the one most likely to be theater. A model that says "I am 0.9 confident" is producing a token, not a measurement. Agnost warns that agents are very good at sounding certain, and recommends tracking actual uncertainty signals: retrieval results that conflict, tool calls returning incomplete data, a user correcting the agent's assumption.

Useful confidence signals come from outside the model's self-report. A reviewer model scoring the draft against a rubric is one, as described in the escalation-path design where a post below the passing score stops before publishing. Retrieval agreement is another: if the top three retrieved passages disagree on the answer, report that as low confidence with the disagreement attached. Schema validation of tool arguments is a third, cheap and deterministic.

The threshold should depend on the action class, not be one global number. A 0.7 floor for a draft email is reasonable. For anything that changes state in a system of record, confidence should not be the gate at all, because a confidently wrong write is still a wrong write. That is why the fourth row exists separately.

What the human gets: the score, the method that produced it, and the competing candidates. If the agent had two plausible plans and picked one at 0.62, show both plans with the evidence for each. A score alone tells the reviewer to be nervous; two candidates let them pick. Add the specific missing evidence if the agent can name it ("the vendor's refund policy page returned 404, so I inferred the terms"), because that turns the escalation into a task the human can actually complete.

One more decision rule: clarify before you escalate when a single question would remove the uncertainty. Agnost draws the line at whether the uncertainty comes from missing authority, missing data or repeated failure. Two plausible readings of the request means ask one question. Missing invoice access means escalate, because no amount of clarification gives the agent the data.

Any non-read operation against a system of record deserves a human eye the first time it happens, and a policy decision after that. This row is where the failure cap is most obviously insufficient: the agent can succeed on every attempt and still be wrong.

The distinction to build is pre-action approval versus post-action review. Promptessor makes this explicit and notes that OpenAI's agent guidance separates automatic guardrails from human review, where review pauses a run before sensitive side effects such as cancellations, edits, shell commands or sensitive MCP actions. Consequential writes belong on the pre-action side. Post-action review is for lower-stakes changes you want sampled for quality.

The mechanics that make this work, drawing on the same guide: the run pauses rather than ending, the pending action is persisted so a slow reviewer does not lose the run, state is revalidated when the human responds because the world may have changed during the pause, and the approval is checked for staleness. Research on verified tool calls supports the same instinct from the reliability side: idempotency keys plus postcondition checks stop duplicate writes when the agent is unsure whether a call landed.

What the human gets: the change, not the narrative.

action: update_record
target: accounts/8842/billing_contact
before:
  email: [email protected]
  phone: "+1-555-0100"
after:
  email: [email protected]
  phone: "+1-555-0100"
reversible: true
reversal: restore previous record from snapshot snap_2291
dry_run: passed, no constraint violations
requested_by: step 12, "sync contact from the signed contract"

Three details earn their space. Reversibility tells the reviewer how much deliberation the decision deserves. The dry-run result catches the failures that a diff hides, such as a foreign key that no longer exists. The requesting step anchors the change to a reason, so the reviewer can judge whether the reason is good rather than only whether the diff is syntactically fine.

For shell commands and code changes, dry runs are harder and the payload should include the exact command or patch, the working directory, and any write paths it touches. If the agent cannot produce a reversible plan, that is itself the signal to escalate rather than proceed.

Every row above ends with the same question: what does the human need to decide in under a minute? That is a documentation problem as much as an engineering one, and the same discipline that makes error messages useful makes escalation payloads useful. Postman's error-handling guidance asks for a clear and consistent structure, descriptive messages, no leaked sensitive data, and requests that carry an identifier and timestamp so interactions can be traced. Speakeasy wants errors detailed enough to help a human understand the problem and structured enough to help a machine decide what to do next. Your escalation record is exactly that artifact.

Two habits pay off. First, treat the escalation schema as code and review it like an API contract. Write the Docs describes writing documentation with the same tools as code, version control, pull requests and reviews; the same approach applied to the payload schema means a new field gets reviewed before it reaches reviewers in production. Second, keep the taxonomy small. GOV.UK's technical writing team describes writing, reviewing, testing, merging, deploying and iterating in the open, which is what you want for the escalation playbook: each incident that exposed a missing field becomes a pull request, not a wiki edit nobody reads.

If you are routing agent traffic across several model providers, keep the escalation layer provider-neutral. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API key, which means your cost attribution and confidence scoring do not have to be rewritten per vendor.