When to Reject a Tool Call Versus Repair It

Reject the call when the failure text itself tells the model what to change. This is the highest-value path because one round trip usually fixes it, and the fix is verifiable.

The canonical examples are structural. A required argument is missing. A value is outside an enum. JSON does not parse. A number was sent where a string was expected. These are exactly the failures that the schema-gate walkthrough from Micheal Lanham describes as "tool argument rot": the model sent {"customer_id": 12345} when the tool wanted a string id plus include_history. The reasoning was fine; the argument shape was not.

The reason rejection works better than guessing is that the model can read the schema. When you return the validator's message, include the failing field path, the expected type and the allowed values, and the model gets a target it can hit on the next try. The TrueFoundry analysis of structured outputs draws the necessary boundary: JSON mode can make syntax predictable, a schema constrains shape, but application validation still has to enforce types and local invariants after generation. Rejection is where that validation speaks back to the model.

Two rules keep the reject path honest. First, never reject the same call twice with the same message. If the model returns the identical invalid argument after reading your first error, the error is not actually actionable, and repeating it just burns a turn. Second, cap the number of rejections per task (two is a common choice) before escalating to a failure.

There is one form of error you should not feed back: anything that embeds secrets or reflected attacker text. Tool error bodies can contain stack traces, internal hostnames or quoted user content. Sanitize before returning, because the model treats everything in the tool result as context. This connects to the broader point in Simon Willison's lethal trifecta writeup: models follow instructions found in content, so an error message is content.

Retry with a corrected prompt when the failure is ambiguous and the tool call cannot cause compounding damage. Ambiguous means the error does not name a fix: a 403 with no body, an empty result set, a response the parser cannot read, a business rule that returned "not allowed" without stating why.

Safe means the call is read-only or idempotent. The Trussed technical guide on tool idempotency is precise about the trap here: a tool call may use a technically idempotent HTTP verb while wrapping a non-idempotent business operation such as charging a customer or sending a notification. RFC 9110 makes GET, HEAD, PUT and DELETE idempotent and explicitly does not extend that guarantee to POST, and a client cannot assume a non-idempotent request was fully processed if no response arrived. So classify by the business effect, not by the verb.

When both conditions hold, a corrected retry is worth its cost because you gain information. A 403 usually means the credential scope is wrong or the resource belongs to another tenant, so the useful correction is to call a discovery endpoint first, or to switch to a mirror tool that only reads metadata. A zero-row result usually means the filter was too narrow, so the correction is to widen it.

What you should not do is resend the same parameters with newer wording. That is not a corrected prompt; it is a hopeful retry, and the cost model below shows how quickly those add up.

A worked example makes the boundary concrete. Suppose an invoice agent calls lookup_customer (read-only) with an email that returns zero rows. Retry is fine: relax to a domain search, cap at one attempt. Suppose the same agent calls charge_card and receives a timeout. The charge may have succeeded. Retrying is exactly the duplicate-charge failure described in the idempotency patterns write-up, where the gap between "the tool ran" and "I heard back" produces order #124 next to order #123. The correct action is neither reject nor retry. It is fail and reconcile.

Fail the task when repair cannot be trusted, and fail it loudly with enough state for a human to resume.

Two conditions trigger this. The first is a non-reversible side effect already in flight. If the call may have written, paid, sent or deleted, and you do not have a confirmed outcome, a repair attempt converts an uncertain state into a known-bad one. The agent stack should move to a reconciliation step keyed by an idempotency key, not to another model turn. The Frontier Model Forum issue brief frames this as the compounding-error risk of multi-step workflows: errors accumulate across steps, and the architecture has to contain them rather than let the agent improvise a recovery.

The second condition is budget exhaustion. Set the budget explicitly, in whatever unit your team can defend:

• attempts per task (commonly two)
• added wall-clock latency (commonly 30 seconds)
• added tokens per session or per user, on a rolling window

The token budget is the one most teams skip and most regret. The retry-budget analysis works the arithmetic: a 20 percent per-step retry rate sounds tolerable, but under context replay a three-step agent lands near 1.7 to 1.9 times baseline tokens, and a five-step agent near 2.2 to 2.5 times. It also cites a single Claude Code session that consumed 1.67 billion tokens in five hours, with 253 "usage limit" errors that did not stop the loop. That is a budget that existed on paper and was not enforced inside the loop.

The decision rule from Google's SRE work, adapted there, is to denominate the budget in retry tokens per user per rolling window rather than retry attempts, and to check it inside the agent loop where the replay actually happens. An HTTP-layer retry policy cannot see a five-step context replay.

It is worth doing the numbers once so the trade-off stops being abstract. Take a three-step agent with a 4,000-token system prompt and tool schemas, where each step adds about 2,000 tokens of history.

Without retries: three model calls, contexts of roughly 4k, 6k and 8k tokens, about 18k tokens total.

With one retry at step three: the failed attempt replays 8k tokens, then the corrected attempt replays 10k (the error text is now in history). Total is about 36k, so roughly double. That single repair doubled inference cost for the task, before counting the extra latency of two more round trips.

Now compare the three actions against that cost.

The insight the table makes obvious: rejecting is almost always cheaper than a corrected retry, because rejection is a single replay and the model does the diagnosis. If you find your agent preferring the middle row, the real defect is usually in the error text you return, not in the model.

A few engineering choices make the decision rule enforceable rather than aspirational.

Classify before you call. Every tool registration should declare read_only, idempotent, destructive and external flags. The decision function reads those flags; the model never decides whether its own retry is safe. The Okta least-privilege guide makes the identity version of this argument: agent access should be scoped to the exact current need, and static broad grants are what let a single bad call become a bad action. Okta cites Gartner survey work finding that 74 percent of IT application leaders see AI agents as a new attack vector.

Attach idempotency keys at intent time. Generate the key when the user's intent is formed, not when the call is made, so a regenerated call during replanning maps back to the same key. The idempotency guide from Trussed notes that LLM agents rarely produce byte-identical repeated requests, so textual deduplication fails; you need a fingerprint or key.

Run discovery before action. For 403s and unknown-resource errors, a read-only discovery call is a cheap retry that gathers information instead of guessing. It also keeps you out of the empty-result ambiguity that makes logic-error recovery expensive.

Log the decision, not just the error. Store the failure class, the action taken, the attempts used and the tokens spent on repair. Without this you cannot tell whether your rejections are working, and you cannot defend a latency or spend increase when someone asks.

Apply least privilege to data, not just tools. An agent that can only see rows it needs returns more informative failures and leaks less when a call goes wrong. Snowflake's row-level security overview explains that policies applied at query time hold regardless of which application tier makes the request, which is the property you want when an agent is the caller. Tinybird's case for row-level access control with LLMs makes the complementary point: enforce security at the data layer, where access is deterministic, because prompt engineering is not a security control. And Oso's trifecta analysis is right that splitting into multiple agents does not dissolve the problem; permissions still have to constrain what any of them can reach.

Consider whether the error is worth fixing at all. One class of failure looks ambiguous but is not: a hallucinated tool name. A benchmark cited in the retry-budget analysis found that 466 of 513 retry attempts, about 91 percent, were spent on tools that did not exist. Rejecting with the list of valid tool names fixes this once; retrying with paraphrased intent does not.

If you are building this gate across several model providers and tool types, the plumbing is where most of the effort goes. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API with prepaid dollar pricing, which keeps the retry-cost accounting consistent across providers: https://quantaintelligence.ai/inference.