A Retry and Backoff Contract Between an Agent and Its Tools

A small, stable error body is enough. Four fields, all required when the response is an error:

{
  "code": "PAYMENT_LOCK_TIMEOUT",
  "retryable": true,
  "retry_after_ms": 1200,
  "stop_reason": null
}

Three rules make the contract work in practice.

First, code is a closed set. Treat it like an enum in the tool schema, so a new failure class requires a schema change and review. Free text codes drift, and a metric built on them becomes useless within a quarter.

Second, retryable is the tool's opinion, not the agent's inference. A rate limit response is retryable. A 400 from a schema validator is not. The tool maps its internal exceptions onto this boolean once, in one place, instead of asking every prompt to remember which status codes are safe.

Third, when stop_reason is set, retryable should be false. The two fields are not independent knobs. A permanent failure that needs a human is exactly the case where the agent must stop, and the stop reason tells the harness what to put in front of the operator: AUTH_EXPIRED, QUOTA_EXHAUSTED, SCHEMA_MISMATCH, DOWNSTREAM_REJECTED.

The response is the contract. Anything else in the body, including a human readable message, is for logs and dashboards, not for the retry decision.

The agent's retry loop is small once the contract is in place. It looks like this:

key = new_uuid()                    # generated once, before the first call
attempt = 0
budget = 4

while attempt < budget:
    resp = call_tool(args, idempotency_key=key)
    if resp.ok: return resp
    if resp.stop_reason: escalate(resp)
    if not resp.retryable: fail(resp)

    base = resp.retry_after_ms or 0
    jitter = random() * base * 0.2
    sleep(max(base + jitter, 0))
    attempt += 1

escalate(BUDGET_EXHAUSTED)

The idempotency key is generated before the first attempt and never regenerated, which is the same rule payment APIs use. The server stores the key alongside its result, so a retry with the same key returns the original outcome instead of executing the side effect twice, as the Stripe idempotent requests documentation describes for its own API. A detailed walkthrough of idempotency keys in REST APIs notes that keys are typically stored with their response for 24 hours to 7 days, which sets a hard ceiling on how long an agent can safely retry a given operation.

There is one subtle trap with the same key. Some servers reject a repeated key if the request body differs, because the key is meant to identify one intent. If the agent regenerates its arguments between attempts, for a slightly different date or amount, the second call may come back with a conflict rather than a deduplicated result. Keep the arguments byte identical across attempts of the same key, or generate a new key and accept that the operation may run twice.

The second trap is the wait itself. retry_after_ms is a floor, not a target. Always add jitter, and always sleep at least the stated value. Amazon's writeup on timeouts, retries and backoff with jitter makes the operator's side of this point: retries without jitter amplify a small outage into a full one, because every client that failed at once retries at once. The tool's returned wait replaces the agent's own backoff schedule, so the agent does not need to know whether the base is 200 ms or 2000 ms; the tool tells it, per failure.

A worked example with numbers. Suppose an agent calls a payment tool that sits behind a shared connection pool. The first attempt returns {code: "POOL_SATURATED", retryable: true, retry_after_ms: 1200, stop_reason: null}. The agent sleeps 1200 ms plus up to 240 ms of jitter, then retries with the same key. The second attempt returns retry_after_ms: 2400, so it sleeps 2400 to 2880 ms. The third attempt succeeds. Total added latency is roughly 4 to 4.6 seconds, well inside a 30 second tool timeout, and the server saw three requests instead of the six an immediate-retry loop would have generated in the same window.

Most retry bugs are not failed retries. They are retries that never end. A tool that keeps returning retryable errors inside its time budget becomes an infinite loop with extra steps, and the agent has no reason to stop because every individual response said "try again."

The attempt budget is the primary stop condition, and it belongs in the harness, not the prompt. Set it per tool class, not per agent. A read-only lookup can tolerate a larger budget than a payment authorization. The tool's own retry_after_ms should be treated as a hint that reduces remaining budget faster when it is large: if the earliest safe wait is longer than the remaining time before the agent must return, the correct action is to stop now rather than wait and fail later.

The stop_reason handles the other case, where waiting is pointless no matter how many attempts remain. Four values cover most tools:

The pattern to avoid is stuffing this information into code alone. A code tells you what happened. A stop reason tells the agent and the operator what to do about it. Keeping them separate means a dashboard can count failures by cause while the runtime follows a small, stable policy.

Cockroach Labs' walkthrough of what write skew looks like is a useful parallel for the operator experience. The interesting question in a concurrency bug is not "what state did we land in" but "which interleaving produced it." Retry metadata plays the same role: the agent's trace should show the key, the attempt number, the code, the wait and the stop reason, so a failed payment can be reconstructed after the fact.

The key must be derived from durable state, or it must be persisted before the first attempt. Generating a fresh UUID at the top of a retry loop is the single most common bug in agent retry code, because it converts every retry into a new operation. A collection of patterns for making agent retries idempotent puts this first on the list for exactly that reason.

For single tool calls, a UUID generated once before the first attempt and written into the step's durable record is enough. For multi step workflows, derive the key from durable identifiers so that a resumed run regenerates the same key. The approach described in the saga and deduplication analysis uses {workflowRunId}:{stepId} and argues the key should be deterministic rather than random, so that a crash and resume produce the same key without needing to have stored it.

The server side has to cooperate. An implementation guide for idempotent REST APIs in ASP.NET Core describes the standard shape: the client sends a unique key in a header, the server checks whether it has seen that key before, and if so returns the stored result without reprocessing. The same source quotes RFC 9110's reminder that a server is free to log each request separately, so a deduplicated retry can still show up in your access logs. That is fine. What must not happen is a second charge or a second ticket.

There is a real cost to be honest about. Every tool that accepts an idempotency key needs storage for the key and its result, plus an expiry policy. The DZone piece on idempotency frames the problem through the phantom timeout: the request arrived, the side effect happened, and the response never came back. The agent genuinely cannot tell whether it needs to retry. A stored result turns that ambiguity into a cheap lookup.

For a tool that writes to a database, the deduplication can ride on a unique constraint rather than a separate store. Insert the idempotency key into a column with a unique index, and a duplicate insert fails fast with a code the tool maps to retryable: false and the original result. That is less code than a cache and it survives process restarts for free.

Agents often call tools that are thin wrappers over a database, and the wait semantics change at that layer. A connection pool is the clearest case. PgBouncer queues client connection requests once a pool reaches its configured size, and only aborts a connection when the client itself exceeds its own limit, as the ScaleGrid overview of PgBouncer explains. That means a saturated pool produces queueing, not an error, and the right retry_after_ms depends on how deep the queue is.

Two things follow for the contract. First, a tool that wraps a database should set retryable: true and a conservative retry_after_ms on pool saturation, and it should not treat a queued connection as a hard failure. Second, the tool's retry semantics depend on the pool mode. Heroku's PgBouncer configuration guide recommends transaction pooling for most applications, and the PgBouncer feature matrix shows why: transaction pooling breaks session state such as SET, LISTEN and WITH HOLD cursors, so a tool that relied on session state in the first attempt may fail on a retry that lands on a different server connection. The failure will look transient to the agent. It is not. Map it to retryable: false with a stop reason, and fix the tool to avoid session state.

The same logic applies to transactions inside the tool. If the tool's unit of work is a transaction, a serialization failure is retryable and the tool should say so with a short wait. A constraint violation is not. Both can surface as the same driver exception class, which is exactly why the mapping has to live in the tool and not in the model's guesswork.

For observability, the query plan is the equivalent of the retry trace. Datadog's guide to EXPLAIN ANALYZE makes the point that actual row counts and actual time per node are what reveal why a query is slow, rather than the plan's estimates. A retry log is the same instrument: per attempt, record the wait actually taken, the code returned, and whether the call reached the server. An agent that waited 1200 ms and still lost is a different bug from an agent that never waited at all.

A contract is only as good as its tests. Five test cases cover most of the risk, and each one should run against a real instance of the tool, not a mock that returns the body you wish it would return.

The duplicate delivery test is the one teams skip, and it is the one that catches a server that accepts the key but never checks it. Replay the same request with the same key and assert that the database has one row, not two. A worked example in the agentic saga article notes that a payment call can return 200 before the confirmation is stored, so the test should assert the final state, not just the status code.

Test with real failures rather than simulated ones. A test harness that returns arbitrary status codes validates the agent's loop, but it does not validate the tool's mapping, so pair it with at least one test against the actual tool under a real, forced failure.

Finally, assert on timing, not just on the final result. A test that passes because the agent eventually succeeded will hide an agent that retried in a tight loop for two seconds before the service happened to recover. Capture the timestamps of each attempt and assert that the gap is at least the value the tool returned.

If your agent calls several kinds of tool (text, image, audio, embeddings), running the same tests across providers is the tedious part. Routing them through Quanta Inference gives one OpenAI-compatible surface and one key, which keeps the test harness small and the retry policy uniform across model types.