When a Tool Call Is Safe to Retry: A Decision Table by Effect

Retry only when the call is idempotent, or when you can record and run a compensation for the first attempt before the second goes out. Everything else needs a human approval gate and a reconciliation check against the external system.

That single rule sorts every write into three effect classes, and each class gets a different retry policy. The rest of this article is the table, then the mechanics behind each row.

Two of these rows are commonly confused. A PUT is idempotent by design, while a POST that creates a resource is not, and repeating the same POST creates a second object, as the HTTP methods primer explains. Making a POST /refund sit behind a retry loop does not make it idempotent. You have to change the operation or add a key, which is why the class depends on the effect, not the HTTP verb.

Idempotent means calling the operation multiple times produces the same result as calling it once. A GET never changes state, so reading a record ten times is safe, and an upsert of a user row keyed by user_id returns the same success result whether the row was created or already existed, as the upsert example in the agent idempotency design guide shows.

The trap is that truly idempotent writes are rarer than they look. "Add a note to the order" is not idempotent: two calls append two notes. "Set the order status to shipped" is. Before you grant a tool the automatic-retry policy, ask one question: if this call lands twice, does the stored state end up the same? If the answer is no, the tool does not belong in this class, regardless of how it is named.

For the idempotent class, automatic retry with backoff and a cap is the right default. Bound it: three or four attempts, a base delay that doubles, and a hard stop that surfaces an error rather than looping. The cap matters because an unbounded retry on an idempotent call is still a way to hammer a struggling downstream service.

The strongest version of this class uses a key the receiver honors. Stripe's approach stores the key alongside the operation result, so a retry with the same key returns the cached result instead of charging again, as documented in the Stripe idempotent requests reference. That pattern is what lets an agent framework retry as aggressively as it likes without the economic result changing.

A compensable write is one you can reverse with a recorded undo step: release the inventory reservation, cancel the hold, close the ticket you opened, void the pending authorization. This is the saga pattern, and the property that makes it work is that the undo is written before the first effect is attempted, not reconstructed from memory afterward.

Here is the worked sequence for an order that reserves inventory, charges the customer, then sends a confirmation:

1. Reserve inventory under reserve:order_551: and write a durable record {kind: reserve, key: reserve:order_551, undo: release_endpoint, status: pending} before the call goes out.
2. Call the reservation service. If the response is lost, do not retry the reservation. Instead, call the release endpoint with the same key. Releasing a reservation that never existed returns success, so the release is safe even when the reserve did not land.
3. Mark the operation record compensated, then retry the reservation with a new operation key.
4. Only after the reservation is confirmed do you charge. The charge is irreversible, so it falls under class 3 and gets no automatic retry.

The rule for this class is precise: retry only after compensation for attempt one has run and committed. If the reserve succeeded and the release fails, you are in an unknown state, and the operation moves to a reconciliation queue rather than a retry loop. The saga write-up on retry semantics when agents touch the real world is worth reading in full for how partial multi-step workflows are unwound.

A note on ordering. If the reservation succeeded and the confirmation failed, the customer has been charged without a confirmation, and retrying the whole sequence runs the charge again. If you implemented idempotency at the charge layer, the repeat is deduplicated, but the inventory was already reserved and a second reservation attempt may fail, as the same source notes. The practical fix is to make each step separately resumable with its own operation key, so a restart resumes at the failed step rather than replaying the sequence.

Some writes cannot be undone: money leaves an account, goods ship, an email lands in an inbox, data is deleted without a backup. For these, no automatic retry is acceptable. The policy is sleep, reconcile against the external system, then escalate to a human with the evidence.

The reconciliation step is the one people skip. It is a query to the external system that answers a concrete question: did this operation already happen? For a payment provider, that is a lookup by your idempotency key or a list query filtered by your reference. For a shipping system, it is a check on the label or tracking number. If you cannot ask that question of the external system, you cannot safely retry the call, full stop. A local database row alone cannot make an email, charge, or deployment exactly once, as the idempotent tool calls guide states plainly.

The approval gate should be concrete, not a vague sign-off. Before attempt two on an irreversible call, the approver should see: the operation key, the original request, the evidence of what the external system shows now, and the action proposed. If the evidence shows the operation completed, the correct action is to record success and skip the retry, not to approve a retry.

A useful boundary case: an operation whose result is unknown because it crossed an external boundary but never finished local bookkeeping. The right state is needs_reconciliation, not failed. Collapsing "unknown whether it happened" into "failed" is how duplicate effects are born, as the idempotent tool calls guide puts it. Your state machine needs that third state, or the approval gate will be asked the wrong question.

It is worth separating the two failures. The first failure is the lost response: the refund committed and the agent did not hear back. Nothing in the tool can prevent that. The second failure is the retry policy that did not ask whether the first attempt had already worked. That one is entirely within your control, and it is the one that turns a lost response into a doubled refund.

The durable-execution crowd has a useful illustration here. Killing a worker after a tool's side effect but before the runtime journaled the result made a mock payment API receive two charges in three of three trials, as the Restate crash-testing write-up reports. Killing before the effect, or after the result was journaled, gave one charge every time. Passing a key derived from the step identifier made the second request a no-op in three of three trials, as long as the receiver honored the key. The lesson is that durability protects completed steps; it cannot protect a step whose effect happened but whose result was never recorded.

That window is the whole argument for the decision table. If the tool is idempotent, the window does not matter. If it is compensable, the window is covered by a recorded undo. If it is irreversible, the window is covered only by reconciliation and a human.

The mechanics are small. Below is the minimum record and the guard that enforces the table, in pseudocode you can translate to your stack. The claim must be an atomic storage operation, typically a unique constraint or INSERT ... ON CONFLICT, and the reconcile step must query the real external system rather than infer success from an exception.

operation_record = {
  operation_key    # stable across retries of one intended action
  payload_hash     # rejects the same key with changed arguments
  effect_class     # idempotent | compensable | irreversible
  status           # in_progress | completed | compensated | needs_reconciliation
  lease_until      # guards concurrent attempts and abandoned workers
  result_json      # the stored response so duplicates get the original answer
  downstream_id    # gives reconciliation something concrete to query
  undo_ref         # required for compensable, e.g. release endpoint and key
}

invoke_tool(operation_key, args):
  hash = sha256(canonical_json(args))
  claim = claim_operation(operation_key, hash, effect_class)
  if claim is payload_conflict: reject "key reused with different arguments"
  if claim is completed: return claim.result
  if claim is in_progress: return RetryLater
  if claim is needs_reconciliation: return escalate_to_human(claim)

  switch claim.effect_class:
    case idempotent:
      return execute_with_backoff(operation_key, args)
    case compensable:
      run_undo_if_recorded(operation_key)   # release, cancel, close
      return execute_with_backoff(operation_key, args)
    case irreversible:
      return escalate_to_human(claim)        # no automatic execution on retry

The model should never generate the operation key. The deterministic workflow layer creates it before asking the model to call a tool, then injects it into the tool context. The model supplies business arguments; the runtime supplies identity and attempt metadata, which is how the idempotent tool calls guide frames the contract. A run ID, trace ID, or tool-call ID usually changes when work is retried, so keep those for observability but never use them to decide whether the business effect already happened.

When you receive an event rather than send a call, the same discipline applies to the inbound side. Webhook providers operate on an "at least once" model, so you will get the same event more than once, as the Hookdeck idempotency walkthrough explains. The durable fix is a table with a unique constraint on the event ID, written before any business logic, with a PENDING to DONE status, and a 200 only after the row is committed, as the Asaas webhook idempotency guide documents. If you are building the sender side, the Standard Webhooks specification is the common contract for signatures, timestamps, and retries, and it exists precisely because every provider had been reinventing this.

One more guard belongs at the edge: SSRF, spoofing, and replay protection for any webhook endpoint, which the Standard Webhooks project calls out as core requirements rather than nice-to-haves. A replay of a signed webhook is a duplicate effect with a valid signature, so idempotency and replay protection are two layers of the same defense.