Letting the Model Choose the Key: Why It Usually Cannot

Most tool schemas grow the field because it is convenient. The model is already producing a JSON object, the downstream API wants an Idempotency-Key header, so someone adds it to the parameters and moves on. Stripe's API documents exactly that header for safe retries Stripe's idempotent requests reference, and many teams mirror the upstream requirement into their own tool definitions without thinking about who should own the value.

The pressure to do this comes from the retry loop itself. Agent frameworks encourage plan, call tool, observe, plan again, and when a call fails the model is rewarded for persistence Tool Calling Has a Tax You're Not Seeing. Persistence without a stable identity becomes duplication. The field stays in the schema because removing it feels like removing safety, when in fact the opposite is true.

There is a subtler version of the same mistake: putting the field in the schema, then instructing the model in the system prompt to "copy the key from the previous message." That instruction is prompt text. It is not enforced, it does not survive a restart, and it does not survive the model deciding the previous call did not happen. The schema is where the guarantee either exists or does not.

A useful test: for each field in a tool schema, ask whether the correct value is a function of (tenant, operation, arguments). If it is, the model cannot add information by choosing it, and every token spent generating it is a chance to get it wrong. Fields like amount, customer_id and currency do carry information from the conversation. Fields like idempotency_key, request_id and trace_id do not.

Take a payment tool with arguments {customer_id, amount_cents, currency}. The orchestration layer has a durable step ID for the charge, for example ord_8812:charge:1.

Attempt one: the dispatch layer derives the key sha256("acme|charge|ord_8812:charge:1"), sends the request, and the payment provider accepts the charge but the response is lost in a network timeout. The charge succeeded server side. A timeout after dispatch does not establish whether the side effect committed, which is why retries need a key the provider can deduplicate Formation's analysis of agent tool retries.

Attempt two: 1.8 seconds later, the client retries. Because the key is derived from the persisted step rather than sampled, the same hash goes out. The provider recognizes it and returns the original charge instead of creating a second one. Monday's API shows the same contract in concrete terms: a replayed request returns a header marking it as a replay, and a concurrent duplicate returns 409 Conflict with a Retry-After header rather than processing twice Monday's idempotency documentation.

Now the failure case. If the model had generated the key, attempt two would carry a different string. The provider would see a brand new request with the same amount, no deduplication record and no reason to refuse. A retry storm of three attempts on a $49.00 charge becomes $147.00 of exposure on a single order, and reconciliation happens after the customer has already seen the statement.

The same numbers apply to email sends, ticket creation and refunds. Anywhere the effect is visible outside your database, a drifted key is a duplicate.

That table is worth committing to code as a decision rule rather than leaving it to prompt instructions.

The derivation function should be boring, deterministic and independent of plan position. A workable version:

import hashlib, json

def derive_key(tenant: str, operation: str, step_id: str, args: dict) -> str:
    payload = json.dumps(
        {"tenant": tenant, "op": operation, "step": step_id, "args": args},
        sort_keys=True, separators=(",", ":"),
    )
    return hashlib.sha256(payload.encode()).hexdigest()

Four rules make this safe.

First, sort the arguments so that key ordering does not change the hash. JSON object key order is not a stable input otherwise.

Second, treat step_id as a durable occurrence identity, not a counter that resets on restart. The charge for this order needs one identity; a separately approved second purchase needs another Formation's analysis of agent tool retries.

Third, persist the derived key and the approved arguments before the first dispatch, in the same transaction as the step record where possible. A crash between changing state and recording completion leaves a gap. Monday's documentation makes the related point directly: store the key alongside the request so retries use the same value, and do not generate a new key on every retry attempt Monday's idempotency documentation.

Fourth, handle changed arguments as a separate decision. If the model changes the amount while the original charge is unresolved, do not silently mint a new key and dispatch. Both amounts could otherwise be charged for one order.

One more consideration: client-side deduplication that lives only in the agent's memory does not survive a crash. If the process dies after dispatch, the persisted key is the only thing that lets the recovered process recognize the operation as already in flight Tool Calling Has a Tax You're Not Seeing.

Do the change in the schema first, then in the dispatcher, then in the tests. Removing the field last is how teams end up with a period where neither side owns the key.

1. Remove idempotency_key (and any similar identifier field such as request_id, trace_id or dedupe_token) from every tool schema the model can see. If the model never sees the field, it cannot fill it in badly.
2. Add the key as a required argument of the tool's server-side handler, not of the model-facing schema. The handler signature becomes the contract.
3. Derive the key in the dispatcher from tenant, operation, durable step ID and sorted arguments, using a function like the one above.
4. Persist the key with the step record before dispatch, and read it back on every retry path, including model-requested retries and recovery after an orchestrator restart.
5. Send the key in the header or field the downstream API expects. Stripe expects the Idempotency-Key header Stripe's idempotent requests reference; Monday documents the same header with a 30 minute cache window and a 1 MB cap on cached responses Monday's idempotency documentation.
6. Encode the trigger table above as code so the dispatcher decides whether an attempt is a retry of an existing operation or a new operation.
7. Add a test that runs the same tool call twice through the dispatcher with different random model outputs and asserts one key and one side effect. Add a second test that restarts the orchestrator between attempts and asserts the key is unchanged.

A useful sanity check on step five is whether the downstream service actually stores and checks the key. Some APIs accept the header and ignore it on certain endpoints. Monday, for example, ignores it on queries because reads are inherently idempotent Monday's idempotency documentation.

There is also a schema-level payoff beyond correctness. Removing a field from every tool definition cuts input tokens on every request in a long conversation, and it removes a category of validation error where the model emits a key in the wrong format or an empty string. Teams that have described the "tool calling tax" point at retries, compensation and support load as the hidden costs Tool Calling Has a Tax You're Not Seeing; removing the model's ability to produce a drifting key addresses the first two directly.

The same reasoning applies to outgoing events. Standard Webhooks is a set of conventions for webhook producers so that consumers get a secure, consistent interface Standard Webhooks. In that scheme, consumers deduplicate on the event ID, which arrives in the webhook-id header Svix's webhook retry best practices. If your agent triggers an outbound event, that ID should come from the durable step, not from the model, for exactly the same reason.

Generating keys for webhook-driven mutations follows the same rule: derive from the triggering event ID rather than a fresh random value per attempt Monday's idempotency documentation. Reliable delivery and safe retries pull in the same direction: exponential backoff with jitter, a published schedule, and a dead letter queue for exhausted attempts Svix's webhook retry best practices Hookdeck's guide to webhook retry best practices. None of that survives a drifting key: the receiver will happily process ten deliveries it cannot recognize as the same event.

The failure classification also transfers cleanly. Senders distinguish retriable failures such as 5xx and connection timeouts from permanent ones such as 400 or 404, and treat 429 as a case for honoring Retry-After Hookdeck's guide to webhook retry best practices. A webhook provider that exhausts its schedule should move the event to a dead letter queue rather than drop it, which only makes sense if the deduplication identity was stable in the first place Hookdeck's dead letter queue guide. Svix publishes an exact retry schedule as an example of what receivers need to reason about delivery Svix's webhook retry best practices.

If you would rather hand the tool layer, key derivation and dispatcher to someone who has built it before, Quanta can take the requirement and build it with you: https://quantaintelligence.ai/

There is one narrow case. A customer asks for two identical purchases in two separate intents, and the model's job is to decide that a second intent exists. Even then, the model should decide that an operation is new, not what the key is. The orchestration layer mints the identity.

The distinction matters because identical arguments alone do not establish identical intent, a point that surfaces in the same guidance on caller-generated identity Formation's analysis of agent tool retries. The model is good at intent. Code is good at identity. Keep the split there.

A second legitimate case is classification. If you need the model to say "this looks like a duplicate of the invoice from earlier in the conversation," that is intent recognition, and the answer can feed a dispatcher decision. The dispatcher still owns the key. The model's opinion changes which step gets created, not what the step's identity hash is.