Setting a Per-Task Budget an Agent Cannot Exceed

Name the ways agents overspend, because each demands a different control.

Recursive tool calls. The agent calls the same tool repeatedly after ambiguous or failing output. Counting volume catches this only if the check runs before dispatch, not after the response returns.

Retry storms. Automatic re-invocation after transient errors, without backoff or a ceiling, multiplies spend silently. Retries are a spend path and need their own accounting.

Self-invoked sub-agents. Children accumulate cost outside the parent's local counters. Practitioner write-ups list subagent budget limits, with explicit timeout caps on spawned children, among the controls that made a measurable difference (Dify discussion #35778). Attribution needs a task identifier on every invocation and aggregation at a control plane that sees all children.

Context-driven over-exploration. Bloated context files can raise inference cost by more than 20 percent per session with little measurable benefit on complex tasks, according to research cited in the same Augment Code analysis. The token budget is the backstop, because it caps the consequence whatever the cause.

Security researchers frame the same problem from the other side: an agent without restrictions can turn a minor logical error or a successful prompt injection into a cost explosion in minutes, so throttling belongs in the governance controls, not only the load-management tooling (AI Agent Rate Limiting is Broken). A hard budget is the control that still works when the agent's reasoning has been subverted.

The outermost spend point is the model call. Put a proxy between the orchestrator and the provider that attaches a task identifier to every request and records cumulative usage (Trussed implementation guide). Two properties make it worth building even if you also enforce inside the loop.

It sees calls the runtime did not make deliberately: SDK retries, secondary calls from a library, calls from a sub-agent that bypassed the parent's code path. And it enforces against a shared store, so concurrent branches draw from one balance instead of private local counts.

The check happens before dispatch. The gateway reads the remaining budget, estimates cost (prompt tokens plus the maximum output the request permits), and rejects the call if the estimate exceeds what is left. Once the provider has billed a request, refusing the response saves nothing, so a conservative estimate that occasionally refuses an affordable call is the right bias.

The reservation pattern is the clean implementation: reserve the estimated maximum atomically, make the call, then settle the difference against actual usage in the response metadata. OpenTelemetry's GenAI conventions standardize exactly those attributes, including input and output token counts and finish reasons, so settlement works across providers without custom parsing (OpenTelemetry for AI Systems).

reserve = estimate_cost(prompt_tokens, max_output_tokens)
if not store.try_decrement(task_id, tokens=reserve):
    raise BudgetExhausted(task_id, "tokens")
response = call_model(request)
store.settle(task_id, actual=response.usage, reserved=reserve)

try_decrement must be a single atomic operation that fails when the balance is insufficient, never a read followed by a write.

The gateway can stop a call but cannot stop the agent from trying again. The loop-level check makes exhaustion terminal.

A typical loop repeats two steps until the model decides to finish: a model call, then execution of whatever tools it requested (LangChain context engineering docs). That cycle is the natural unit of accounting. Before each model call, decrement by the estimated prompt size; before each tool execution, decrement the call counter by one. If either fails, raise a terminal error and do not hand control back to the model.

Why terminal rather than soft? An agent told it is low on budget will often economize by skipping verification and producing a confident but unchecked answer. You have then paid for most of the task and received an unreliable result. A terminal error preserves partial state for inspection and lets the caller decide whether to grant more.

Loop detection belongs here, because only the loop knows what the agent is trying to do. Track call signatures (tool name plus normalized arguments) and treat repeated near-identical calls as the signal rather than raw volume; a legitimate research agent may make many distinct calls while a broken one makes a few identical ones (Trussed implementation guide). When a signature repeats past a threshold, inject a corrective observation or terminate, without waiting for tokens to run out.

There is a production-tested refinement. Spend is skewed: practitioner data from a six-agent deployment found a small set of frequently running tasks consumed most tokens, and moving those to smaller models with light context produced most of the savings (Dify discussion #35778). Encode that: tight ceilings for routine task classes, generous ones for complex classes.

Framework middleware is where this usually lives. LangChain's middleware API hooks any step in the agent lifecycle, placing the decrement at model-call and tool-call boundaries without touching agent logic (LangChain docs). Whatever the framework, the requirement is the same: the hook runs synchronously in the path, not as an async observer reporting afterward.

Tools are the second spend path and often the more dangerous one, because a call can hit a paid third-party API, a database, or an internal service whose cost never appears on a model bill.

Route invocations through a proxy that enforces the call counter and per-tool ceilings. A search tool and a code-execution tool have wildly different cost and risk profiles and should not share one number. The Model Context Protocol is now the common transport for tool connections (our comparison of MCP, Skills and AGENTS.md), which helps here: a proxy that speaks the protocol's flow can sit in front of every server unchanged.

The timer is the piece most teams forget. Token and call counters are checked at boundaries, so an agent stuck inside one long step never reaches the next boundary. A wall-clock deadline, checked by a watchdog that can cancel the in-flight operation, is the only control that catches a hung dependency. Set it from the p99 of successful runs and cancel rather than wait, because a hung step also blocks the work queued behind it.

Missing one layer turns control back into accounting. A gateway alone cannot stop retries; a loop check alone cannot see a sub-agent with its own counters; a timer alone cannot stop a fast loop burning tokens in seconds.

Concurrency is where budgets quietly break. An agent fans out into three parallel sub-agents, each holding a local view of one million remaining tokens. Each branch spends eight hundred thousand on legitimate work. Every branch stayed within its own view, the task spent 2.4 million against a one-million ceiling, and no component noticed.

The fix is a central store with an atomic conditional decrement, so a decrement either fully succeeds against current state or fails. Every enforcement point, in every process, for every sub-agent, decrements the same record keyed by task identifier.

def try_decrement(task_id, tokens):
    for attempt in range(max_attempts):
        rec = store.get(task_id)
        if rec.remaining_tokens < tokens:
            return False
        if store.compare_and_swap(task_id, rec.version,
                                  remaining=rec.remaining_tokens - tokens,
                                  version=rec.version + 1):
            return True
    return False          # contention failure is exhaustion: fail closed

Two details do the work. The versioned compare-and-swap makes a concurrent write fail the swap and retry against fresh state. And an exhausted retry returns exhaustion rather than proceeding on stale state, so an unreachable store means refuse to spend, not spend optimistically. A budget that fails open under database pressure is not a control.

Sub-agents inherit the parent's record rather than getting a fresh one. Give each child a sub-ceiling against the shared balance (say a quarter of what remains) enforced in the same atomic operation, which bounds fan-out without escaping parent accounting. Deadlines compose differently: a child's effective deadline is the earlier of its own limit and the parent's remaining time.

Retries need the same treatment. A retried call is new spend, not a replay. If your SDK retries automatically, the retry must pass the gateway reservation again, or a transient provider error becomes a billing multiplier. Set the retry ceiling well below the task's tool budget so one flaky dependency cannot consume the whole allowance.

An agent that can edit its own budget will. That is not malice; it is an optimization loop doing what optimization loops do. The adversarial case is concrete: a single malicious input can be amplified into a flood of resource-intensive actions, so the budget must not be reachable by the thing it constrains (AI Agent Rate Limiting is Broken).

Keep the record outside the agent's writable surface. If the agent has a sandbox filesystem, the budget is not a file on it. If it has a shell, the budget is not an environment variable the shell can export. The agent holds an opaque task identifier; the record lives in the gateway's store.

Let the agent read its remaining budget but never write it. Telling an agent what it has left is useful, because a well-behaved agent economizes. Letting it set the number is not. Expose a read-only endpoint and keep the decrement path on the enforcement side.

Log every decrement with the actor, in the same trace hierarchy that carries model and tool spans. OpenTelemetry's GenAI conventions define that span structure, including token usage and finish reasons, and vendor-neutral instrumentation audits across providers without rewriting your logs (OpenTelemetry for Generative AI). That trail is what lets you reconstruct an incident afterward.

Context files carry policy, not credentials. Instruction files such as AGENTS.md shape behavior, but they are context, and context can be summarized or overridden during a long run. Practitioner discussion converges on the same point: those files are useful for commands, conventions and terminology, but they are not a security boundary (Hacker News discussion). The same goes for skill packages that load on demand: they teach an agent how to work, not what it may spend (VS Code Agent Skills docs).

The deeper tamper vector is context itself. A long run can cause the model to lose track of its instructions, as documented in the context rot and context poisoning literature (Splunk's production guide). The budget still holds, because it lives outside the model's reasoning. Budgets do not depend on model cooperation, which makes them the one control that keeps working when judgment degrades.

A research task class gets 200,000 tokens, 40 tool calls, and 10 minutes.

Step one: the gateway estimates a 6,000-token prompt with a 2,000-token output ceiling, reserves 8,000, and settles at 6,100 when the response reports actual usage. Remaining: 193,900.

The agent requests a search. The proxy drops the call counter from 40 to 39, forwards it, and logs latency. The result is 3,000 tokens of output that the next prompt must carry.

By step eight the prompt has grown to 31,000 tokens from accumulated history, so the call settles at 30,800 and remaining sits at 118,000. The same work cost roughly five times more than at step one, which is the triangular effect and the reason the budget is a running total.

At step fourteen the call counter reads 4 left and tokens read 41,000. The agent asks for a search its loop detector recognizes as a near-identical repeat of two earlier calls. The loop injects a corrective note, the agent changes approach, and spends 12,000 tokens on a fresh angle. Loop detection worked before the hard budget had to.

At minute nine the watchdog sees the current tool call waiting 90 seconds on a third-party API. Successful runs have a p99 of seven minutes and the deadline is one minute away, so it cancels. The proxy marks the call failed, and the agent, still holding token and call budget, tries another source and finishes at nine minutes forty with 9,000 tokens and two calls left.

Had any ceiling been crossed, the runtime would have raised a terminal error, written partial state to the trace, and returned a result marked incomplete. That is the budget working, not failing.

If you want one OpenAI-compatible key across text, image, video, audio and embedding models, prepaid and priced in US dollars, Quanta Inference makes per-task counting uniform: the same reservation pattern applies whichever kind of model a task reaches for.