Caps on an Agent Finance Tool: Per Run, Customer and Day

It helps to stop arguing about which layer should own the cap and instead assign each layer the job it is actually good at.

AWS documents its own gateway throttling as token bucket applied on a best-effort basis, explicitly to be thought of as targets rather than guaranteed request ceilings 20. Take them at their word. A gateway is a fine place to stop a runaway loop from hammering your service, and a poor place to guarantee that customer 4471 does not exceed 50 dollars today.

The tool layer earns its keep for a different reason. Agents that receive an explicit budget signal behave differently: the Budget Tracker work from Google Cloud AI Research and Google DeepMind found that simply granting agents a larger tool-call budget fails to improve performance because standard agents lack budget awareness and hit a ceiling, while continuous budget signals produced more favorable scaling curves 17. Telling the model "you have 3.20 dollars of 10.00 left for this run" changes its planning. That does not replace the database check, it just makes the check fire less often.

Lightweight gateway implementations show the same shape. LiteLLM's iteration budgets apply a max iterations count and a per-session dollar cap, tracked by a trace ID, and return 429 when either is exceeded, with counters expiring after one hour by default 19. That handles a runaway loop inside one session. It does not handle two sessions for the same customer on the same day, which is why per-customer and per-day counters still belong in your own store.

The core primitive is a reserve that cannot oversubscribe. You hold a budget row per (customer, day) and a separate hold per run, and you claim against both inside one transaction.

CREATE TABLE spend_budget (
  scope_kind   text    NOT NULL,   -- 'customer_day', 'run'
  scope_id     text    NOT NULL,
  limit_cents  bigint  NOT NULL,
  used_cents   bigint  NOT NULL DEFAULT 0,
  reserved_cents bigint NOT NULL DEFAULT 0,
  PRIMARY KEY (scope_kind, scope_id)
);

CREATE TABLE spend_ledger (
  idempotency_key text PRIMARY KEY,
  scope_id        text   NOT NULL,
  amount_cents    bigint NOT NULL,
  state           text   NOT NULL,  -- 'reserved', 'settled', 'released'
  created_at      timestamptz NOT NULL DEFAULT now()
);

The claim is a single statement per scope, and it either matches one row or none:

UPDATE spend_budget
SET reserved_cents = reserved_cents + $amount
WHERE scope_kind = $kind
  AND scope_id   = $id
  AND used_cents + reserved_cents + $amount <= limit_cents
RETURNING limit_cents, used_cents, reserved_cents;

Zero rows returned means the cap would be broken. Run both the customer-day claim and the run claim in the same transaction. If the second returns zero rows, roll back so the first claim never happened. This is the reserve half.

Settlement follows the external call. On success, move the reservation into used; on failure, release it. Both are trivial updates:

-- settle
UPDATE spend_budget SET reserved_cents = reserved_cents - $amt,
                        used_cents     = used_cents + $amt
WHERE scope_kind = $kind AND scope_id = $id;

UPDATE spend_ledger SET state = 'settled' WHERE idempotency_key = $key;

Two rules make this safe under load. First, the ledger insert and the budget update go in the same transaction, so a crash cannot leave a reservation with no record or a record with no reservation. Second, the ledger insert uses the idempotency key as a primary key, so a duplicate request fails the insert before it touches any counter. Stripe's design document is explicit about this: the server correlates the key with the state of the request, and on a retry after a response failure it simply replies with the cached result 5. A practical guide to the same mechanism notes that stores should keep keys for a window long enough to cover typical retries, usually 24 hours to 7 days, and reject a key reused with different request parameters 6.

Worked example: one customer, one day, three overlapping runs

Assume a customer-day cap of 5,000 cents and a per-run cap of 2,000 cents. Three runs start within the same second.

Run C fails with a budget error even though its own run counter is empty, because the shared day counter is nearly full. That is the correct outcome: the customer-level ceiling is the promise you made.

Note step 7. Reservations should be conservative, but settlement records the actual charge, which can be lower. That gap is the difference between a hold and a debit and it is why you keep them in separate columns.

A cap is only as good as the identity of a spend attempt. Generate the key at the tool boundary, before the first call, and reuse it on every retry of the same logical action. Client-side guidance is consistent on this: create the key once, keep it, and resend the same key on every retry, and never build a key from a timestamp or a random number because two distinct payments could collide 2.

The backend's response table is small and worth memorizing:

The concurrency detail matters for production. Multiple requests with the same key can arrive at once, and the system must guarantee only one of them executes while the others wait or receive the cached response, which is typically achieved with database-level or distributed locks 6. In a single relational database, the primary key on the ledger is your lock, and you do not need a separate distributed lock at all. Check the ledger first, and if the key exists, return its stored state.

One more subtlety: the provider call needs its own key, derived from yours and saved before the call. If your process crashes after the provider charged but before you recorded the result, the retry reuses the provider key, and the provider returns the original charge instead of creating a new one 3. Your reservation stays held through the crash and settles on the retry.

Rate limits and monetary caps are different instruments and it helps to be explicit about which you are tuning.

A token bucket refills at a steady rate and permits bursts, which is what you want for protecting a service from a hot loop. A day cap is a step function: no spending past the boundary until reset. Mixing the two produces confusing behavior, such as a customer who hits a "budget" error at 11pm and is fine again at 11:01pm because your bucket refilled.

Budget semantics should be pure accumulation with an explicit reset boundary. Compute the day key from the customer's billing timezone, not from the server's clock, and store it as a string such as 2026-03-14:acct4471. Store the boundary in the key itself, so a timezone change produces a new row instead of retroactively editing an old one. When you reschedule the boundary, never mutate past rows; start a fresh row and leave the audit trail alone.

For burst protection, add a separate token bucket per customer in front of the tool, with a low refill rate. AWS's own documentation describes the mechanics accurately: the token bucket permits a burst and then settles to the steady rate, and the throttle is a target rather than a guarantee 20. Use it to shed load, not to enforce money.

There is one hard case worth calling out. If your day cap is denominated in a currency and your customers spend in several, accumulate in a single base currency using a rate you snapshot at reservation time and store on the ledger row. Recomputing historic spend at today's rate makes yesterday's overrun disappear, which is a compliance problem in a regulated context.

The most consequential design decision is what happens when the cap is reached. Silent throttling, where the tool quietly returns a degraded result or a smaller amount, is tempting because the agent keeps going, and it is wrong for money.

A hard stop returns an error the agent can reason about and the operator can audit. For an agent loop, that error should be structured and terminal for the run:

{
  "error": "budget_exceeded",
  "scope": "customer_day",
  "scope_id": "2026-03-14:acct4471",
  "limit_cents": 5000,
  "used_cents": 3850,
  "reserved_cents": 2000,
  "retry_after": "2026-03-15T00:00:00Z"
}

Two properties of that payload do real work. It tells the model why it cannot proceed, which converts a retry loop into a different plan, and it names the scope and the reset time, which lets the harness decide whether to escalate to a human. The distinction between human-in-the-loop, human-on-the-loop and human-in-command oversight is well established, with in-the-loop meaning a person can intervene before each consequential action takes effect, and the main failure mode being approval fatigue and rubber-stamping 10. A budget error is a natural escalation trigger, but only if it is loud.

Gate the riskiest writes the same way. Practitioner guidance from MLflow is blunt that read, write and destructive operations should carry different approval requirements, and that standing permissions to send email, move funds, or modify production without explicit human approval should be prohibited as a structural control rather than a UX preference 13. A cap is the automated version of that control; approval is the version for one-off exceptions above it.

Pair the error with observability you can actually query. Agent observability extends the traditional metrics, logs and traces with evaluations and governance, because agents are non-deterministic and their errors produce downstream effects that chat-only systems do not 14. Emit a span every time a reservation is made or refused, tagged with the scope ID, so a budget question becomes a filtered trace query instead of a reconciliation exercise.

Caps fail in boring ways: an off-by-one at the boundary, a settlement that runs twice, a rollback that leaves the reservation stranded. Test these explicitly.

The concurrency tests are the ones that catch real money. Run them with real parallelism against a real database, not an in-memory stub, because the guarantee you are buying comes from the row-level lock. A test that passes with a single-threaded fake proves nothing about the property you care about.

Then rehearse the operational side. Ask what a support agent sees when a customer says the tool refused a payment, and make sure the answer requires one query, not three. The ledger holds the idempotency key, the scope, the amount, and the state, which is enough to explain any refusal.

The pattern is portable. If you self-host a gateway, set channel-level budgets and iteration caps there for the LLM calls, and keep the money counters in your own database as described. If your agent runs through a single API, budget tracking per request is easier when every model call bills through one place, which is the pitch for Quanta Inference: text, image, video, audio and embedding models through one key, OpenAI-compatible and prepaid, so a spend record from one provider covers all of them.

For teams whose agents do real finance work rather than toy demos, the cost of getting this wrong is a customer with a double charge and an auditor with questions. If you want the counter design reviewed against your actual schema, or want the agent built with these controls from the start, describe it at Quanta and a specialist picks it up.