Budget Caps on Model Spend: Where to Put the Limit

Feature caps, team caps and customer caps do different work. Confusing them is how you end up with a cap that fires too late or blocks the wrong thing.

Feature caps are your runaway protection. An agent that retries a malformed tool result and loops until a guard limit fires bills tokens on every iteration Braintrust's cost tracking playbook. A tight per-feature daily cap on the agentic surfaces, with a looser cap on deterministic call paths, catches the loop within hours rather than at month end. Set feature caps from measured cost per resolved task, not from total tokens, because an agent that solves a problem in one expensive call can be cheaper than one that grinds through ten cheap ones Big Data Boutique's production playbook.

Team caps are governance, not safety. They answer "how much is the analytics team allowed to spend this month on experiments" and they work best as a soft alert at 80 percent and a hard stop at 100 percent. The nuance worth knowing: in systems with both personal and team budgets, a key that belongs to a team draws on the team budget, and the key owner's personal budget does not apply LiteLLM's budget documentation. Decide that precedence explicitly, or you will argue about it during an incident.

Customer caps protect margin. If you sell a usage-based plan, a customer cap is a commercial term as much as a technical one. It needs to be visible to the customer before it fires, and it should be expressed in their units (credits, requests, or dollars), not in raw tokens.

This is the decision most teams get wrong, usually by defaulting to alerts because hard stops feel hostile.

A soft alert changes nothing about execution; it notifies owners that spend is trending over plan. A hard cap stops serving requests at the threshold, and it is the only control that changes who carries the cost of a runaway workload an analysis of hard caps versus budget alerts. That incident writeup describes a pre-production worker loop that generated $7,152.89 in eight days against a normal monthly development baseline of about $11, more than three orders of magnitude over baseline, with alerting and quotas present but no platform-level stop.

A middle option, constrained mode, keeps the workload running under tighter limits: cheaper models, smaller context windows, lower token ceilings, restricted tool access. Guidance from practitioners is to use alerts for awareness, constrained mode when graceful degradation is acceptable, and hard caps when the cost of continued execution exceeds the cost of interruption NHI Mgmt Group's decision guide.

Here is a decision rule you can apply per surface:

1. If a human is waiting and the feature is revenue-critical: soft alert plus constrained mode. Never a hard stop without human review.
2. If a human is not waiting (batch, background, evaluation): hard stop. Interruption costs nothing a rerun cannot fix.
3. If the workload is autonomous, agentic, or externally exposed: hard stop, because spend spikes often accompany other control failures NHI Mgmt Group.
4. If the workload is a pilot with no named owner: hard stop at a low ceiling. Unowned spend is the risk you are insuring against.

One trap: alerts only work if someone is on the other end. If the alert goes to a channel nobody watches, you have a hard cap in name only. Pair every alert with a named owner and an approval path.

You cannot cap what you cannot attribute. The minimum useful logging set is input tokens, output tokens, model, and metadata tags for feature, customer, and environment, captured per request NeuralTrust's token monitoring guide. Anything coarser will leave you guessing during the first overrun.

Tag at the call site, not by parsing logs after the fact. Post-hoc parsing forces you to reconstruct context the application already lost, while a label attached at the call survives downstream changes to dashboards and alerting Braintrust. Keep low-cardinality fields (feature, deployment, model) as chartable dimensions and high-cardinality fields (user, agent run) as filterable metadata only, or your dashboards will become unusable.

A worked example makes the stakes concrete. Suppose a 500-word background block is added to every prompt in a feature. That is roughly 667 input tokens per call. At an input price of $2.50 per million tokens, the addition costs about $0.00167 per call, which across 100,000 daily calls is about $167 per day, or roughly $5,000 per month from a single prompt design choice Worklytics' tracking guide. A feature cap set from last month's uncapped baseline would absorb that silently. A cap set from cost per resolved task flags it.

Providers increasingly expose the raw data you need. OpenAI publishes a Usage API and a Cost API that return usage and cost data broken down by model and project, which is enough to build your own rollups OpenAI's usage API cookbook. OpenTelemetry-style trace attributes such as token counts per span let the same numbers flow into your existing observability stack.

If you would rather not operate the accounting layer yourself, a model gateway with per-key and per-customer budgets can carry it. Quanta Inference (https://quantaintelligence.ai/inference) exposes text, image, video, audio and embedding models through one OpenAI-compatible API key with prepaid, US-dollar pricing, so a single budget applies across modalities instead of one per vendor.

The failure mode of a poorly designed cap is not overspend; it is a service outage on a false positive. Design the fallback before you set the number.

A practical sequence, in order of escalation:

• Downgrade. Route the next request for that feature to a cheaper model that clears the quality bar. Routing routine traffic to smaller models while reserving frontier inference for hard reasoning is the highest-leverage cost lever in most stacks Bifrost. During budget pressure, the same routing becomes a degradation strategy.
• Defer. If nobody is waiting, move the work to an asynchronous batch queue. Batch APIs from major providers discount asynchronous work relative to the synchronous price and deliver within 24 hours NeuralTrust's batching guide. A cap that reroutes background work to batch converts an outage into a delay.
• Queue with a message. For user-facing features, return a clear state: "This feature has reached its usage limit for today and will resume at midnight UTC." Tell the user what happened and when it resets. Silent failure reads as a bug; an explicit limit reads as a product decision.
• Ask a human. For high-value accounts, route the request to an approval flow rather than a block.

Watch the interaction between caps and caching. If a cap downgrades traffic to a model that does not share the original model's cache, the fallback can cost more than the capped path for cache-heavy features NeuralTrust's reduction guide. Model the fallback's effective cost including cache behavior, not just its sticker price.

And never let a cap block without a fast path to raise it. During an incident, someone needs the authority to lift a cap in minutes. Write that authority into the runbook alongside the threshold.

A defensible starting point is three times the trailing weekly average for a feature, expressed as a daily cap, with the team cap at 110 percent of the sum of its feature caps. The margin absorbs normal variance without absorbing a loop.

Set thresholds as percentages of the plan, not absolute dollars. A common pattern is an alert at 80 percent, constrained mode at 95 percent, and a hard stop at 100 percent TrueFoundry. This gives owners a window to react before the ceiling, and makes the ceiling itself rare enough that nobody campaigns to remove it.

Below is a working configuration sketch showing how caps compose. The numbers are illustrative.

budgets:
  feature:
    - name: support-reply-draft
      daily_cap_usd: 60            # 3x trailing daily average
      on_limit: downgrade          # fall back to a cheaper model
    - name: agent-research
      daily_cap_usd: 150
      on_limit: hard_stop          # autonomous, unowned at the edges
  team:
    - name: growth
      monthly_cap_usd: 2400        # 110% of the sum of feature caps
      alert_at: 0.80
      constrained_at: 0.95
      hard_stop_at: 1.00
  customer:
    - plan: usage_based
      monthly_cap_usd: <contract value>
      on_limit: ask_approval

Review the caps whenever the model mix or prompt template changes. A prompt change that adds context raises cost per call for every session before total spend becomes a billing problem Braintrust, so treat a prompt version change as a trigger to re-baseline the cap.

The insight most first-pass setups miss: caps are not a ceiling on ambition, they are a way to make the cost of a mistake bounded. A team with a tight feature cap and a generous team cap will ship more experiments than a team with one shared org cap, because they are not competing with each other for the same headroom.