Testing That a Cap Fires: Fixtures for the Refusal Path

Placements differ in what you can actually control from a test, so each fixture needs a different lever.

Two rows deserve emphasis. Concurrency fixtures are the ones people skip, because holding permits open feels artificial, but reservation-based limiters exist precisely to close the race where many concurrent requests each pass a zero-cost check before any of them reports usage (Kuadrant docs). Cost fixtures are the ones people get wrong, because the cap is often enforced against spend read from a database.

An input full of filler words is not a token fixture. If your fixture says "this prompt is 8,192 tokens" and the real tokenizer says 9,500, the test passes while production overflows. Use the model's own tokenizer where you can: tiktoken mirrors the byte-pair encoding used by OpenAI models, and one worked example shows that the characters-divided-by-four heuristic estimated 8 tokens for a string that actually encoded to 11, a 37% miss (galileo.ai). That error direction matters: heuristics can undercount, so a fixture built on one can sit under the limit and never fire.

Treat the same caution for chunked retrieval inputs. The token limit applies to the assembled prompt, so the fixture must total system prompt, retrieved chunks and expected output, not just the chunks you control (MongoDB, LangChain). Microsoft's documentation is a useful reminder that the hidden system prompt can be substantial, roughly 1,200 tokens by their estimate, before your own content is counted (Microsoft Learn).

For cost fixtures, work in fractions of the limit rather than a dollar figure copied from a console. A fixture that seeds spend to max_limit - one_input_unit and then submits a request whose known input cost is two units will fire, and it will keep firing when prices change. Price tables move; the ratio does not.

Your fixture should assert the refusal object field by field. Minimum viable shape, in language-neutral terms:

{
  "ok": false,
  "error": {
    "code": "BUDGET_EXCEEDED",        // stable, machine-readable, documented
    "limit": "team_monthly_cost",      // which placement fired
    "observed": 412.60,                // the value at decision time
    "threshold": 400.00,               // configured limit
    "retryable": false,                // or a retry hint
    "retry_after_seconds": 86400,      // only when retryable is meaningful
    "correlation_id": "8f2c..."         // same id as the log row
  }
}

Four assertions you should make and three you should refuse to make:

• Assert the code is a stable string from a documented set, not a free-form message.
• Assert the limit name identifies the placement, so an operator can tell a per-key cap from a team cap. Team and member budgets are enforced at different scopes in practice, and a key that belongs to a team resolves against team budgets rather than personal ones (LiteLLM).
• Assert the retry hint is present and consistent with the reset window, since teams do distinguish "stops serving" from "notifies" and "degrades instead" (nhimg.org).
• Assert the correlation ID appears in both the object and the log.
• Do not assert the dollar value exactly; assert the relationship to the threshold.
• Do not assert on a message's wording unless the message is part of your public contract.
• Do not assert a status code alone.

Status codes matter for the transport layer, not the logic. A 429 without a Retry-After header is a cap that tells the caller nothing, and header-level checks belong in the same test as the object-level ones (agentgateway, Beeceptor).

A refusal that is not logged is indistinguishable from a crash during an incident review. The assertion is mechanical: after the call under test returns, fetch the log row by correlation ID and compare a fixed set of fields against the refusal object.

def test_team_cost_cap_writes_matching_log(db, client, team_id, correlation_id):
    seed_team_spend(db, team_id, limit=400.00, current=399.50)

    resp = client.post("/v1/complete", json=oversized_input,
                       headers={"x-correlation-id": correlation_id})

    assert resp.status_code == 402
    body = resp.json()
    assert body["error"]["code"] == "BUDGET_EXCEEDED"
    assert body["error"]["threshold"] == 400.00

    rows = db.query("select * from request_log where correlation_id = %s", correlation_id)
    assert len(rows) == 1
    row = rows[0]
    assert row.outcome == "refused"
    assert row.error_code == body["error"]["code"]
    assert row.limit_name == body["error"]["limit"]

The field comparison is what catches drift. Teams that log cost per request and expose a log store with full cost data already have the raw material (dev.to); the fixture's job is to prove the refused path writes a row too, with the same ID and the same code. If your gateway logs refusals with a different ID generator than the one returned to the caller, this test is the only place that will ever notice.

A worked example of why the log assertion is worth it: an operator filters logs by correlation ID from a customer ticket, expects one refused row, and finds none. Without the test, the gap surfaces only when someone is already reconciling a bill.

The fixture lives in your process. The bill does not. A hard cap that returns a refusal while an upstream request is already in flight may still be charged for that request, and provider-side token budgets often count after the response, meaning the request that pushes you over completes and the next one is rejected (agentgateway). Local rate limiting on one replica also keeps counters independent of other replicas, so a fixture that proves one process refuses says nothing about the fleet (agentgateway).

So pair the object assertion with a second one: assert that the call under test never reached the provider adapter. In practice, inject a fake adapter that records calls and assert the recorder is empty on the refusal path. That single assertion is the difference between testing the conditional and testing the cap.

Global limits add a fleet dimension that a unit fixture cannot cover. That is what an integration test is for, and it deserves a separate, slower test with a shared counter store, not a fixture that pretends a local counter is global (Gravitee, MLflow).

Order of work matters more than tooling. Do it in this sequence.

1. List every placement on the request path and the exact unit it caps. If you cannot name the unit, you cannot build the fixture.
2. For each placement, write the seed step first. Seeding is where most fixtures actually fail, since spend caps read from a store and memory caps read from the assembled prompt.
3. Write the over-limit input using the real tokenizer or a real spend increment, and assert the measured value in the test so a later tokenizer or price change fails loudly.
4. Assert the refusal object, comparing fields against the configured limit rather than hard-coded numbers.
5. Re-fetch by correlation ID and compare the log row field by field.
6. Assert the provider adapter recorded zero calls.
7. Add one negative fixture per placement: a request exactly at the limit must succeed. Without it, a cap set to zero by accident passes every test you just wrote.

A checklist for review:

Two rows are worth singling out. Fetching the log row by recency works perfectly until your suite runs in parallel, then it flakes and gets deleted. And the at-limit success case is the one that catches the opposite bug: strict-greater comparisons that refuse the last allowed unit.

If your caps are enforced inside a gateway you do not control, the same fixture pattern applies one layer up, against a mock that replays the gateway's refusal. Container queues, image tokens and long contexts also change the size of a single request, so fixtures that assume text-only input will understate consumption (Microsoft Learn, LLM API).

When the caps span several providers and bill shapes, the fixture set grows with each adapter. Consolidating text, image and audio calls behind one key that is already metered removes some of that adapter surface, which is the practical argument for Quanta Inference (https://quantaintelligence.ai/inference) as the single place where your fixtures seed and assert.