Blocking a Run When the Budget Check Fails: Exit Codes and Logs

The contract is the whole design. Keep it small enough to fit on one screen.

• Exit 0: budget check passed, proceed.
• Exit 10: budget check failed, run is refused.
• Exit 11: budget check could not be completed (missing config, tokenizer error, cost API unreachable). Also a refusal.
• Any other non-zero: unexpected error, treat as refusal.

Separating 10 from 11 matters for alerting: 10 means a human decision is needed or expected, 11 means the gate is broken and someone should fix the plumbing. Both block. Nothing else does.

Streams: JSON refusal on stdout, one human-readable line on stderr, and a copy of the JSON to a log sink. Never write the refusal JSON to stderr only, because CI systems and shell wrappers commonly mix or swallow it, and never write human text to stdout, because the parser will fail.

#!/usr/bin/env bash
set -euo pipefail

REFUSAL=$(mktemp)
if budget-check \
     --job-id "$JOB_ID" \
     --budget-id "$BUDGET_ID" \
     --manifest "$MANIFEST" \
     --requester "$GITLAB_USER_EMAIL" \
     > "$REFUSAL" 2> >(tee /dev/stderr); then
  exit 0
else
  code=$?
  cat "$REFUSAL"
  echo "budget gate refused run $JOB_ID (exit $code), refusal id $(jq -r .refusal_id "$REFUSAL")" >&2
  exit "$code"
fi

The indirection is deliberate: the wrapper never inspects the JSON to decide. It only forwards the exit code. Any logic that reads the JSON and then decides whether to proceed is a second gate that can disagree with the first.

The rule for fields is simple: every field must be something a person will ask about during the override discussion. If nobody will ask, drop it.

{
  "refusal_id": "ref_01J8ZK4Q7M3X9V2B6N0P1C5T8R",
  "kind": "budget_exceeded",
  "job_id": "etl-invoice-extract-4821",
  "budget_id": "ai-monthly-prod-2026-11",
  "observed_spend": { "amount": 31250.40, "currency": "USD" },
  "limit": { "amount": 30000.00, "currency": "USD" },
  "projected_run_cost": { "amount": 1840.00, "currency": "USD" },
  "headroom_after_run": { "amount": -3090.40, "currency": "USD" },
  "requester": { "id": "u_8841", "email": "[email protected]", "principal": "ci/etl-service" },
  "occurred_at": "2026-11-14T09:12:03Z",
  "files": {
    "count": 1247,
    "total_bytes": 402118872,
    "truncated": false,
    "list": ["inbox/2026-11-13/0001.pdf", "..."]
  },
  "reason": "projected completion would exceed monthly limit",
  "override": { "required_role": "finance-owner", "route": "#ai-spend-approvals" }
}

A few fields deserve comment.

refusal_id is the join key for everything downstream: the approval record, the audit entry, the reopened incident. Generate it once, at refusal time, and never regenerate it. If the gate is retried and refuses again for the same reason, reuse the id only if the job id, budget id and observed spend are unchanged; otherwise issue a new one.

Money is never a bare float. Amount plus currency, both in every object, so a limit in one currency and an observed spend in another cannot be silently compared. Store amounts as integers in minor units internally and render as decimal, or use a decimal type; never float for currency.

files carries the count and the list, because the first question in an override conversation is always "how much of this is optional?" A count of 1,247 files with a list is actionable; a count alone forces someone to rerun the estimate to find out. Cap the list length (say, the first 500 with truncated: true) so a runaway manifest does not produce a 40 MB log entry.

requester separates the human from the pipeline identity. A service account approved the change; a person should be on the hook. Log both.

Include the cost projection inputs, not just the number. If the projected cost came from token counts, the log should say which counting method was used, because the estimates differ meaningfully. Rough English heuristics put one token at about four characters or three quarters of a word, per OpenAI's token documentation, but technical documentation and source code run to roughly 1,400 and 1,500 to 2,000 tokens per 1,000 words respectively, per Iternal's comparison table. When the gate uses a tokenizer rather than an estimate, say so. The OpenReplay guide to token counting makes the operational point: heuristics are fine for capacity planning and wrong for billing, and the count must come from the tokenizer of the model actually being called, because the same sentence produces different counts across providers.

The override is not "rerun with the flag set." It is a decision, recorded before the rerun, tied to the refusal id, and consumable once.

Three roles, three permissions:

Name a person, not a team. "Approvals go to #ai-spend" is a queue where requests die. A named approver with a named backup is a decision. The approval record should contain:

• The refusal id it applies to, exactly.
• The approver's identity and the role that permitted the override.
• The timestamp of approval.
• The scope of the exception: this run only, or all runs for this job until a stated time.
• The approver's stated reason, in their own words, one sentence minimum.
• A hash of the refusal payload the approver saw, so a later change to the estimate cannot be smuggled past an approval granted against different numbers.

The last one is the detail teams skip and regret. If the approval says "approved for $31,250 observed spend" and the gate later recomputes $41,000 because the manifest changed, the hash mismatch must force a fresh approval. Compute the hash over the canonical JSON serialization with sorted keys and no whitespace.

The rerun path becomes: gate runs, finds an approval record whose refusal id matches and whose payload hash matches, and passes with exit 0 while logging the approval id as the reason for the pass. Any other state is a refusal. Do not add a --force flag that skips the lookup; if the lookup must be bypassed, that is a break-glass action with its own log entry and its own review, not a flag on the normal gate.

Approval workflows in agent systems have a well-documented failure mode: they become a bottleneck because the request lacks the context to decide quickly. StackAI's guide to human-in-the-loop approvals argues for routing to the correct role automatically and supplying an evidence pack that makes the decision in seconds. Your refusal payload is that evidence pack. If the approver has to open another tool to find out what the run was for, the gate will be routed around within a month.

Test the refusal, not the pass. The pass path is tested by every successful run in the pipeline; the refusal may fire once a quarter, which is exactly why it breaks silently.

A staging suite that takes minutes:

1. Set the staging budget limit to zero, currency matching the budget record. Run the gate. Assert exit code 10 and that observed_spend.amount > 0.
2. Validate the payload against a schema. Check every required field is present, that files.count equals len(files.list) when truncated is false, that amounts are integers in minor units, and that occurred_at parses as RFC 3339.
3. Point the gate at an unreachable cost API. Assert exit code 11 and a payload with kind: "budget_check_unavailable". This is the test that catches fail-open regressions, and it is the one teams omit.
4. Post an approval with a hash that does not match the payload. Assert the rerun is still refused, exit 10.
5. Post a matching approval. Assert exit 0, and that the log line names the approval id.
6. ASTM standard not needed here; instead assert that the same approval id cannot pass a second, different refusal. Reuse must be impossible.

Two more worth adding once the basics hold. First, a concurrency test: two jobs hold the same budget, both estimate spend that individually fits, and start simultaneously. The second must be refused or the budget must reserve headroom for the first; decide which, and test that the refusal records the in-flight job id. Second, a currency mismatch test: a budget in one currency and observed spend in another must produce a refusal with kind: "budget_misconfigured", not a silent comparison.

One practical note on the token counts that feed the estimate. If the gate counts tokens itself, it should use the tokenizer matching the target model. For OpenAI models that is tiktoken, selectable with tiktoken.encoding_for_model(model), per OpenAI's token documentation. For a quick command-line count of files or directories there is the count-tokens package, which uses tiktoken and supports directory patterns and JSON output, and the felvin-search token-count utility as a smaller alternative. The Winder practical guide is a useful reminder that token counts vary by language and script in ways that English-only heuristics miss, so a Spanish or Japanese corpus will trip a gate that was tuned on English.

If you are counting tokens by hand, note that the count you get from a plain string is not the count the provider bills. Message structure, tools and schemas all add tokens, which OpenAI's documentation calls out directly, and the Zuplo explainer on token-based limiting notes that the reliable figure comes from the usage fields in the response. Log the provider's reported usage post-run and compare it with the gate's estimate; a persistent gap means your estimate is wrong and the limit is effectively a different number than the one people agreed to.

A nightly invoice-extraction job, 1,247 PDFs, budget ai-monthly-prod with a $30,000 monthly limit, month-to-date actuals $31,250 at the time the gate runs.

The gate computes projected run cost from the manifest: 1,247 files, average 14,000 input tokens each, plus 700 output tokens each. That is about 18.3 million input tokens and 0.87 million output tokens. At a hypothetical rate of $0.80 per million input and $2.40 per million output, the run costs roughly $14.70 plus $2.09, about $16.80. That is tiny against a $30,000 budget.

But the budget is already over. Observed spend $31,250 against a limit of $30,000 means whatever the run costs, the gate refuses, exit 10, kind: "budget_exceeded", headroom after run negative $1,266.80. Note what the gate did not do: it did not ask whether the projected run was small. A limit that has already been exceeded is not reopened by a cheap job. If you want cheap jobs to be exempt, that is a different budget with its own limit, not a rule inside this gate.

Now the same example with the budget healthy at $20,000 spent. Projected cost $16.80 plus a conservative 1.7x multiplier for retries and overhead, which Iternal recommends for projecting spend, gives about $28.60. Headroom after the run is $9,971.40 and the gate passes, exit 0.

Those are the only two outcomes the wrapper needs to understand. The third, budget_check_unavailable, is the one that saves you. If the cost API is down at 02:00, exit 11, refuse, page the on-call. The run waits an hour. The alternative is discovering the problem from the invoice.

For teams running many of these pipelines against multiple providers, the per-provider accounting gets awkward fast. Quanta Inference puts text, image, video, audio and embedding models behind one API key with prepaid, dollar-denominated pricing, which removes the cross-provider reconciliation from the gate's job: https://quantaintelligence.ai/inference. The gate still needs its own logic, but it compares one number against one limit.

Once the gate is in the path, four things need watching.

Alert on exit 11 separately from exit 10. A refusal is a policy outcome; an unavailable check is an incident. Route them to different channels.

Track how often the gate passes with an approval attached. A gate that passes with an approval half the time is not a budget, it is a formality. If approvals are frequent, the limit is set wrong or the manifest scope is too broad.

Keep refusal payloads for at least as long as the financial audit period requires, and keep the approval records longer. They are the evidence that a spend was authorized. The payload includes identity and file names, so treat the retention store as data with access control, not a plain log bucket.

Add a post-run reconciliation check: compare the gate's projected cost with the provider's reported usage, and log the delta against the refusal id or the pass record. A pattern of underestimation means the gate needs a higher multiplier, and the fix is a parameter, not a code change.

If the gate ends up serving multiple teams with different budgets and approvers, the routing gets harder than the checking. Quanta Experts are built for that pattern: each expert reads your context and returns finished deliverables, which suits the evidence pack and routing side of the workflow better than the arithmetic: https://quantaintelligence.ai/agents.