Estimating Token Cost Before You Write a Check

Every check has four cost components. Estimate them separately, because each one fails differently.

1. Fixed overhead. The system prompt, guardrails, formatting instructions and tool schemas. These ride on every call whether or not the user says anything. For a tool-heavy agent, the schema block alone can be large, and it is easy to forget because you never type it.

2. Variable input. The user message, retrieved documents, file contents, conversation history, and any injected context. This is the part that grows and the part you can control.

3. Completion. The output the model will generate: a short classification label, a structured JSON object, a patch, a full document. You cannot count this exactly before you run, so you bound it.

4. Multiplier. The per-token input and output prices you were quoted, plus any cache discounts. Output tokens usually cost more than input, so the completion bound deserves attention.

A practical estimator takes the sum of parts 1 and 2 as an exact count, adds a bounded estimate for part 3, and applies part 4. The exact count is the hard part, and it is harder than most people expect.

The order of operations matters. Provider model guidance is written around the request you send, and OpenAI's own documentation for its current models shows the request carrying a model name and an input string, with tools and instructions configured alongside per the OpenAI API usage guide. The counting should mirror that structure exactly: the same fields, the same nesting, nothing summarized.

The most common estimation error is counting the wrong thing. People count the user message, or they count words and divide. Neither is the payload. The payload is the full list of messages and tools you hand to the model.

The right approach is to measure the actual request object. Serialize the exact request you intend to send, then pass that serialized form to a tokenizer or a counting utility. This avoids the two silent errors that break most estimators: dropping the tool schemas and miscounting whitespace.

A sketch of the workflow in Python:

import json
from your_tokenizer import count_tokens  # wraps a tokenizer for the target model

def estimate_input_tokens(system_prompt, tools, retrieved, user_message):
    payload = {
        "system": system_prompt,
        "tools": tools,
        "context": retrieved,      # list of chunks you will actually send
        "user": user_message,
    }
    text = json.dumps(payload, separators=(",", ":"))
    return count_tokens(text, model="your-model")

Three details matter here. First, serialize the request the way the API will serialize it, because whitespace and key order change the count slightly. Second, include the tool schemas. They are part of the request, and on agent workloads they are often a large fraction of it. Third, count with the tokenizer of the model you are actually buying, not a general-purpose one, because vocabularies differ and the difference is multiplicative per Firecrawl.

The same analysis found that raw HTML is dramatically worse than markdown for the same content: 4,103,706 tokens as HTML versus 190,817 as markdown across fifteen pages. Format choice dominates model choice for cost. So the input you count should be the input you intend to send, after cleaning, not the raw scraped page.

When you cannot run a tokenizer, you fall back on ratios: tokens per word, tokens per character, tokens per line of code. These are useful for early sizing and dangerous for billing. The widely repeated guidance that English prose runs about 1.3 to 1.5 tokens per word is a planning range, not a billing number, and the same source notes that code and JSON land outside it because braces, keys and punctuation all count differently per the tokens-to-words converter's own caveats.

Here is a comparison of the content types you will meet, and what each does to an estimate:

The multilingual row deserves its own warning. Tokenizers trained mostly on English fragment other languages more heavily, and a study across ten production tokenizers found compression differing by 100 percent between extremes on emoji and numeric content and by 20 to 50 percent on everything else per Firecrawl's summary of the tokenizer research. The same work found that English essays compress about 25 percent better than technical writing. If your check processes international content or dense technical material, an English-prose-calibrated ratio will be badly wrong.

For a hands-on feel for how a specific string tokenizes, a tokenizer explorer is useful during design like the GPT for Work token counter, but it should not be the production estimator. Production estimation should use the tokenizer or counting path for the model you are actually buying.

Fixed overhead is where agent estimates break. A single-turn prompt might be 300 tokens of user text. The same task inside an agent can carry thousands of tokens of system instructions and tool schemas before the user says anything. The OpenAI agent-building guide is explicit that an agent consists of a model, tools and instructions, and that tools extend the model's capabilities by calling external systems per OpenAI's practical guide to building agents. Every one of those tools has a description and a parameter schema, and those are input tokens on every call in the loop.

A worked example makes the proportions concrete. Suppose a support-triage agent has:

• A system prompt and guardrails of 1,200 tokens.
• Eight tool schemas averaging 250 tokens each, so 2,000 tokens.
• A retrieved policy excerpt of 3,000 tokens.
• A user ticket of 200 tokens.
• A structured completion of about 400 tokens, with roughly 150 of those spent on reasoning you cannot see.

Input is 6,400 tokens, completion is 400. If the quoted input price is 2 US dollars per million tokens and output is 8 US dollars per million, the estimate is (6,400 x 2 / 1,000,000) + (400 x 8 / 1,000,000), or about 1.3 cents plus 0.32 cents, roughly 1.6 cents per call. Run that as a workflow with three model calls and two tool round trips and it becomes roughly five cents. Run it 20,000 times a day and the line item is about a thousand dollars a day. The fixed overhead, not the user's 200-token ticket, drives the number.

Cached context changes this math in your favor. If the system prompt and tool schemas are stable, caching the prefix can cut input cost substantially, with savings depending on the provider and workload per Mintive's monitoring guide. Cache-read tokens are roughly 90 percent cheaper than fresh input, which is why the effective-tokens formula GitHub's engineering team uses weights cache reads at 0.1 and fresh input at 1.0 per Glean. But caching only helps if your estimate models it. If you predict full input price on 6,400 tokens but only 3,400 are fresh, you overestimate and might cancel a check that was affordable. Estimate both cases: the cold-cache cost and the warm-cache cost.

This is also where prompt bloat hides. Small template changes accumulate: a few-shot example here, a longer tool description there. A few-shot example or a longer tool description can raise input tokens on every call without causing an obvious failure as Braintrust's token tracking guide notes. Your estimator catches this if it runs on the real payload, because the fixed overhead line grows quietly and visibly in the prediction.

Prompt engineering itself affects overhead. The Prompt Engineering Guide notes that prompt design covers not just phrasing but the broader set of techniques for interfacing with models, including augmentation with domain knowledge and external tools per the Prompt Engineering Guide. Augmentation with retrieved documents is exactly the variable block that dominates estimates.

A ratio-based estimate is a prior. A measured run is evidence. The discipline that makes estimation trustworthy is a small calibration set: pick five to ten representative inputs, run them, record the actual usage fields, and compare predicted against actual.

Provider responses carry the ground truth. OpenAI returns usage fields such as prompt tokens, completion tokens and cached-token details, and Anthropic returns input and output token fields plus cache read and creation fields per Braintrust's tracking guide. OpenTelemetry defines a portable schema for the same data, with attributes for input, output, cache read, cache creation and reasoning tokens, which is useful if you already standardize on it. Capture the real usage numbers, not a guess reconstructed from text length.

Then compute the correction factor:

factor = actual_input_tokens / predicted_input_tokens

If your predicted input is consistently 10 percent low, your serializer is dropping something, most often tool schemas or a retry's history. If predicted completion is far low, your completion bound is too tight, and that is the expensive error because output tokens cost more than input. The median output-to-input price ratio sits around four to one across providers, with some premium reasoning models reaching eight to one per Glean, so a completion undercount is roughly four times as costly as an input undercount of the same size.

A few calibration habits worth adopting:

• Calibrate per task type, not once for the whole system. A classification check and a code-generation check have different shapes.
• Recalibrate when the prompt or tool set changes, because that is exactly when the ratio shifts.
• Keep the calibration inputs in version control alongside the prompt. They are test fixtures for the estimator.
• Record the p90 of the actual-to-predicted ratio, not just the mean. Budget decisions should survive the tail.

It also helps to calibrate against how the model is actually invoked. OpenAI's model documentation notes that its top model reached stronger results in several evaluations using substantially fewer output tokens, and that estimated API cost per task was lower than earlier models despite higher per-token pricing per the OpenAI API usage guide. The lesson for estimation is that tokens per task is the unit to calibrate, not price per token.

The Berkeley RDI position paper on autonomous software development reports that a team of sixteen parallel agents built a working C compiler for under 20,000 US dollars per the Berkeley RDI analysis. That is a striking number precisely because someone had to decide it was plausible before the agents ran. Estimation is what makes that decision possible rather than reckless.

An estimate that does not change behavior is decoration. The decision rule is simple and it should be written down before the check runs.

Compute predicted cost as input tokens times input price plus completion tokens times output price. Compare it to the check's budget. If the prediction is within budget, run. If it exceeds it, do not run and hope; change the inputs.

The levers, in order of typical payoff:

1. Trim retrieved context. Drop chunks below a relevance threshold, or retrieve fewer. This is usually the largest variable block.
2. Drop unused tools. Every schema you remove is fixed overhead gone from every call in the loop.
3. Split the task. Smaller checks with narrower context are cheaper per run and easier to reason about, a point that also shows up in guidance on keeping specs tight enough to fit the attention budget per Addy Osmani's spec-writing guide.
4. Shorten the completion bound. Ask for a narrower output shape, or cap the response.
5. Switch the model tier for the cheap steps. A retrieval or classification step rarely needs the most capable model, and OpenAI's agent guide recommends establishing a baseline with the best model then swapping in smaller ones where results hold per OpenAI.

Set the budget from the value of the work, not from the estimate. A check that resolves a support ticket is worth more than one that drafts a comment. Cost per resolved task ties spend to outcomes and is a better budget unit than raw token totals per Glean's metrics.

One more decision rule that saves arguments: estimate the whole trajectory, not the first call. Agent loops can retry inside a trace, and the expensive step may sit deep in a run while the top-level view looks normal as Braintrust warns. Multiply your per-call estimate by your expected number of calls, and add a retry allowance for the steps that fail. If you already log usage per call, the same structured usage data that feeds retrospective dashboards can feed the calibration set, so the two views stay consistent per Braintrust.

If you want the estimate and the run to share a billing surface, running the check through an OpenAI-compatible gateway means one price list and one usage record across model types, which simplifies the calibration loop rather than adding a second source of truth. Quanta Inference (https://quantaintelligence.ai/inference) is one option for that, with prepaid pricing in US dollars.