How to Write a Model Router That Adapts to a Price Change

Start with an explicit schema. Every model gets separate input and output rates per million tokens, because the ratio between them drives the routing decision. Here is a compact config shape:

{
  "currency": "USD",
  "per": 1_000_000,
  "models": {
    "premium": { "input": 10.00, "output": 50.00 },
    "budget":  { "input": 0.10, "output": 0.50 },
    "mid":     { "input": 2.00, "output": 10.00 }
  },
  "qualityTolerance": 0.02,
  "defaultModel": "mid"
}

The rates above are the kind of spread you actually see in published price tables, where premium tiers charge much more for output than budget tiers (the cost guide). The exact numbers matter less than the structure: input and output are separate keys, the unit is explicit, and nothing about cost lives in code.

Keep four things in config, not just prices:

• The per-model input and output rates, with the unit declared.
• The quality tolerance, which is the premium you are willing to pay for the better model on a given task.
• The default or fallback model for requests that cannot be classified.
• Per-tier budget caps if you run a gateway, since caps are also policy data.

Where the cost math lives matters too. Put the arithmetic (estimate tokens, multiply by rates, compare) in one pure function, and put the rates in a file or config service. Then the pure function never changes when a price changes, and the config never contains logic. That separation is what makes a price update a data change rather than a code change. If you want to route across many providers from one key while you stage that migration, Quanta Inference exposes text, image, video, audio and embedding models through a single OpenAI-compatible API with prepaid dollar pricing, which removes one class of rate surprises from your config.

The useful threshold is per request, because the winner depends on the input/output mix. Define the cost of model m for a request as:

cost(m) = inTok * m.input / 1e6 + outTok * m.output / 1e6

Now the decision rule. For a given request profile, the budget model wins when its cost plus the quality tolerance is below the premium model's cost:

use_budget  if  cost(budget) + tolerance <= cost(premium)
else use_premium

Solve that inequality for the ratio you actually control. If output tokens dominate, the answer is basically a comparison of output rates; if input dominates, it is a comparison of input rates. A useful shortcut is the break-even token ratio: the input/output split at which the two models cost the same. Worked example, using the rates above:

• Premium: $10 input, $50 output per million tokens.
• Budget: $0.10 input, $0.50 output per million tokens.
• With 1,000 input and 200 output tokens: premium = $0.010 + $0.010 = $0.020; budget = $0.0001 + $0.0001 = $0.0002. Budget wins by roughly 100x.
• Now invert the rates, as a hypothetical price change might: premium drops to $0.10 input and $0.50 output while budget rises to $10 input and $50 output. Same request: premium = $0.0002, budget = $0.020. The decision flips with zero code changes.

That last paragraph is the whole point of the pattern. Nothing in the router changed; the config moved, and the router produced the opposite, correct answer. Helpers can be small enough to inline:

def cost(model, in_tok, out_tok):
    return in_tok * model["input"] / 1e6 + out_tok * model["output"] / 1e6

def choose(cfg, in_tok, out_tok):
    p = cfg["models"]["premium"]
    b = cfg["models"]["budget"]
    if cost(b, in_tok, out_tok) + cfg["qualityTolerance"] <= cost(p, in_tok, out_tok):
        return "budget"
    return "premium"

Two practical refinements. First, token counts are estimates, so use a tokenizer rather than word counts; the same text produces different token counts depending on model and encoding, and English text runs about 1 token per 4 characters as a rough guide (OpenAI's token guide). Second, if you want to reason about whole-model economics rather than single requests, there is a related break-even calculus: private hosting versus an API typically breaks even in the millions of tokens per day range across modeled scenarios (the self-hosting study). That is a different decision from per-request routing, but it shares the same discipline of making thresholds explicit.

The router's job is to turn a request into a token estimate, run the comparison, and record the decision. It should never hold a model's price, a model name chosen for cost reasons, or a threshold as a literal. A clean shape:

class Router:
    def __init__(self, cfg, estimator):
        self.cfg = cfg
        self.estimator = estimator

    def route(self, request):
        in_tok, out_tok = self.estimator(request)
        choice = choose(self.cfg, in_tok, out_tok)
        return {
            "model": self.cfg["models"][choice]["name"],
            "reason": choice,
            "in_tok": in_tok,
            "out_tok": out_tok,
            "config_version": self.cfg["version"],
        }

Three design habits make this survive price churn:

• Version the config and stamp every routing decision with that version, so when spend shifts you can tie it to a specific price edit.
• Keep model identity (the string your client sends) in config too, so a price change and a model swap are the same kind of edit.
• Reload config on a signal (a file watch, an environment refresh, a feature flag) rather than on deploy, and validate the new config before swapping it in.

That last point is where routers quietly break. If you reload without validating, a typo that drops a rate to zero routes every request to that model until someone notices. Validate on load (all models present, rates positive, tolerance non-negative, currency and unit declared) and reject the config if any check fails. Gateways that centralize model selection often expose exactly this split, a tier config object that every other function reads from, so a model change is a single-file edit while the routing logic stays model-agnostic (the gateway guide).

A checklist for the config contract:

The tests that matter are the ones where prices move. Keep two layers.

Boundary tests pin the exact crossover. For a fixed config and a known token mix, assert the router picks budget just below the threshold and premium just above it. Then run the same test with a shifted config where the roles invert, and assert the router inverts with it. That single pair catches the most common regression: someone hard-codes a comparison or a model name while "fixing" something else.

Property tests cover the space you will not hand-write. Generate random token mixes, random price pairs within realistic bounds, and random tolerances, then assert invariants that must always hold:

• Monotonicity: as the budget model's output rate rises (all else fixed), the share of requests routed to it never increases.
• Symmetry: swapping the two models' price objects and swapping their labels produces the same routing decisions.
• Dominance: if the budget model is cheaper on both input and output, every request routes to it when tolerance is zero.
• Determinism: the same request and config always produce the same decision and reason.
• Total: every response names a model that exists in config.

These invariants are cheap to write and they fail loudly the moment a price edit lands with the wrong sign, a swapped key, or a stubbed rate. Pair them with a golden-file test: a batch of representative requests, a config, and the expected decision log. When a price changes intentionally, regenerate the golden file and read the diff. If the diff moves traffic the way you expected, accept it; if it moves traffic for requests that should not care, you found a bug.

One more test worth having: a config validator test that feeds malformed configs (missing rate, zero rate, negative tolerance, wrong unit, no default) and asserts each is rejected with a specific error. Routers fail more often through bad data than bad math.

Cost is one axis; quality and latency are the others, and a router that ignores them will happily send complex reasoning work to a model that cannot do it. The research framing is useful here: routing picks the model that maximizes a quality score subject to a budget constraint, so the cost comparison should be bounded by a quality requirement, not run standalone (the routing survey). Other work shows routing the easy majority of traffic to smaller models can cut blended cost sharply because most requests do not need frontier capability (the routing survey), and that routing is distinct from failover, which handles errors rather than economics (the gateway guide).

Practically, encode the quality requirement as data too: a task class in config that maps to an allowed model set and a tolerance. A classification task might allow budget and mid with a tight tolerance; a multi-step planning task might require premium regardless of price, because a cheap wrong answer costs more than a premium right one. Then the cost comparison runs only within the allowed set. That keeps price changes consequential where they should be and inert where quality is non-negotiable.

Two more guards from the routing literature and practice:

• Prefer a cheap classifier or heuristic over a model call for the routing decision itself, since a frontier model call just to decide which model to use defeats the purpose (the routing survey).
• Measure actual outcomes, because cascades that escalate only when a cheap answer fails a verifiable check often beat static choices, but only if the failure check is real (the routing survey).

If your routing spans speech, image or embeddings alongside text, one place to centralize credentials and model strings while you tune the thresholds is Quanta Inference and the consulting call, where a 30-minute session can map your token mix to a routing policy. Neither changes the design above: the config and break-even math stay yours.

A price change is a production event, so treat it like one even though no code ships.

1. Edit the config on a branch, with the new rates and a bumped version.
2. Run the full test suite, including the shifted-price fixtures, and read the golden-file diff to see which requests change destination.
3. Deploy the config as data, not as code, and let the reload path validate it before activation.
4. Watch the routing log for the share of requests per model, and compare against the predicted shift from the golden-file diff.
5. Keep the previous config version available so rollback is a pointer change.

The numbers to watch are the ones your router already logs: share of requests by model, average estimated tokens in and out, and cost per request computed at the new rates. If the predicted shift and observed shift disagree, the classifier or the token estimator is off, not the price math. That is the moment to check the estimator against real usage, because token counting depends on the encoding and on message structure, and a plain-text count does not include all tokens in an API request (OpenAI's token guide).

Finally, document the decision rule next to the config. Whoever edits prices next needs to know that the tolerance is a deliberate quality premium, that output rates drive the decision for generative tasks, and that a config with no default is rejected. The rule should read like a sentence: use the cheaper allowed model unless its cost plus the tolerance exceeds the premium model's cost for this request's token mix. That sentence is the router, and the config is just its numbers.