Costing a Self-Review Gate: Tokens, Minutes and Review Time

Token cost looks trivial until you count what a reviewer actually reads. A review pass needs the diff, the files it touches, the project conventions, and often the test files. A change touching a dozen files can easily push six figures of input tokens into the prompt, and providers segment long-context requests onto higher rates, which one pricing guide calls "a pricing event, not just a longer prompt" Silicon Data.

On the output side, reasoning models bill their internal thinking at the output rate. One developer cost breakdown cited by a pricing analysis describes a five-dollar estimate turning into a twenty-dollar charge because reasoning tokens were counted as output Amnic. For a review agent that reads broadly and reasons per finding, output can dwarf input.

A worked example. Take a mid-sized pull request:

The input bill is under forty cents; the output bill is a quarter. The rates above are illustrative and match the shape of published short-context pricing OpenAI pricing; substitute your own provider's published numbers and re-run the arithmetic. The point is not the exact figure but the ratio: output and reasoning are where token spend concentrates, so trimming output verbosity or capping findings per pass moves the needle more than shaving input.

Two levers compress this bucket without harming review quality. Prompt caching discounts repeated context such as convention files and architecture notes, and deterministic static analysis removes mechanical checks (unused imports, formatting, obvious dead code) from the model's job entirely. One analysis of token cost in AI coding workflows argues the cheapest token is the one you never send, because deterministic tools handle the mechanical slice at near-zero marginal cost Codacy. That is the right sequencing: linters first, model second.

Every self-review pass that runs tests also burns machine time. That time has a price, and it is the bucket teams forget.

The math is the same whether you rent runners or own them:

command_cost = command_minutes * loaded_minute_rate

command_minutes is the sum of minutes across every command the pass triggers: lint, type-check, unit tests, and any integration tests it pulls in. loaded_minute_rate is what a minute of that machine actually costs, including the runner's amortized price, idle overhead and any hosted-CI markup. If you run on a hosted CI service, the service's per-minute rate is a defensible proxy.

The lever here is not the rate, it is the minutes. Smart test selection and parallel execution can cut build time substantially: Harness reports AI-powered test intelligence reducing test cycles by up to 80 percent Harness. Even a fraction of that matters, because the pass may run tests several times as the author iterates.

Most of those minutes are avoidable. The classic mistake is running the full suite on every push. A self-review gate should run a fast slice: lint and format checks (seconds), type checks (seconds to a minute), and only the tests the diff touches. The broader suite can wait for the human-triggered pipeline. Pre-commit hooks are the right model here: they run the cheap, mechanical checks before code review so the reviewer is not distracted by trivia pre-commit. The same principle applies to the gate: keep its command footprint small and targeted.

A second decision rule for this bucket: cap command minutes per pass. Set a ceiling (say, fifteen minutes across all triggered commands), and have the pass fail closed to "needs human review" when it exceeds the ceiling. That protects you from a pathological test selection that re-runs a slow integration suite five times.

This is the bucket that decides whether the gate makes economic sense, and the one most teams guess at. Measure it, do not estimate it.

The field study is straightforward. Pick a sample of pull requests (fifty is enough to start). Record the reviewer's wall-clock minutes per PR before the gate exists. Turn the gate on and record again, splitting your sample by whether the pass changed the reviewer's behavior. Then:

net_human_minutes = mean(review_minutes_with_gate)
                  - mean(review_minutes_without_gate)

Attach your loaded reviewer rate (fully loaded cost per minute, including benefits and overhead). A senior reviewer's loaded rate is high, which is why even a small saving per PR can pay for a cheap pass. The published pattern matters here: one guide describes a ten-developer team spending four to seven hours per day of senior time on routine review, roughly half of senior capacity, and puts LLM review at ten to fifty cents per PR Understanding Data. If that range holds for your team, a pass that saves even five reviewer minutes per PR clears its own cost comfortably.

There is a second, harder-to-quantify saving: round trips. An automated pass that flags a missing null check or an unvalidated parameter before the human sees it removes a review cycle, not just a few minutes. The same source describes external contributors cycling three to seven days from first submission to merge because each round of feedback waits on a human Understanding Data. Cutting one round trip can save hours of calendar time, even though the human minutes saved are small.

Now the honest part: the gate can add minutes. Diff-only reviewers are noisy, and a team that loses trust in the tool within a week is a common failure mode, with the usual cause being eighty percent trivial comments drowning the twenty percent that catch real bugs dev.to. Noisy feedback makes net_human_minutes positive, and a positive net_human_minutes has to be paid for out of quality gains alone, which is a much harder case to make. Configuring severity thresholds and suppressing low-value feedback is the precondition for this bucket turning negative.

Put the three buckets together for a single pull request and see the decision fall out.

Assumptions, all stated so you can swap them:

• Model: 180,000 input tokens at $2.00 per million, 25,000 output tokens at $10.00 per million (shape of published rates OpenAI pricing).
• Commands: 8 minutes of lint, type-check and targeted tests at $0.10 per minute.
• Human: loaded reviewer rate of $1.50 per minute (a plausible fully loaded figure for a senior engineer; recompute with your own payroll numbers).
• Measured effect: the gate removes 6 minutes of reviewer time per PR on average.

The pass saves about $7.59 per PR, and the token and command costs are rounding errors next to the human bucket. The decision rule writes itself: run the gate only where the saved human minutes exceed the pass cost, and here the saved minutes (six) clear the fixed costs (a little over a dollar) by more than an order of magnitude.

Now flip one variable. If your measured effect is a loss of two reviewer minutes per PR (noisy feedback, reviewers reading every comment), net_human_minutes is +2, adding $3.00. Total pass cost becomes $0.61 + $0.80 + $3.00 = $4.41 per PR, and now you are paying for the gate. The same gate is a bargain on a clean codebase and a waste on a noisy one, and the only thing that changed is the measurement.

At scale, the difference compounds. Ten PRs a day at $7.59 saved is roughly $1,900 a month (based on a 21-workday month), against $4.41 spent is roughly $925 a month of net cost. Same team, same tool, opposite sign.

Write the rule down and let it run automatically. The gate should compute its own economics from logged data, not from a meeting.

def should_run_gate(pr, history):
    est = estimate_pass_cost(pr, history)
    saved = estimate_saved_review_minutes(pr, history)
    if est.total > saved * REVIEWER_MINUTE_RATE:
        return "route to human, skip pass"
    return "run pass"

Three guard conditions keep the ledger honest:

1. Cap tokens per PR. Refuse to run when estimated input plus output exceeds your ceiling (for example, a change touching hundreds of files). Large diffs are exactly where token spend explodes, and also where reviewers most benefit, so decide the ceiling from data rather than intuition.
2. Cap command minutes per PR. Fail closed to human review when the triggered commands exceed a time budget.
3. Re-measure quarterly. Reviewer rates, model prices and the pass's own accuracy all move. A rule that paid off last quarter can invert.

Logging is the whole game. Track per PR: input and output tokens, command minutes by stage, the presence or absence of the gate, reviewer start and end timestamps, and number of review cycles. With that data you can compute every term in the formula from observed reality rather than assumption. If you are running several models behind the gate (a cheap classifier for routing, a stronger model for deep review), pointing them all at a single inference endpoint keeps cost attribution in one bill instead of several; Quanta Inference exposes text and embedding models through one OpenAI-compatible API key with prepaid US-dollar pricing (https://quantaintelligence.ai/inference).

One more comparison worth making before you commit: the human-only baseline. Some teams assume the choice is between expensive human review and cheap automation. That is false. The real comparison is between (a) human review of uncleaned code, (b) human review of code already cleaned by cheap deterministic hooks, and (c) human review of code cleaned by hooks plus a model pass. Pre-commit hooks are the cheapest layer and should exist regardless of whether you add the model pass, because they remove style noise from the reviewer's attention without any model spend pre-commit. The model pass is a marginal improvement on top of that, and its budget has to be judged against layer (b), not layer (a).

The same reasoning applies to how the pass is built. A pass that runs on every push to main, on every branch, and on every draft PR multiplies its own cost by the number of events rather than the number of changes that actually get reviewed. Scope it to pull requests that are marked ready for review, cache the convention context across passes, and skip the model entirely when the deterministic checks fail, since a failing lint is not a reason to spend tokens on a review.

Finally, remember the purpose of the gate. It exists to protect the human gate, not to replace it. That means its success metric is not "how many comments did it post" but "how many reviewer minutes did it save" and "how many review cycles did it eliminate." A pass that posts nothing useful but shaves the diff down to the parts that need judgment has done its job. A pass that posts two hundred comments and saves nobody anything has not, however cheap its tokens.