Pricing a Judge's False Pass and False Fail

The core move is to stop asking "is the judge accurate?" and start asking "which mistake is more expensive, and by how much?"

False pass. The judge says pass, the output is bad, and it ships. The cost is whatever badness costs you in that workflow: a refund, a re-run of a downstream job, an incident, a customer who leaves. Estimate it as a per-incident figure and multiply by incident frequency if you have that history. If you do not, use the most defensible internal number you can and write down its basis.

False fail. The judge says fail, the output is fine, and a person rechecks it. The cost is reviewer minutes times the loaded hourly rate, plus any delay. This one is usually easy to measure: time a reviewer on a sample of 20 items, divide, multiply.

There is a multiplier worth knowing before you set the false pass figure. In a pipeline where every layer is validated by a model with its own error rate, errors compound: a 5% error rate validated by another 5% judge gives roughly 0.95 x 0.95, an error rate of about 9.75% (GovTech AI Practice). If your judge sits inside a chain of automated checks, charge the false pass cost accordingly.

The other reason to price these separately rather than reach for one accuracy number: the choice of reported metric can move a headline accuracy figure from 0.551 to 0.899 on the same verdicts purely through protocol choices (arXiv:2606.00093). A metric that sensitive cannot drive a spending decision. A dollar figure can.

Here is the arithmetic, with a worked example. Suppose:

• False pass cost: $40 per bad output shipped (support time plus a partial refund).
• False fail cost: $6 per good output rechecked (10 reviewer minutes at $36 an hour).
• The underlying data: 20% of outputs are genuinely bad.

A judge at a given threshold produces two rates: the share of all items that are false passes (bad items it waved through) and the share that are false fails (good items it rejected). At a threshold where the judge catches three quarters of bad outputs and rejects 5% of good ones:

Now move the threshold in the stricter direction: catch 95% of bad outputs, but reject 18% of good ones.

Stricter wins here, by $976 per thousand. Loosen it further in the other direction and the answer flips: catch 50% of bad outputs, reject only 1% of good ones, and you get 100 false passes at $4,000 against 8 false fails at $48, a total of $4,048. So the shape of the curve is what matters: false pass spend falls as you tighten, false fail spend rises, and the minimum sits where the two lines cross. That crossing point is your threshold.

The formula you can put in a sheet:

expected_cost_per_1000 =
    (false_pass_rate * 1000 * cost_false_pass)
  + (false_fail_rate * 1000 * cost_false_fail)

Note how much the answer is driven by the ratio of the two costs. At $40 versus $6, the ratio is about 6.7 to 1, so the optimum tolerates roughly six or seven wasted reviews to prevent one bad ship. Halve the false pass cost to $20 and the optimum moves back toward the middle. This is why the short answer says to leave the threshold fixed until one of the dollar figures changes: a threshold is only optimal relative to a cost ratio.

A threshold written into a prompt gets overwritten by the next person who tunes the rubric. A threshold written into a sheet with its inputs survives. Keep one page per judge with five blocks.

Block 1: scope. Judge name, what it scores, the unit of judgment (single response, whole turn, or full session), and the positive class you are detecting. Arize's alignment guide recommends pinning these four things, plus which error is worse, before choosing any metric (Arize).

Block 2: the frozen judge. Model identity, full prompt, examples, output schema, decoding settings, and the rubric version string. Also record the handling rule for invalid outputs, since a judge that returns nothing must land in a cell of the matrix rather than vanish (OneUptime).

Block 3: the two dollar figures, each with the date it was set, the person who set it, and the basis. "Customer support ticket average $18, three tickets per bad ship, source: support dashboard export" beats "$40" alone.

Block 4: the threshold table. One row per candidate threshold with the four raw cells, the two rates, the two spends, and the total. Pick the minimum and bold it.

Block 5: the sampling note. How many annotated items the rates came from. A minimum viable annotation set is 15 to 20 per error code, and below about 10 the confidence interval on kappa is wider than plus or minus 0.30, which makes the number statistically meaningless (aws-samples). If your false pass rate rests on four observed bad items, say so and treat the threshold as provisional.

If your evaluation set is not representative of traffic, the rates in block 4 are wrong no matter how good the arithmetic is. Stratified sampling, splitting data by risk tier or usage segment so each group is fairly represented, keeps precision and recall honest for minority groups that would otherwise be swamped (Statsig). Rate your false passes by stratum if you can, because a bad ship in a high-risk segment may cost far more than $40.

Teams often frame this as a binary choice. The cost model makes it a three-way comparison, and the middle option usually wins.

The lesson is in the last two rows. In this example, reviewing the 144 rejects costs $864, less than the $6,000 to review everything, and the total is still less than a loose threshold. The judge is not replacing humans; it is routing to them.

Neither judges nor humans are unbiased, and the biases are systematic rather than random: judges show self-preference, verbosity bias, and sensitivity to prompt wording, while human annotators drift in interpretation across a long day (GovTech AI Practice). A hybrid with targeted human calibration is what practitioners describe as the reliable setup, and treating evaluation as ongoing engineering rather than a one-time configuration is the difference between teams that trust their numbers and teams that abandon them (Splunk).

For a fuller picture of where human labels come from, and why ground truth is itself a construct, the human-to-judge alignment workflow (measure human-to-human agreement first, then compare the judge against the same reference on the same items) is the right frame (Arize).

Write down the triggers now, while the sheet is fresh, so nobody moves the threshold on a hunch.

Move it when a dollar figure changes. A refund policy changes, review rates change, or a new high-risk segment enters the traffic mix. Recompute; do not re-tune the prompt as a shortcut.

Move it when the judge changes. A new model version, a new rubric version, or a new few-shot example set is a different judge. Re-run the calibration set and rebuild block 4.

Move it when the confusion pattern drifts without the judge changing. Production data shifts. Monitoring for drift and deciding between recalibration and retraining is the standard practice in classification systems, and the same logic applies to judges (Adnan Masood). The article is paywalled beyond the summary, but the decision list in the preview is the useful part: thresholding tied to cost, tier-specific thresholds, and a high-recall filter feeding a high-precision verifier.

Do not move it because the reported accuracy moved. As the agreement-metrics analysis shows, protocol choices alone can swing reported accuracy without changing a single verdict (arXiv:2606.00093). If the number moved and the verdicts did not, the sheet did not change either.

If you are running judges across several models or several vendors, the practical friction is usually plumbing rather than statistics: one key, one schema, consistent logging of verdicts and costs. Quanta Inference puts text, image, video, audio and embedding models behind a single OpenAI-compatible API with prepaid dollar pricing, which keeps the per-item cost side of the sheet in one currency (https://quantaintelligence.ai/inference).

Pricing the false pass at zero because "the judge will catch it." It will not; that is what a false pass means. If a downstream human catches it, the cost is that catch, not zero.

Pricing the false fail as only the judge call. The judge call is cheap and roughly constant across thresholds. The false fail cost is dominated by reviewer time, which is what actually moves with the threshold.

Forgetting the base rate. If only 2% of outputs are genuinely bad, even a low false pass rate produces few bad ships, and the false fail side dominates the total. Run the sheet at your real prevalence, not at 50/50.

Treating the threshold as a quality setting. It is a spending setting. A lower threshold does not make the judge better; it changes which error you pay for.

Ignoring the number of ratings per item. If you plan to compare two judge configurations and claim one is better, one rating per item is often not enough to make that claim reliably; research on rating counts finds that even 5 to 10 responses per item can be insufficient for significance testing on common metrics (arXiv). That does not block a cost calculation, but it should stop you from declaring a threshold "significantly better" on thin data.

Skipping the adjudication step. If two reviewers disagree on an item, that disagreement is signal about your rubric, not noise to be averaged away. Preserve ties and low-consensus cases rather than assigning an arbitrary label (Arize).

One last framing point from the broader evaluation literature: percent agreement tells you whether raters matched, while a chance-adjusted metric tells you whether the match is meaningful enough to support a decision, and the right choice depends on the decision you are supporting (NHI Mgmt Group). Your decision is a spending decision. That is why the sheet's output is dollars and a threshold, with kappa reported beside it as a health check rather than a target.