How to Write an Escalation Rule for Model-Human Disagreement

The threshold is the amount of disagreement you treat as normal. Below it, the model wins by default. Above it, a human decides.

Do not guess this number. Measure it. Run the model's summary pass against a set of checkpoints where a human has already recorded ground truth, then compute the spread of the differences. Set the threshold at roughly the largest difference you are willing to explain as noise.

Take a latency field as a worked example. The model reads smoke.api.p95_latency_ms from the run telemetry and reports 412. The human reads the same telemetry and reports 690. The absolute difference is 278 ms. If your measured noise band on that field is plus or minus 50 ms, the disagreement is 5.6 times the band, well outside tolerance, and the run is blocked pending a decision.

For a rate field the logic is the same, but the noise is larger and depends on sample size, so read the baseline rate and the minimum detectable effect before you set a band. The three inputs to any sample size calculation are the baseline rate, the minimum detectable effect, and the desired power, and a low baseline needs far more observations before a difference means anything (Atticus Li). One published example: a page converting at 2 percent needs about 80,700 visitors per variation to detect a 10 percent relative lift at 95 percent confidence and 80 percent power, and only about 21,000 per variation to detect a 20 percent lift (Mida). If your smoke test has 40 sessions, a three point swing in a rate field is noise, not a disagreement, and the threshold should say so.

Cro Metrics makes the same structural point: sample size depends on traffic, conversion rate, and confidence level, and no single number fits every test (Cro Metrics). Thresholds are per field for the same reason.

The money row is deliberate. A cent level mismatch on a checkout total is either a rounding bug or a wrong expectation, and both deserve a stop.

Name a role, not a team. "Engineering decides" is an escalation rule that fails the first time two engineers disagree, which is exactly the moment the rule exists for.

The role needs four properties, all of which appear in guidance on where humans should check AI work: the evidence to judge, clear criteria, enough time, and the power to reject or escalate (The Metalearner). A reviewer without rejection authority is decoration.

Write the rule with a primary and a fallback:

escalation_rule:
  id: smoke-checkout-total
  checkpoint_field: smoke.checkout.order_total_cents
  tolerance_abs: 0
  primary_owner: release-engineer-on-call
  fallback_owner: payments-tech-lead
  decision_deadline: 60m
  default_if_no_decision: hold_run

Two choices in that snippet carry most of the weight.

The first is default_if_no_decision. Silence must resolve to one state, and for a blocking field the safe default is to hold. If your process instead resolves silence to "run anyway," the rule is decorative and the model wins by attrition.

The second is the fallback. Escalation needs a destination and a default decided before the case arrives, not after (The Metalearner). If the primary owner is asleep, the run either waits or moves to the fallback, and the rule should say which.

Keep the owner list short. A rule that names the whole team has named nobody.

Every conflict resolves one of two ways. Write both branches explicitly, including what gets written down.

Model wins

Model wins when the absolute difference between the model's value and the human's value is less than or equal to the threshold. The model's value stands as the checkpoint result. The human's value is attached to the checkpoint record as a comment, with reviewer identity and a timestamp. The run proceeds.

The human note is not nothing. If the same reviewer files the same kind of comment three runs in a row, the threshold is probably wrong, and the threshold gets reviewed.

Model wins requires no evidence beyond the recorded values, because nothing is being overridden.

Human wins

Human wins when the difference exceeds the threshold and the named owner signs off on the human's value. This is an overturn, so it needs evidence from both sides:

The model's side matters more than it looks. LLM summaries frequently describe intent rather than behavior. A study using mutation analysis found summary accuracy fell from 76.5 percent on single functions to 17.3 percent on multi-threaded systems, and on a set of 50 human-written programs the summaries were accurate only 49.3 percent of the time because models described algorithmic intent rather than actual mutated behavior (arXiv). A model summary asserting that a test passed because the code "should" return a value is not evidence. The evidence is the value read from the artifact.

The same paper found the gap is not permanent and not closed: newer models scored 85.3 percent on the same task and still struggled to separate implementation details from standard patterns (arXiv). Translation for the rule: a strong model will win most low-threshold comparisons, and the threshold is what keeps the remaining cases from being settled by confidence alone.

Summaries also introduce facts. One review reports that 25 percent of CNN/Daily Mail summaries from traditional LLMs contained hallucinations, and that 71 percent of named entities in LLM summaries fell outside the source document's scope (newline). That is the argument for making the model cite the exact log line or telemetry query it read: an entity the model introduced cannot be traced to a source, and a value that cannot be traced loses by default.

Invert the evidence burden deliberately. The human should not have to prove the model wrong. The model should have to prove the value it wrote is traceable, and if it is not, the human wins automatically.

The tie case

If the difference exceeds the threshold and no owner signs off before the deadline, the checkpoint is marked unresolved. The run does not start. The incident is logged against the checkpoint ID, and the next review of the rule starts with the question of why the owner was unavailable.

There is a statistical twin of this problem that trips teams constantly. When someone checks a run repeatedly and stops when the result looks significant, the false positive rate inflates: with a z-test constructed for a 5 percent false positive rate, checking after the first two observation pairs pushes the true rate closer to 10 percent, and it keeps climbing with repeated looks (Spotify Engineering).

That is the statistical version of the disagreement your rule exists to settle. A human who reads checkpoint data mid-run and forms an opinion has effectively peeked, and the model's summary of a partial run has the same defect. Two guardrails:

First, freeze the checkpoint. Compare the model's value and the human's value from a single snapshot, captured at the same moment, with a run ID on both. A rule that compares a model summary of the full run against a human's reading of a partial run will escalate every time and teach everyone to ignore it.

Second, if you genuinely want continuous monitoring, use a method built for it. Sequential testing changes the confidence interval so the false positive bound holds no matter how many times the metric is evaluated (Statsig). Group sequential designs, alpha-spending, and always-valid methods each control error differently, and the choice depends on whether you can fix the number and timing of looks in advance (MetricGate). In a smoke run, the honest version is usually to look once, at a defined checkpoint, and escalate from that snapshot.

A rule that lives in a wiki is not a rule. Two artifacts make it real.

The first is a machine-readable definition stored next to the smoke suite, as in the YAML above, version controlled, and loaded by the pipeline that decides whether to start the next run.

The second is a decision log. Every escalation writes a row: checkpoint ID, field, model value, human value, difference, threshold, outcome, owner, and timestamp. That log is how you tune thresholds with data instead of opinion, and it is the same record you need for governance. Model governance covers controlling access to models, tracking interactions with results, and documenting the basis of decisions, and MLOps pipelines without it stall before production (ml-ops.org).

A minimal log write, in a shape a pipeline can emit:

jq -n \
  --arg field "smoke.checkout.order_total_cents" \
  --argjson model 4599 \
  --argjson human 4899 \
  --arg outcome "escalated" \
  '{field:$field, model_value:$model, human_value:$human,
    abs_diff:($model-$human|fabs), threshold:0, outcome:$outcome}' \
  >> escalations.jsonl

Run that across your checkpoint fields and you have the tuning dataset for next quarter's thresholds.

One more audit point: review thresholds on a schedule, not during an incident. Tuning a threshold mid-run is peeking by another name, and it turns the threshold into whatever value avoids an argument today. When an escalation does resolve and the run proceeds, record the owner and the resolution. The alternative, a run that starts because nobody objected, is how a rule quietly dies.