Which Repairs Need a Human: A Policy for Meaning Changes

Write the rule as a single gate with four pass conditions. A repair may be auto-accepted only if all four hold:

1. The change is reversible without information loss.
2. The information needed to reverse it is present in the same record.
3. The change does not touch any field consumed by a downstream calculation.
4. The change cannot alter which entity, time, or category the record refers to.

If any condition fails, the repair goes to the approval queue. In practice that yields a short auto-accept list and a longer review list, which is the correct shape: automation should be the narrow path and review the wide one, not the reverse.

The table is the whole policy in one screen, and it is worth pinning next to the pipeline code so that adding a new repair rule forces an explicit choice about which column it belongs in.

Unit change with a stated source unit. A supplier feed sends {"weight": 120, "unit": "lb"} and the warehouse schema stores kilograms. The repair multiplies by 0.45359237 and writes 54.43, retaining the original and its unit in an audit column. Nothing about the claim changed; only its representation did. This is the textbook case for reversible conversion, and unit-conversion literature shows why the source unit must travel with the number: conversion errors in scientific computing are a recognized class of defect precisely because values are interpreted in the wrong unit system Chizeck et al. on error detection and unit conversion. Auto-accept, log the conversion factor, done.

Unit change without a stated source unit. Same feed, but now the payload is {"weight": 120} and a legacy note in the vendor contract says pounds. The pipeline converts. Then the vendor changes its export, and 120 was kilograms all along. The record now asserts 54.43 kg instead of 120 kg, a 2.2x error, and nothing in the record flagged the assumption. This one belongs in the queue, with the reviewer shown the inferred unit, the evidence for it, and the age of that evidence. If the evidence is a contract clause with a date, the reviewer can see whether it still applies.

Enum swap. A ticketing system migrates statuses. The mapping table says "done" becomes "closed", and someone adds "done" becomes "resolved" because the two sound similar. They are not the same: one usually ends the workflow, the other usually keeps it visible for a follow-up window. Swapping them changes what the record asserts about the case's lifecycle, and any SLA report or backlog count computed from status will shift. A reviewer who sees the old value, the new value, the mapping rule's rationale, and the list of downstream reports that filter on status can make this call in seconds. An automated rule cannot.

The reason to insist on human review for meaning-changing repairs is that the information needed is often absent from the record itself. Vocabulary drifts: teams repurpose a status column for payment failures or fraud alerts, and the technical container stays the same while its semantic payload moves ThoughtWorks on semantic drift. The same term can mean different things in two pipelines, and the divergence is invisible to standard accuracy and data-quality metrics Dataiku on semantic drift.

Agents make this sharper because they produce plausible, well-formed output while their behavior drifts away from intent without throwing errors Trussed on semantic drift in agents. A repair agent that proposes a unit conversion has produced a technically valid suggestion. Whether it is contextually correct is a separate question, and that gap between "technically authorized" and "contextually authorized" is where approval gates earn their keep Scalekit on human-in-the-loop tool calling.

The research on command gating offers the most useful warning here. In a study of 1,709 real-world command denylists, 69.0 percent to 98.6 percent were fragile, because the same goal can be reached by many different commands that the list did not anticipate Characterizing Denylist Fragility in AI Agents. Repair rules have the same property: you can enumerate the repairs you know about, but not the ones an agent will invent. That is an argument for routing by class of change, not for trying to enumerate every bad case.

Most approval queues die from volume and missing context. A documented failure: a claims-intake agent routed every decision to human approval, the queue backed up within weeks, reviewers cleared items in seconds, and a misclassified claim was paid out with a human signature attached ReadyForRole on human-in-the-loop design. The fix is not more reviewers. It is routing by risk and giving each reviewer enough evidence to decide quickly.

The same problem has a measured version on the agent side: approval fatigue leads users to approve about 93 percent of command execution requests, which collapses the queue into an allowlist Characterizing Denylist Fragility in AI Agents. If your reviewers approve nearly everything, the gate is decorative.

Design the queue item so the decision takes ten to thirty seconds. A workable schema:

{
  "repair_id": "rp_01HX",
  "record_id": "sku_88213",
  "field": "shipping_weight_kg",
  "original": {"value": 120, "unit": null},
  "proposed": {"value": 120.0, "unit": "kg"},
  "repair_class": "unit_conversion_without_source",
  "rule_id": "units.infer_from_vendor_contract.v3",
  "evidence": [
    {"type": "contract_clause", "ref": "MSA-2021 s.7", "asserted_unit": "lb"},
    {"type": "conflict", "ref": "sku_88214", "asserted_unit": "kg"}
  ],
  "downstream": [
    {"consumer": "freight_cost_model", "impact": "high", "sensitivity": "linear"}
  ],
  "blast_radius": {"rows": 412, "distinct_suppliers": 3},
  "reviewer_role": "logistics_data_owner",
  "sla_minutes": 60
}

Three fields do the heavy lifting. evidence shows why the pipeline proposed what it did, including conflicts. downstream tells the reviewer what breaks if they are wrong. blast_radius tells them whether this is one row or four hundred. Without those three, the reviewer is asked to re-do the investigation, which is why queues that omit them get rubber-stamped.

Two queue mechanics are easy to miss. First, batch by rule, not by row: if one inference rule is wrong, the reviewer should see all 412 affected rows in one item with a single approve or reject. Second, treat every decision as training data. Capture the reviewer's reason code, and after a class of repair accumulates consistent approvals, promote it to automatic acceptance with a sampling rate for quality assurance. That is the progressive autonomy ladder, and it is the only way the queue shrinks over time ReadyForRole.

The pipeline shape is a classifier in front of every repair, with the auto path and the queue path sharing the same audit log.

MEANING_FIELDS = {"quantity", "unit_price", "tax_rate", "duration_s", "amount"}

def route(repair, record):
    if repair.field in MEANING_FIELDS:
        return "queue"
    if repair.class_ in {"unit_change", "date_shift", "enum_swap"}:
        if repair.class_ == "unit_change" and record.get("unit"):
            return "auto"          # reversible, source unit present
        return "queue"
    if repair.class_ in {"whitespace", "case", "encoding", "format"}:
        return "auto"
    return "queue"                 # unknown class defaults to review

Note the default. Anything the classifier does not recognize goes to review, because the cost of a wrong auto-repair is silent corruption and the cost of an unnecessary review is a few seconds of a reviewer's time. That asymmetry is the whole justification for the policy.

Two engineering details make the gate trustworthy. Idempotency: a repair proposal should carry a deterministic key derived from the record and the rule, so a retried repair cannot be applied twice or produce a second queue item Trussed on tool call idempotency. And structured error handling: when the repair service rejects a proposal for a schema mismatch, the proposal should be regenerated with the error in context rather than retried unchanged, because blind retries replay full context and multiply cost tianpan.co on retry budgets.

If you are building the repair agent itself, structured outputs enforce the proposal shape so the gate always receives a complete object: original, proposed, rule, evidence Agenta's guide to structured outputs. Without that, the gate spends its time repairing the repairs, which is exactly the failure pattern practitioners call tool argument rot Lanham on JSON schema.

Log every proposal, every route decision, every approval and rejection, and the reason code. That log answers the regulator question "who approved this automated change and why," which is the same evidence requirement that applies to agent actions generally Dataiku on human-in-the-loop governance. It also gives you the two numbers that matter: the auto-accept rate by repair class and the rejection rate by class.

A rejection rate near zero for a class over a few hundred items is the signal to promote it to automatic acceptance with sampling. A rejection rate above a few percent is the signal to fix the rule or widen the queue. Reviewing those two numbers monthly is a far better use of a data owner's time than clearing individual items.

If the queue is growing because you have many repair classes and no one to own the policy, that is a good moment for a short working session rather than more tooling. A free 30-minute AI discovery call can map your repair classes to routes and identify which ones are ready to automate: Quanta Consulting. And if the repair agent spans text extraction, embedding-based deduplication, and image or scanned documents, running them through one API key removes a layer of glue code from the pipeline: Quanta Inference.