Sampling Approvals: How Often to Review Agent Output by Task Risk

Two inputs decide the tier.

Cost of a bad output. Not the cost of the task, the cost of the error. A bad draft email costs a minute. A bad refund costs the amount plus the dispute plus the auditor's question. The Aembit team frames the same split as read-only intelligence work versus write actions and external communications, where autonomy for the second category should be earned rather than assumed in its guardrail writeup. Salesforce makes the parallel point that the question shifts from "did it say the right thing?" to "did it do the right thing?" once the agent can call tools and update records in its guardrail guide.

Volume. A low-volume action that is expensive to reverse deserves full review even at two actions a day, because reviewer cost is trivial. A high-volume action with a cheap error should be sampled, because full review at that volume becomes either a rubber stamp or a backlog. The worst combination is high volume plus expensive errors plus full review: reviewers approve hundreds of items in a row without reading them. That looks like a control and produces no signal.

The StackAI approval guide draws a similar line, requiring approval when the next action is irreversible, costly, regulated or high blast radius, and lists payment initiation, access provisioning and production database writes among its examples in its decision framework. The table above turns that list into rungs you can assign a task type to on a Monday morning.

Two rules keep tiers honest. A task inherits the tier of its worst consequence, not its typical one: an agent that usually writes internal notes but occasionally emails a customer is tier 3. And tier changes require a recorded decision, never a drift. If the sample rate is 10 percent and the backlog is growing, that is a signal to raise automation after evidence, not to quietly stop looking.

The common starting point of 10 percent sounds reassuring and often is not. What matters is what the sample can detect.

If the true failure rate is p and you review n random actions, the chance you see at least one failure is 1 minus (1 minus p)^n. At n equal to 10 percent of 500 daily actions, you review 50 items. A 5 percent failure rate gives roughly a 92 percent chance of catching at least one failure that day. A 1 percent rate gives roughly 39 percent. A 0.2 percent rate gives roughly 9 percent. A 10 percent sample on modest daily volume is a fair detector of widespread breakage and a poor detector of rare, serious breakage. Rare and serious belongs in full review, not sampling.

For a population proportion, the required sample size at a chosen confidence level and margin of error is n equal to z squared times p times (1 minus p) over the margin of error squared, rounded up as shown in this worked example. A statistics text notes the same tradeoff directly: a smaller sample is cheaper and less accurate in this chapter on sample size. At 95 percent confidence, z is 1.96, and assume p equals 0.5 until you have a baseline, because that maximizes the required size.

Run the numbers. To estimate a failure rate within plus or minus 5 percentage points at 95 percent confidence with no prior estimate, you need 385 reviewed actions. Within plus or minus 3 points, roughly 1,067. That sets your cadence: at 200 actions a day and a 10 percent sample, 385 reviews is a two to three week rolling window, not a daily check.

import math

def reviews_needed(confidence_z=1.96, margin=0.05, assumed_rate=0.5):
    n = (confidence_z ** 2) * assumed_rate * (1 - assumed_rate) / (margin ** 2)
    return math.ceil(n)

def chance_of_seeing_a_failure(p, n):
    return 1 - (1 - p) ** n

print(reviews_needed(margin=0.05))   # 385
print(reviews_needed(margin=0.03))   # 1067
print(round(chance_of_seeing_a_failure(0.01, 50), 2))  # 0.39

The decision rule: for tier 1, sample whichever is larger, 10 percent of daily volume or the number that gives a 5 point margin across your review cadence. For tier 2, target a 3 point margin. For tier 3 and above, review everything, then sample the reviews to check reviewer quality.

A sampling policy is an economic decision, so write it as one. Compare the daily cost of reviewing against the daily expected cost of the failures review would catch.

Worked example. A tier 2 task runs 1,000 actions a day. A reviewer handles 60 items an hour at about 40 dollars an hour loaded, so a review costs about 0.67 dollars. Suppose one bad output costs 300 dollars to fix and the failure rate is 3 percent: 30 failures a day, 9,000 dollars of expected damage. Full review costs about 667 dollars a day and catches most of it, but only if reviewers read carefully. A 20 percent sample costs about 133 dollars, about 200 reviews, enough to estimate the rate within a couple of points and catch any failure mode occurring in more than roughly 1.5 percent of actions. Sampling wins by a factor of seven. Change the failure cost to 5,000 dollars and full review is trivially worth it, with dual approval for the worst subset cheaper than one incident.

The rule that falls out: automate when the expected cost of caught failures exceeds review cost by the margin you need; review everything when a single failure's cost exceeds a week of review cost. In between, sample and watch the trend.

This is where the approval log earns its keep. MLflow's review queues describe the practical version: capture each interaction as a trace, route flagged traces into a shared queue, have a human answer a pass or fail question, and accumulate judgments into a dataset of real failure modes that can later train an automated judge in the review queue writeup. Guidance on agent evaluation makes the same point from the other side, stressing that human review and automated evals should share one instrumentation path so the traces you review are the traces your graders see in the Confident AI guide. Sampling is how you generate that labeled data at a defensible cost.

A sampling policy fails quietly if the review is thin. A reviewer shown a one-line summary will approve almost everything. The approval should arrive with the proposed action, its arguments, the context that produced it and, for anything with a side effect, the likely blast radius.

The security framing helps. Guardrails for agents are described as enforceable technical boundaries covering what an agent can access, which tools it may run, and when actions should be blocked or escalated, with dynamic authorization evaluating each tool call at the moment of execution per this guardrail overview. Review is one of those boundaries and needs the same rigor as an access rule.

The audit side is where most teams are weakest. An LLM trace tells you what the model said; an authorization record tells you whether the action was permitted, for which agent, on whose behalf, under which policy, with who approved it and when as argued in this audit trail guide. That matters when a reviewer approves a hundred items a day: months later you must reconstruct that a specific person approved a specific action at a specific time. A Dataiku survey of data leaders found that only 5 percent say AI output is traceable 100 percent of the time, a fair warning that the log is usually thinner than the team believes per the Dataiku governance writeup.

If you cannot fill a row for an action, it was not reviewed in any meaningful sense, even if someone clicked approve.

There are a few established ways to pause an agent for a human.

Interrupt and resume pauses the run at a defined point, hands the proposed action and context to a reviewer, and continues after a decision. It is the strongest control and the easiest to explain to an auditor, and it is what frameworks expose as approval-required tools through approval wrappers or per-tool interrupt configuration in Deep Agents. The cost is latency and state management, since a paused run must be persisted and resumed.

Human as a tool exposes the approval as a call the agent makes, so the agent decides when to ask. It is flexible for demos, but the agent's judgment becomes part of the control, a weak place to put it. Early discussion in the Pydantic AI project made this point: prompting inside a tool call works when you are happy for the run to hang, while returning structured data for a human to confirm before executing is more durable as discussed in this issue thread.

Exception-only review lets mature, low-risk workflows run and routes only flagged items to a person, batched so reviewers see many small items at once. It is the right default for tier 1 and tier 2 once you have a baseline, and it keeps the human step from becoming a bottleneck, since delays usually come from routing and missing context rather than the review itself per the StackAI guide.

Fallback escalation routes to a person when confidence is low or a rule fires, the natural companion to sampling: the sample is baseline measurement, escalation is the safety net.

A production build uses at least two. Approval gates on tier 3 and 4, exception-only review with sampling on tier 1 and 2, and a shared review queue so human judgments feed back into evaluation. If agents run across several providers, logging and routing are easier when calls come through one endpoint; Quanta Inference puts text, image, audio and embedding models behind one OpenAI-compatible key, keeping the trace format consistent whichever model produced the action.

A sampling policy is only real if it has a cadence and a trigger to change.

Weekly: read the sample, record the observed failure rate and review count, compare with last week. Check approval time too: reviewers averaging two seconds per item are clicking, not reviewing.

Monthly: re-check tier assignments. A task whose volume tripled, whose scope widened, or whose failure rate rose by more than a point should be re-tiered, usually upward. A task at or below its target rate for several consecutive months with no missed failures in the sample can move down one tier, with the decision recorded.

On any incident: an incident from an auto-approved action is an immediate downgrade until you can explain why the sample missed it. An incident a reviewer approved is a signal to improve the evidence pack, not necessarily the sample rate.

Writing this down prevents two failure modes: review theater, where everything is "reviewed" and nothing is checked, and silent autonomy, where the review mode changed one config commit at a time and nobody can say when. Both show up in the same place: the ratio of logged actions to reviewed actions, by task type, over time. If that ratio moves without a recorded decision, the policy is fiction.

If you would rather have someone run this with you than design it from scratch, the Quanta team builds and reviews agent systems and hands back working software; describe the workflow and scope on the main Quanta page and a specialist picks it up.