AI coding

How Much Context a Batched Approval Item Needs

Antonio Evans9 min read

A reviewer's screen split into three panels.
Key points

Key points

  1. 1Every item in a batched approval queue must carry four fields: the exact field-level diff, the stated reason with a policy code, the evidence that the change does what the reason claims, and the blast radius of a wrong call.
  2. 2Items can be grouped into one approval pass only when all four fields match within tolerances you declare up front, or the batch silently hides a special case.
  3. 3The queue, not the reviewer, does the comparing: it surfaces the shared fields once, then pins any diverging item next to its neighbors so the difference is visible at a glance.
  4. 4You need explicit split rules, because a batch that cannot break itself apart turns fast approvals into an automated outcome nobody actually read.

The reviewer is the bottleneck, and the queue is what throttles them

Most writeups about approvals approach the problem from the agent's side: how to pause, how to persist state, how to resume. That is real work, but it is the easy half. The harder half starts when the item lands in a queue and a human has thirty seconds to decide whether it is safe.

Treat that queue as a service with arrivals, service times and a finite worker pool, not as a checkbox in a design document. One documented case had approval backlog reach fourteen thousand pending items within forty-eight hours against a three-person team processing roughly two hundred items an hour, with average approval latency around six and a half hours as described in this analysis of approval queues. The agent had not misbehaved. The reviewer pool was simply sized against design-doc traffic and the queue absorbed the difference.

That framing changes what "context" means. Context is not a nice-to-have that makes reviewers feel informed. It is the thing that sets service time. A reviewer holding a complete item can decide in ten to thirty seconds, which is the target for a well-built evidence pack per StackAI's HITL workflow guidance. A reviewer handed a summary and a link has to go dig, and the dig is where the queue expands.

Batching is the lever that makes this arithmetic work. If one hundred near-identical changes each take twenty seconds individually, that is thirty-three minutes of review. Shown as one shared frame plus one hundred diff rows, the same work can collapse to a couple of minutes, and the reviewer's attention goes where it belongs: the rows that do not fit. The catch is that batching multiplies whatever is wrong with your item schema. Approved in a pass, a missing blast radius is now missing one hundred times.

The four fields, and what each one must actually contain

An approval item that can survive batching carries the exact change, the stated reason, the evidence and the blast radius. Each field has a failure mode that is specific to being batched, which is why none of them can be swapped for a prose summary.

1. The exact change. Not "updated the customer record." The field-level diff: which object, which field, the old value, the new value, and the operation. A reviewer scanning a column of diffs can spot the row where region changes from us-east-1 to eu-west-1 because it breaks the pattern. A reviewer reading "updated customer record" one hundred times cannot, and that is exactly the gap a malformed item hides in.

2. The stated reason, with a policy code attached. Free text is unrankable. Attaching a machine-readable code (for example RETENTION_CLEANUP, PRICE_SYNC, SEGMENT_BACKFILL) lets the queue group by cause instead of by tool, and lets you measure later whether a given reason code correlates with incidents. The reason should also name the source of the instruction when there is one, because agentic systems merge system prompts, retrieved data and user input into one context, and conflicting signals let untrusted text steer the action as Teleport explains in its prompt injection analysis.

3. The evidence. The artifact that shows the change does what the reason claims. For a code change, the test output for the affected path. For a data change, the dry-run log or the count of affected rows. For a config change, the plan output. For a content change, the rendered preview. Evidence is what turns "trust me" into "verify this," and it is the field reviewers use to complete a decision in seconds rather than open a second tool per the evidence-pack pattern.

4. The blast radius. A named statement of what breaks if the call is wrong, including reach (how many records, customers or services), reversibility (can it be undone, and how), and detection (would you know it went wrong). This is the field that connects the item to policy. A restart in staging and a restart in the payments namespace are different actions and belong in different tiers even though the command text differs by one string as NOFire's runtime policy guide puts it.

A useful habit: require the agent to produce all four as structured fields, not as prose it composes at pause time. Structured output is what makes the batch grouping below possible, and it is what an audit trail can later be reconstructed from.

FieldWhat it must containWhat breaks when it is a summary
Exact changeObject, field, before value, after value, operation typeThe one divergent row looks identical to its neighbors
ReasonShort rationale plus a policy code, plus instruction source if anyCannot group by cause, cannot trend incidents by reason later
EvidenceTest output, dry-run log, affected-row count, rendered previewReviewer leaves the queue to verify, service time explodes
Blast radiusReach, reversibility, detection, named ownerPolicy tiering becomes guesswork, escalation has no target

Approval needs the right fields to be meaningful at all, batch or not

The reason a summary is insufficient is not aesthetic. An approval is an authorization decision, and authorization requires enough specificity that a reviewer is actually approving this invocation with these arguments, right now. OAuth scope answers whether an agent may call a category of tool at all, which is a different question from whether this specific call should run a distinction Scalekit draws clearly. An approval gate that shows only the tool name and not the arguments has quietly reverted to a scope check with a human face.

Approval should also be narrow in time and scope. Authorizing one update should not grant standing authority, and grants should expire with the task rather than outliving it as Oso recommends for least-privilege agents. In a batch, narrowness gets subtle: what you are actually granting is "these N items, as shown, matching the shared frame." That is why the queue's execution step must bind the approval-record to the request it showed, so a payload edited after approval does not ride in on the same decision.

This is also where approval staleness bites. An item can sit in a queue while the world moves: the record was edited, the deploy landed, the price changed. Approval that was correct at pause time may be wrong at resume time, and the failure most approval implementations skip is revalidating state between the two per this HITL design guide. Batches make staleness worse because they take longer to assemble and approve than single items, so the revalidation check needs to run at the batch level and per item.

Finally, write a timeout policy per action type before launch. An expired item needs a defined default: approve for internal reversible work, hold and escalate for anything that leaves your system or moves money, cancel for time-sensitive actions, and re-plan for proposals built on data that may now be stale per these timeout defaults by reversibility. A global "approve anything older than a day" rule is how a dangerous item nobody wanted to sign off on gets through automatically.

How a batched queue should lay out on the reviewer's screen

The queue's job is to make the common case cheap and the uncommon case loud. That means a three-band layout rather than a list of cards.

Band one: the shared frame, stated once. The tool, the target resource class, the reason code, the blast-radius summary and the evidence link for the group. The reviewer reads this once and it is true for the whole run, or the group should not exist.

Band two: the variation table. One row per item, showing only what differs: the identifier, the changed field, before, after, the affected-row count and a verdict column. This is where a reviewer's pattern recognition does its best work, because the normal shape of a row is established in the first few seconds and the odd row stands out visually.

Band three: the split tray. Any item that failed a group check sits below, expanded, with a note naming which check pulled it. The reviewer can approve the run and handle the outliers individually in one pass.

A worked example. Suppose one hundred records get a status field changed from pending_review to approved under reason BATCH_QA_PASS, each backed by a test-run log, all in the staging tenant, each reversible by flipping the field back. The shared frame holds. Then three rows arrive where the diff also touches owner_email, one of which points at an external domain. Those three fail the diff-shape check and land in the split tray, with the external one flagged for a second reviewer because it crosses a trust boundary. The run of ninety-seven costs one decision. The three cost real attention, which is exactly where attention is worth spending.

That layout also keeps the batch honest about approval fatigue. Constant prompts train reviewers to click through, and the fix is to start with materially consequential actions and use incident data to widen automation later, not to front-load everything as ThreatLocker's approval guidance recommends. If your queue can show ninety-seven greens per hundred items, reviewers stay calibrated to the meaning of green.

The checks that force a batch to stop and split

Grouping is a computation, not a vibe. Define the checks as code and run them whenever an item joins a group. If any check fails, the item is pulled out and reviewed on its own. A workable starter set:

  • Same tool and same operation. A PATCH and a DELETE against the same object never share a group.
  • Same target class and environment. Production and staging are separate groups by definition; so are two tenants.
  • Same reason code. Different codes mean different justifications, which means different evidence expectations.
  • Identical diff shape. The same set of fields changes in the same direction for every item. Any extra or missing field is a split.
  • Blast radius within tolerance. Affected count inside a declared band per item (not just in total), same reversibility class, same named escalation owner.
  • Evidence present and matching. Every item has its own evidence artifact, and the artifact's run identifier is unique, not one log reused across the group.
  • No policy escalation. Any item that trips a deny or escalate rule under your policy engine is removed before a human ever sees the batch.

That last check matters because instructions in a prompt are text the model reads, and a gate outside the model is a rule it cannot argue with as the runtime policy guide stresses. Batches should compose only from items the policy layer already cleared as approvable.

Here is a compact version of the grouping logic you can adapt:

def group_key(item):
    return (
        item.tool,
        item.operation,
        item.target_class,
        item.environment,
        item.reason_code,
        frozenset(item.diff_fields_changed),
    )

def splittable_reason(group):
    shapes = {frozenset(i.diff_fields_changed) for i in group}
    if len(shapes) > 1:
        return "diff shape differs"
    if len({i.evidence.run_id for i in group}) != len(group):
        return "evidence reused across items"
    counts = [i.blast_radius.affected for i in group]
    if max(counts) > 3 * max(1, min(counts)):
        return "affected count outside 3x band"
    if len({i.blast_radius.escalation_owner for i in group}) > 1:
        return "escalation owner differs"
    if any(i.policy_decision != "require_approval" for i in group):
        return "policy decision not uniform"
    return None

Two properties are worth stating explicitly. First, every check is mechanical, so a batch either holds together or breaks in a way you can explain to an auditor. Second, the split reason is shown to the reviewer, which turns a silent exception into a visible one. If you want the grouping thresholds tuned to your own traffic, an analyst who has seen your queue data is worth more than a generic default, and that is the sort of work Quanta's experts take on when a team would rather not build the instrumentation from scratch.

One more design decision: cap batch size and cap batch age. A group of five hundred items takes long enough to assemble that every item in it is more likely to be stale by approval time. Smaller groups with per-group revalidation keep approval fresh and split decisions cheap.

Measuring whether batching is actually safer

Batching feels faster, and it usually is, but the number you care about is not items per hour. It is agreement: on a sample of batched runs, how often would a reviewer, looking at one item in isolation with full context, have reached the same decision as the batch pass? Re-show a random slice of items individually and compare. When agreement drops, the grouping tolerances are too loose or the frames are hiding something.

Then plot reviewer accuracy against queue depth and time of day, because the same reviewer produces different judgments at item three and item three hundred per the queue analysis cited earlier. The point where accuracy falls to the agent's own error rate is the point at which the human gate is no longer adding safety. You cannot find that point without the plot, and you cannot build the plot without structured fields on every item.

That is the through-line: the four fields are not paperwork. They are the inputs to grouping, the inputs to your audit trail, and the inputs to every measurement that tells you whether the queue is working. If a coding agent is writing the approval UI or the policy layer, Quanta skills can be installed directly into it so the agent works from a consistent set of patterns rather than reinventing the item schema per feature.

FAQ

FAQ

Why is a summary of the change not enough for a batched approval?

Because the batch's whole value is pattern recognition across rows, and a summary erases the pattern. A reviewer can spot the one row where an unexpected field changed only if the queue shows field-level diffs; a hundred summaries that all read "updated account settings" are indistinguishable, and the odd item disappears. The summary is the frame, not the item.

How many items should one approval pass cover?

There is no universal number, but two caps do the work: a maximum item count and a maximum group age. Larger groups take longer to assemble, and every item in the group is more likely to be stale by the time it is approved. Revalidate each item's state at approval time and keep groups small enough that revalidation is cheap.

What should happen when one item in a batch fails a grouping check?

It gets pulled into its own review, with the failed check named in plain language on the review screen. The remaining items can still be approved as a run. The failure should never be silent or folded into the group note, because a batch that hides its exceptions is worse than no batching at all.

Do batched approvals satisfy audit requirements?

Only if the approval record captures what the reviewer actually saw, not just that a decision was made. Store the shared frame, the per-item diff, the evidence artifact identifiers, the policy snapshot and the request hash, and bind the execution step to that record so an edited payload cannot reuse the decision as the Temporal approval pattern structures it.

What happens to items nobody approves?

That is a policy decision per action type, made in writing before launch. Internal reversible work might default to approve, anything that leaves your system or moves money should hold and escalate, and time-sensitive actions should cancel rather than fire late. A blanket auto-approve after a fixed delay is the one default to avoid, because it reliably passes the item nobody was comfortable approving per these timeout defaults.

Sources

Sources

  1. The Approval Queue That Became Your Critical Path
  2. Human-in-the-Loop AI Agents: How to Design Approval ...
  3. How to Prevent Prompt Injection in AI Agents
  4. How to implement runtime policy enforcement for AI agents
  5. Human-in-the-Loop Tool Calling: Approval Gates for AI ...
  6. How to Prevent Over-Permissioned Agents
  7. Human-in-the-Loop AI Agents: Approval Workflows & Best ...
  8. Approval Timeouts for AI Agents: Defaults by Reversibility
  9. What should require human approval when using AI agents?
  10. Approval Pattern | Temporal Documentation
About the author

About the author

Antonio Evans
Antonio Evans

Antonio Evans is the founder of Quanta Intelligence and a fractional CTO who builds HIPAA-ready AI systems for health and life-science companies. Before that he was technical co-founder of FriendsEAT for twenty years, CEO of Socialdraft, and a data engineer at IBM. Antonio lives with his dog Santino in Lisbon, Portugal.

All articles