Approval Fatigue: Measuring Reviewer Load and Tuning the Gate

The schema below is deliberately boring. One row per approval request, written by the approval service, with the tool name and risk class denormalized so you can group without joins.

CREATE TABLE approval_events (
  request_id      uuid PRIMARY KEY,
  tool_name       text NOT NULL,          -- e.g. 'send_customer_email'
  risk_class      text NOT NULL,          -- 'auto' | 'queue' | 'block'
  model_confidence numeric,               -- classifier score, nullable
  requester       text NOT NULL,          -- agent or workflow id
  shown_at        timestamptz NOT NULL,   -- first render to reviewer
  decided_at      timestamptz,            -- approve/reject click
  decision        text,                   -- 'approved' | 'rejected' | 'expired'
  reviewer        text,
  fields_edited   int DEFAULT 0,          -- did reviewer change the proposal?
  planted_error   boolean DEFAULT false   -- was this a seeded bad action?
);

Two design rules make this usable. First, store shown_at from the browser, because server-side timestamps on a paginated queue will understate how long an item sat unread. Second, never update rows in place: append a new event when the decision changes, so an approval that was later reversed stays visible in the history.

Then compute the three metrics as a weekly rollup per tool and per reviewer:

SELECT tool_name,
       count(*)                                                 AS requests,
       percentile_cont(0.5) WITHIN GROUP (ORDER BY decided_at - shown_at) AS median_seconds,
       avg(CASE WHEN decided_at - shown_at < interval '5 seconds' THEN 1.0 ELSE 0 END) AS fast_approval_rate,
       avg(CASE WHEN fields_edited = 0 AND decision = 'approved' THEN 1.0 ELSE 0 END) AS no_edit_approval_rate
FROM approval_events
WHERE shown_at > now() - interval '7 days'
GROUP BY tool_name;

Read the output in pairs, not in isolation. A tool with a five-second median and a 95% fast-approval rate is a tool whose gate is decorative. A tool with a 40-second median and a 30% edit rate is one where the reviewer is genuinely doing work, and it may deserve to keep its gate or even gain a stricter one. A tool with a two-minute median and almost no rejections over months is either well tuned or badly sampled, and the planted-error rate tells you which.

The common mistake is treating gate placement as a property of the tool. It is a property of the tool plus the action instance. A send_customer_email tool sending an internal status update and one sending a refund confirmation are not the same risk, so the routing decision needs both the tool's risk class and the instance's context.

The rule I recommend to teams has three steps, applied in order:

1. Reversibility first. If the action cannot be undone within your normal window (money moved, data deleted, message sent outside the organization), it never auto-approves regardless of model confidence. The orbiresearch timeout analysis draws the same line: actions that leave the system or move money "never go through on silence" see the defaults-by-reversibility pattern.
2. Blast radius second. Count how many records, users or downstream systems one call touches. A bulk update touching 40,000 rows deserves a gate even if each individual change is reversible.
3. Confidence last, and only as a tiebreaker within the reversible, small-blast-radius set.

The table encodes the same four timeout defaults (approve, escalate, cancel, re-plan) that orbiresearch recommends, chosen per action type rather than globally from the same timeout guide. The important discipline is that the person who owns the queue writes the default for each action type in advance, in writing. It is a product decision, and it is the one decision teams most often leave to whichever engineer wires up the queue.

The confidence component needs a calibrated score. If your classifier's 0.9 does not mean what it says, the routing thresholds are noise. Keep the threshold conservative at first and widen it only when the override data supports it.

An approval queue without a timeout policy rots in one of two predictable ways. Either nothing happens on timeout and the queue fills with items nobody decided until reviewers learn to ignore it, or someone gets tired of the backlog and adds a global "approve anything older than 24 hours" rule. The second failure is worse, because it automatically clears the single most dangerous item in the queue: the one nobody felt comfortable approving described as rebuilding the rubber stamp with a delay.

Temporal's approval pattern gives you the mechanics for free. A workflow blocks on a condition with a timeout, an approver submits a decision via a signal, and the timeout path escalates or rejects if no signal arrives the pattern and its timeout handling are documented with code. Microsoft's agent framework exposes the same idea at the tool level: wrap a function in an approval-required wrapper and the run returns a request object that the caller resolves with a yes or no as shown in the framework's tool approval tutorial. Cloudflare's workflow approval makes the durable wait explicit, with a timeout that can run for days without keeping an agent alive via its waitForApproval primitive.

Whichever primitive you use, the review service should record the timeout outcome as a decision. An expired item that escalated is not the same as one that was cancelled, and both need to appear in the weekly rollup so you can see whether the timeout is firing on normal days. If it is, the timeout is too short or the queue is understaffed, and reviewers will start ignoring it.

Approval fatigue is partly a volume problem and partly a presentation problem. A request that shows the raw tool call and a wall of JSON costs the reviewer more attention than one that shows the decision they are being asked to make: what will happen, to what, how many things it touches, and what happens if nobody answers.

Two lessons from tool design transfer directly. First, single-responsibility tools make gates far easier to judge. A manage_customer tool with an action parameter forces the reviewer to work out which mode is being invoked before they can assess risk, whereas a dedicated suspend_customer tool tells them exactly what is about to happen the single-responsibility argument is made with code examples here. Second, tight schemas make invalid states impossible, so the reviewer does not need to check whether the action is well-formed, only whether it is wanted the case for constrained schemas and enums.

The request payload matters too. Return structured, actionable detail rather than a bare error code, because the reviewer and the agent both need to know what failed and how to fix it the agent-facing error design pattern. And write the approval prompt the way you would write a job posting for a new hire: what the role is, when to act independently, and when to escalate, in a few sentences the reviewer can re-read quickly the job-posting framing.

A practical test: hand the request screen to someone outside the team and ask them to state, in one sentence, what will happen if they click approve. If they cannot, the screen is the problem, not the reviewer.

Here is a worked example, using invented but plausible numbers so you can substitute your own.

Start with a tool that sends customer emails. In week one it produces 2,400 requests, the median time to decision is 6 seconds, 88% of approvals happen in under five seconds, and 4% of approvals edit a field. The agent's underlying quality is fine; the gate is not. Apply two changes. First, auto-approve the subset that is internal, templated and addressed to a known contact, which is 70% of volume. Second, keep the gate for anything with a custom body or a new recipient, and put the recipient's full address and the first 200 characters of the body in the request screen.

Re-measure after two weeks. Volume on the gate drops to about 720 requests, the median time to decision rises to 35 seconds, fast approvals drop to 12%, and the edit rate rises to 18%. The gate is now doing work. That is the signal you are looking for: fewer items, slower decisions, more edits.

Do the same exercise on every tool with a gate, and you will usually find a familiar distribution. A small number of tools generate most of the approval volume, and within those, a small number of action types generate most of the risk. The tuning work is to pull the low-risk volume out of the queue and raise the cost of the remaining gates until the median decision time reflects actual reading.

Two guardrails on the process. Never tune a gate from the approval rate alone; a tool whose approval rate falls from 97% to 92% after a change may simply have started surfacing harder cases. And keep a standing sample of auto-approved actions that a person audits weekly, so pulling volume out of the queue does not mean pulling it out of oversight.

The classifier is only as good as the feedback loop. Each reviewer decision, and specifically each rejection or field edit, is a labeled example of what should have happened, and those labels are the cheapest training data you will ever get. Feed them back into the risk classifier on a schedule, and re-derive each tool's gate placement from the fresh override data.

This also gives you a clean way to decide when to buy tooling versus build it. If you are running agents in code, the approval primitive is free in LangGraph and the OpenAI Agents SDK, and either is a reasonable default for a team that already writes agents as graphs or tool lists per the comparison in this approval gate review. A durable workflow engine earns its cost when approvals wait hours or days or span systems as orbiresearch and Temporal both describe. What you should not do is buy a managed review queue and assume it solves the problem; the queue is the easy part, and the metrics above are what make it mean anything.

If you would rather not build the approval service and the routing logic yourself, Quanta Inference gives you one OpenAI-compatible endpoint for the classifiers and models behind each gate, priced in prepaid US dollars, so the risk-scoring call and the generation call share one key (https://quantaintelligence.ai/inference). That leaves your team's time for the part that actually needs judgment: deciding, per action type, where the gate sits and what happens when nobody answers.