A risk review typically ends with a list: here are the tools the agent can call, here is what each one touches, here is what looked dangerous. That list is an inventory, not a policy. The next step is deciding, per action, whether a person must sign off before it runs and what the fallback is when nobody responds.
Two framing mistakes cause most of the trouble. The first is treating human oversight as a switch at the level of the agent: this agent is supervised, that one is not. The second is assuming a well-written system prompt is a control. Neither survives contact with production.
The more useful framing is per action. As one governance write-up puts it, the same support agent might draft replies with no approval, require a human before issuing a refund over a threshold, and be blocked entirely from closing an account: one agent, three different oversight rules (Arthur AI). The unit of decision is the tool call with its arguments, not the agent.
There is a reason the decision has to sit outside the model. LLMs do not reliably separate instructions from data, so any content the agent processes is a candidate instruction (OWASP LLM01). A prompt that says "never delete production data" is a suggestion to a system that reads attacker-controlled text. The approval gate has to be enforced by the component that actually executes the call.



