How to Do a Security Review of an AI Agent

Christian Schneider, who runs security architecture reviews of agentic systems, argues that component-by-component methods such as STRIDE miss agent attacks because those attacks "move the system through legitimate states until it betrays itself." The chain is not "input leads to data leak" but "input, retrieval bias, planning goal shift, tool invocation, aggregated exfiltration" (Christian Schneider). The post summarizes the failure in one line: traditional threat modeling "treats attacks as isolated events while attackers treat them as stateful campaigns."

Schneider's workflow organizes discovery around five zones:

The point of the zones is tracing. Schneider notes that attacks rarely stay in one zone: an injection enters through inputs, shifts the plan, triggers tools, and may persist in memory or spread to other agents. For each high-value asset, walk the path backwards and ask which input could set it in motion.

The OWASP GenAI Security Project publishes a Top 10 for Agentic Applications, which it describes as a peer-reviewed framework developed with more than 100 experts, researchers and practitioners (OWASP agentic Top 10). Its ten risks make a good checklist. The summaries and mitigations below come from the field guide written by members of the OWASP Agentic Security Initiative core team (OWASP ASI field guide).

For each row, record whether the risk applies, what control exists, and how you verified it. "The prompt tells it not to" is not a control.

Most serious findings in agent reviews come down to access. Okta defines least privilege for agents as limiting access "to only the resources and actions explicitly required to complete its current task, not its general purpose, its potential future tasks, or everything it might conceivably need" (Okta). It also warns of a common pattern: agents provisioned with broad credentials during development that are never reduced once the workload is stable.

Check these points:

1. Each agent has its own identity. Okta recommends a unique identity per agent, separate from human and shared service accounts, recorded in a central registry with owner, purpose, data scope and creation date. Shared credentials make attribution difficult and widen the blast radius.
2. Credentials are short-lived and scoped to the task. Okta notes that static, long-lived credentials give far more access than any single task requires. The OWASP field guide recommends task-scoped tokens per agent or workflow.
3. Actions run in the user's context. When an agent acts for one user, it should not use an account that can reach every user's data.
4. Authorization is enforced outside the model. Permission checks belong in code and in downstream systems, never in the prompt.

Then go one step further. Zenity argues that least privilege "isn't enough," because an agent can stay within its permissions and still behave in ways inappropriate for its purpose. It proposes least agency, which asks "what autonomous actions should this agent be permitted to take, in what sequence, under what conditions, and subject to what oversight?" (Zenity). Zenity also warns about scope creep: an integration added months after launch without a security review leaves the agent with permissions nobody formally authorized. In practice, review the action list as carefully as the access list: an agent that can read every invoice may be acceptable, while an agent that can read every invoice and send email without approval is not.

Tools turn model output into real effects. For each tool, ask what the worst call would be with arbitrary arguments, then constrain it.

• Prefer narrow tools. "Send a refund under a set amount to this order's customer" is reviewable. "Call any payment API" is not.
• Validate arguments in code. The OWASP field guide recommends an intent gate that validates the tool name, arguments, schema and rate limits before any call runs (OWASP ASI field guide).
• Sandbox execution. The Frontier Model Forum notes that deterministic infrastructure controls such as sandboxing, filesystem scope and network egress policy determine what an agent can reach and whether data can leave the system (Frontier Model Forum).
• Treat generated code as untrusted. Run it non-root, with limited network and a confined file system, and separate the component that writes code from the one that executes it, as the OWASP guide recommends.
• Vet third-party tools. The Frontier Model Forum describes developers routing third-party integrations through directories of vetted tools. It also points out that a compromised tool is both a code execution threat and a prompt injection vector, because tool output flows straight into the model's context.

Check egress last. If the agent can make arbitrary outbound requests, any injection that reaches private data has a way to send it out. An allow-list of destinations is one of the cheapest and strongest controls in the whole review.

Memory. The OWASP field guide's example of memory poisoning is forged policy documents uploaded to a shared store that loosen refund rules, so the agent keeps approving bad refunds for weeks (OWASP ASI field guide). Check who can write to memory and retrieval stores, whether writes are scanned, whether memory is segmented by user and tenant, and whether every entry records its source.

Other agents. If agents message each other, check that channels are authenticated, messages are signed, approvals cannot be replayed and messages follow strict schemas. Also check delegation: the field guide describes a finance agent with executive-level scopes passing its token to a generic database agent, which a manipulated prompt then uses to reach HR and legal data.

Human approvals. The Frontier Model Forum calls human oversight a core safeguard and recommends confirmation before high-stakes or irreversible actions (Frontier Model Forum). It also notes the limit: in multi-agent systems, reviewing every output becomes impractical and causes fatigue. Review where approvals sit and whether they are meaningful. The OWASP guide's trust exploitation example is a poisoned invoice that leads the agent to recommend an urgent payment to a "verified" account, with a confident explanation and a large Approve button. An approval screen should show the exact action, its source and its risk, not just the agent's summary.

A design review finds what the reviewer can imagine. Adversarial testing finds the rest. Microsoft's documentation describes AI red teaming as probing for content and security risks specific to generative AI, and recommends automated scans at the design, development and pre-deployment stages, plus scheduled red teaming runs after deployment (Microsoft Learn). It scores results as an attack success rate per technique and risk category, which gives a number to track across releases.

A practical test plan for a small team:

1. Plant instructions in every input zone, including documents, emails, retrieved pages and tool output, and check whether the agent's plan changes.
2. Attempt exfiltration through every output channel: links, images, emails, file writes and tool arguments.
3. Attempt privilege escalation by asking the agent to act for another user or tenant.
4. Poison memory in a test environment and check whether later sessions are affected.
5. Test approvals by checking whether a risky action can complete without the human step, for example through a different tool.
6. Re-run on every change to the model, prompt, tool list or permissions.

Microsoft notes that automated probing complements manual red teaming rather than replacing it. Automated scans cover known attack patterns at scale, and people find the chained, business-specific attacks.

Record the review so the next person can trust it. For each finding, write the attack path across zones, the asset at risk, the control added and the test that proves it works. Schneider's workflow ends by turning attack paths into requirements that an auditor can check, using a published mapping from the OWASP agentic risks to the AIUC-1 standard for AI agents (Christian Schneider). Even without a formal standard, the same discipline helps: every risk should end with a control and evidence.

Agents change faster than most software. Treat any of these as a trigger for a fresh review:

• A different model or model version
• A changed system prompt or planning logic
• A tool added, or a tool's permissions widened
• An added input source, such as a mailbox or a shared drive
• Another agent joining a multi-agent workflow

Teams that publish agents on a platform such as Quanta Intelligence can apply the same checklist to each agent's tools and data access before it goes live.