Business

Agent Tool Permissions: A Scope Matrix Your Team Can Enforce

Antonio Evans11 min read

A wall chart sized like a real operations matrix: rows labeled with agent roles (Research Assistant, Support Triager, Billing Reconciler, Code Reviewer), columns showing Tool, OAuth Scope, Credential.
Key points

Key points

  1. 1A scope matrix has one row per agent role and one column per tool, OAuth scope, credential, data store and approval gate, with each cell marked allowed, denied or allowed with approval.
  2. 2Build it by pulling the concrete scopes and keys out of each vendor's questionnaire answers, then reduce every scope to the narrowest one that still does the job.
  3. 3Mark every write, spend or external-send action as requiring a named human approver, and encode that rule in the system rather than in a document.
  4. 4Enforce the matrix with a separate service identity per role and a policy check before each tool call, so the table becomes a runtime control instead of a PDF.

Why the questionnaire is not a control

A vendor security questionnaire produces claims. It asks what data a product touches, how long prompts are kept, whether your inputs train a model, and which third-party model sits in the request path the Reco AI questionnaire. That is the right starting point, but an approved questionnaire answers a question about the product as it behaved at signature. An agent that holds a live OAuth token to your CRM makes a fresh decision every time it runs.

The gap matters most where agents act rather than answer. A vendor can pass a standard assessment honestly and still pose a materially different risk, because AI products plan and execute multi-step actions on connected systems, and because the assessed product gets swapped and reconfigured after signature the Reco AI questionnaire. Asking which OAuth scopes a product requests into core systems is now one of the qualifying criteria for whether a vendor needs AI-specific diligence at all the Reco AI questionnaire.

The evidence for treating this as an operating problem is blunt. In one survey, 13 percent of organizations reported breaches involving AI models or applications, and 97 percent of those reported lacking proper AI access controls IBM's Cost of a Data Breach Report as cited by Atlas Systems. The same write-up records that 63 percent of breached organizations either had no AI governance policy or were still developing one Atlas Systems.

None of those figures come from a questionnaire. They come from what agents did with permissions they already had. The scope matrix is the artifact that closes the distance between "the vendor said read-only" and "the token can actually write."

Inventory the agent roles and their business functions first

Before you can scope anything, you need a list of who is acting. Start with the business function, not the technology. A useful first pass looks like this:

  • Support triage agent: reads tickets, drafts replies, never sends.
  • Sales research agent: reads CRM records and public web pages, writes notes, never changes deal stages.
  • Claims or case review agent: reads case files, drafts determinations, never approves.
  • Coding agent: reads a repository, opens pull requests, never merges to production.
  • Finance reconciliation agent: reads ledger and bank data, proposes matches, never posts a journal entry.
  • Marketing content agent: reads brand assets, drafts copy, never publishes to a public site.

Two rules make this inventory usable. First, one row per role, where a role is defined by the job it does rather than by the model or vendor behind it. If two different assistants both draft support replies, they are one role with one set of permissions. Second, name the human or process each role acts for, because attribution is the first thing an auditor asks about PlainID's permission control framework. An agent running under a single service account called something like svc-claims-ai shows forty thousand database queries and no way to tell which adjuster was behind any of them PlainID.

The policy layer should already have told you which of these roles is allowed to exist. A workplace AI policy needs to state what employees may and may not use AI for, whether they need permission, and which tools are banned, and it should name the responsible officer for violations the Brightmine AI policy guide. Treat that policy as the authority the matrix implements. Where the two disagree, the policy wins and the matrix gets fixed.

Extract concrete scopes from the vendor answers

Questionnaire answers are useful only if they resolve to named permissions. "Least privilege" and "read-only access" are not scopes. Go back through each vendor response and pull out the specific strings and credential types, and push back where an answer stays abstract.

A practical extraction checklist, modeled on the permissions and logging sections of a structured vendor questionnaire the Wavect EU AI vendor security questionnaire:

What you need from the vendorWhat a usable answer looks likeRed flag
Exact OAuth scopes requestedNamed scope strings per integration, with read and write separated"We request standard access"
Credential typeOAuth delegated token, service account key, or personal API keyShared API key across customers
Token lifetime and refreshShort-lived access token, refresh flow documentedTokens that never expire
Data stores touchedNamed systems and fields, including logs and embeddings"We need the whole record"
Action limitsWhich actions the product can take without a humanNo stated autonomy limit
Audit loggingPer-action records with agent and human identityGateway logs only

Three questions in that list do most of the work. Ask which data the product sends to a model and whether it is the vendor's model or a third party's, ask how long prompts, outputs and embeddings are retained and whether you can set retention to zero, and ask whether you can exclude specific data classes such as personal data, health data or source code from AI processing the Reco AI questionnaire. An answer of "we can't specify" or a third party they will not name is a stop, not a maybe the Reco AI questionnaire.

For health-adjacent data, the bar is higher and the answers must be contractual. What data the vendor touches, how long they keep it, whether they train on it, which controls protect it, who their subprocessors are, and who acts if something goes wrong should all appear in writing, alongside a current SOC 2 Type II report and breach terms with deadlines Censinet's six questions. Their rule is worth quoting into your own process: paper beats promises, and if the vendor will not put these points into the contract, do not proceed Censinet. Be aware that a cloud provider can be a business associate even when it cannot view encrypted health data, so the fourth-party chain counts too Censinet.

Narrow every scope to least privilege

Least privilege means granting the minimum permission needed for the function and nothing more Auth0 on OAuth and least privilege. In OAuth terms, scopes are the mechanism that expresses that boundary, and they are defined by the resource server rather than by the standard, so your narrowing work happens per API Auth0.

Start by refusing the request that asks for nothing specific. Requesting an access token without a scope parameter is a security anti-pattern and a direct violation of least privilege, because authorization servers handle that case inconsistently Auth0. For agent workflows, the narrowest useful scope reduces what a stolen token or a misused integration can reach, which is the whole point when a single integration can control many resources at once the NHI Mgmt Group glossary.

Scope inflation is the most common failure, and it happens for a boring reason: it is easier to build once against a broad permission than to design carefully NHI Mgmt Group. A workflow that works with a broad token hides the fact that the same job could have been done with a smaller, safer permission set NHI Mgmt Group. Review scopes as part of design, not as cleanup before launch.

A worked narrowing example for a support triage agent, which is typical of what teams find:

Requested at pilotNarrowed for the roleReason
Full mailbox read/writeRead messages and read threadsThe agent drafts, humans send
Send mail on behalf of usersDeniedExternal send needs a named approver
Read and write customer recordsRead records, limited fieldsTriage does not change customer data
Access to all labels and foldersOne triage inbox plus labelsLimits blast radius
Long-lived refresh tokenShort-lived token, scoped refreshLimits replay window
Unrestricted search APIRead-only searchBlocks write paths entirely

Notice that three of the six changes cost the agent nothing in capability. The draft still drafts. Only the actions that could cause irreversible harm were removed.

Map the result onto the other side of your estate as well. If your internal controls already mandate multi-factor authentication and least privilege for staff, your vendor requirements should require equivalent controls for vendor access into your environment CyberProof on third-party risk. Aligning first-party and third-party expectations stops a vendor-only loophole from opening up CyberProof.

Mark write, spend and external-send actions for human approval

The decision rule that makes a matrix usable is simple enough to apply in a meeting: if the action is irreversible outside your systems, or moves money, or reaches a person you do not control, it needs a named human approver. Reads and drafts can run unattended. Sends, posts, merges, deletes, payments and contract commitments cannot.

Practitioners who work on agent security put the same boundary in operational terms. Tool permissions decide what the agent can read, write, send, spend, deploy, delete or execute, so start read-only and add write access slowly the Kingy agent security guide. Coding agents should run in branches, disposable workspaces or sandboxes, and never get a daily machine, production secrets and auto-merge on day one Kingy. Financial, payment, payroll, tax and banking tools need hard limits: test mode, spending caps, approvals, audit logs and separate accounts Kingy. If you cannot explain what the agent can read, write, spend or send, do not connect it yet Kingy.

Your own AI policy is the place this rule gets teeth. An employee AI policy should explicitly prohibit using AI to make legal or financial decisions, commit the company contractually, or replace professional judgment, while allowing drafting and summarization under conditions the CRI employee AI policy guidance. The same prohibitions apply to an agent acting on an employee's behalf. An agent cannot do what the policy forbids the person to do.

Approval gates also need a shape. Each gated action should record four things before it proceeds: the agent role, the human who approved, the exact action and target, and the policy version that allowed it. That is what turns an audit question into a lookup instead of a two-week reconstruction across prompt logs, tool logs and database logs that share no identifier PlainID. Logging a model like an agent over-collects, but logging an agent like a model misses the action and the chain of reasoning that produced it Collibra on AI audit trails.

Enforce the matrix at runtime

A matrix in a spreadsheet is a plan. Enforcement needs two mechanisms: one identity per role, and a check before every tool call.

Per-role service identities. Give each role its own credential, scoped to exactly what the row allows. Never let two roles share a token, and never let a role borrow a human's personal token. The pattern that gets teams in trouble is the borrowed service-account model: one credential granted a role broad enough to avoid errors during testing, left attached long after the pilot ends PlainID. Where a role acts for a specific employee, record that delegation so the log keeps the human in the chain PlainID.

A pre-call policy check. Before the agent invokes any tool, a small authorization service looks up the (role, tool, action, target) tuple in the matrix and returns allow, deny or require approval. This inverts the usual model: instead of granting access and trusting logs to explain it later, you make a decision at the moment of action and record it PlainID. A minimal check, expressed as configuration rather than a specific vendor's product, looks like this:

role: support_triage
tool: email
actions:
  read_messages: allow
  send_message:
    decision: require_approval
    approver: support_lead
    ttl_minutes: 30
  delete_message: deny
retention: 30d
logging:
  record: [role, delegated_user, action, target, policy_version, decision]

A few properties of that block matter more than its syntax. The gate carries an approver role rather than a name, so it survives staff changes. The approval expires, so a stale yes cannot be replayed tomorrow. The decision and the policy version are logged together, which is what an examiner looks for PlainID.

Sandboxing the blast radius. Permissions limit what an agent may do; isolation limits what happens when it does the wrong thing anyway. Agents make mistakes and can be manipulated by hostile content, and browser agents are especially exposed because a malicious page can carry hidden instructions Kingy. Treat isolation as the second layer, not a substitute for scoping.

Review on a cadence, not on demand. Standing privilege accumulates because roles granted for a pilot stay attached after it ends, and agents make that worse because their behavior shifts with each prompt PlainID. A quarterly certification designed for human employees cannot keep pace with agents a platform team can create in an afternoon PlainID. Review each matrix row when the tool changes, when the vendor changes a model, or on a fixed interval, whichever comes first. Model changes and their notification terms belong in the vendor questionnaire for exactly this reason Wavect.

Re-classify existing vendors. A tool approved as a project tracker can ship AI features later, and re-classification is one of the most commonly skipped steps in AI risk programs Reco AI. Put the re-check on the renewal calendar, since that is when you have leverage.

Common failure modes and how the matrix prevents them

FailureWhat it looks likeMatrix fix
Scope inflationAgent built against admin access "to avoid errors"Every cell starts at the narrowest scope and widens only with a recorded reason
Standing privilegePilot credential still attached after launchRow review tied to renewal and model-change events
Shared identityOne service account for several rolesOne credential per role, delegation logged
Invisible actionsOnly gateway logs existPre-call decision recorded with policy version
Questionnaire driftApproved once, never revisitedPermissions section re-collected at each renewal
Policy mismatchAgent does what the handbook forbids staff to doPolicy prohibitions copied into the deny column

The last row is the one teams miss. If the handbook bars employees from uploading confidential material to public tools and bars AI from committing the company contractually CRI, then an agent holding a personal API key to an external model is a policy breach with a technical implementation. Handbooks are being updated for exactly this reason, and access and use controls belong in them Morgan Lewis on AI and employee handbooks.

Where this lands for a working team

The matrix is small enough to maintain and specific enough to enforce. Most organizations find between five and fifteen roles that matter, and a page or two of cells covers them. The work is in the extraction: getting named scopes out of vendors, writing down the deny column, and wiring the pre-call check into your agent runtime.

Two practical starting moves. Pick your highest-blast-radius role, the one touching money or customer records, and build its row first, including one approval gate you can demonstrate end to end. Then schedule a review of every existing AI vendor against the permissions questions in a structured questionnaire Wavect, because the answers you collected at signature have likely drifted since.

If your agents route through several model providers and you want the identity and scoping work to sit behind one interface rather than one per provider, Quanta Inference puts text, image, video, audio and embedding models behind a single OpenAI-compatible API key, which keeps the credential surface you have to scope and audit smaller.

FAQ

FAQ

What exactly goes in a cell of the scope matrix?

A decision, not a description. Each cell records allowed, denied or allowed with approval, and for any approval cell it names the approver role and the expiry window. If a scope is not represented in a cell, treat it as denied by default, because that is how the runtime check will behave.

Should I use OAuth scopes or role-based access control?

Use both, at different layers. Scopes constrain what a token can reach at the API, while roles describe which set of tools a job function needs Auth0. The matrix joins them: the row is the role, the cell names the scope or credential, and the policy check reads both before acting.

How do I handle a vendor that will not name its exact scopes?

Hold the decision. An answer that cannot specify which data goes to a model, or which third party sits in the path, is a red flag rather than a detail to negotiate later Reco AI. The same applies to subprocessors: get the named list, and if the contract will not carry the obligation, do not move forward Censinet.

What should an approval gate record?

The agent role, the delegated human, the exact action and target, the approver, and the policy version in force at the time. Without the policy version attached, a log tells you what happened but not whether it was permitted under the rules you had approved PlainID.

How often should the matrix be reviewed?

On three triggers: a fixed interval, a tool or model change, and contract renewal. Agents outpace quarterly access reviews because a platform team can create one in an afternoon PlainID, and products gain autonomous features between assessments Reco AI, so the calendar has to carry some of the load.

Sources

Sources

  1. AI Vendor Security Questionnaire: 40 Questions to Ask - Reco AI
  2. AI Vendor Risk Assessment Questionnaire for Compliance ...
  3. AI Agent Permission Control Framework Built for Audits
  4. 10 things to include in your AI policy | Brightmine
  5. EU AI Vendor Security Questionnaire: 45 Questions - Wavect.io
  6. AI Vendors Want Your Data. Here Are the Six Questions to ...
  7. OAuth 2.0 Access Tokens and The Principle of Least ...
  8. What Is Least Privilege OAuth Scope? Definition & Examples
  9. How to Manage and Mitigate Third Party Risk
  10. AI Agent Security Guide: MCP, Sandboxes and Tool ...
  11. Employee Use of AI: Why You Need a Policy and What to Include
  12. AI audit trails: What to log for models and agents, and how ...
  13. AI in the Workplace: It May Be Time to Update Your Employee Handbook
About the author

About the author

Antonio Evans
Antonio Evans

Antonio Evans is the founder of Quanta Intelligence and a fractional CTO who builds HIPAA-ready AI systems for health and life-science companies. Before that he was technical co-founder of FriendsEAT for twenty years, CEO of Socialdraft, and a data engineer at IBM. Antonio lives with his dog Santino in Lisbon, Portugal.

All articles