Testing Agent Tool Permissions With Negative Cases

The most important decision is whether the tenant id is an argument the agent can set. There are three configurations, and they fail differently.

The third row is the structural defense. Removing tenant identity from the tool schema means no tool signature can express a cross-tenant read at all. In the same study, with the parameter removed, no tool signature could express the out-of-scope read, though a small number of trials escaped by forging writable scope, which shows the interface invariant depends on cryptographically protected context in the arXiv paper.

A practical rule: if the agent cannot name a tenant, you have one class of test (resource ownership). If it can, you have two (ownership plus tenant scoping), and the second one is the one attackers aim at. Keep the tenant id in the credential and out of the schema wherever your protocol allows it.

The row-level equivalent is already well understood for multi-tenant Postgres. The common pattern adds a tenant_id column to every tenant-owned table, enables RLS, writes a policy comparing that column to a transaction-local setting, and sets the setting before any tenant SQL runs in the session-context walkthrough. AWS describes the same goal from the other direction: centralize isolation enforcement so it does not depend on every SQL statement a developer writes in the AWS database blog. Your tool layer needs the same centralization, and the tests prove it exists.

Every negative case has five parts. Write them in this order and the test almost writes itself.

1. A fixture in the wrong scope. Create a real resource owned by another tenant or user. Not a random UUID that does not exist: a genuinely foreign record. Asserting "another tenant cannot" against a row that actually belongs to another tenant is the difference between a test and a decoration in the Supabase discussion on generated RLS tests.
2. An authenticated actor with a known scope. The agent's credential must map to a specific tenant and user, so the expected denial is unambiguous.
3. A tool invocation payload that names the foreign identifier. This is the attack surface you are testing, so name it explicitly.
4. An assertion that the action did not happen. Not just that an error came back. Check the foreign record is unchanged.
5. An assertion on the error shape. Status, code, and message content.

Step 4 is the one teams skip. Supabase test suites use pattern 2 from the Makerkit documentation: as the unauthorized user, assert nothing is visible, then switch to a privileged role and confirm the data actually exists, so you know the filter worked rather than the fixture being missing in the Makerkit database testing guide. Do the same for writes. Denial plus unchanged state.

A naming convention helps as the suite grows. The Basejump guide recommends XX-feature-being-tested.sql for database tests in the Basejump pgTAP guide. Mirror it for tools: tools/get_invoice.cross_tenant.test.ts, tools/update_ticket.foreign_user.test.ts.

Take a support agent with three tools: get_customer, update_ticket, and send_email. It runs for tenant t-north on behalf of user u-42. A second tenant, t-south, has customer c-900 and ticket tk-900.

The legitimate call:

{
  "tool": "update_ticket",
  "arguments": {
    "ticket_id": "tk-101",
    "status": "resolved",
    "note": "Customer confirmed the fix."
  }
}

The negative call, where the model has been steered by injected text in a customer email to name the other tenant's ticket:

{
  "tool": "update_ticket",
  "arguments": {
    "ticket_id": "tk-900",
    "status": "resolved",
    "note": "Closed on request."
  }
}

Expected result: the call fails, tk-900 keeps its original status, and the audit log records one denied attempt with the actor, the tool, and the requested resource. A wrong write in your own database can be rolled back. A wrong write in a system you do not own generally cannot: the action is external, often irreversible, and it can complete without raising an exception because the credential was valid in the Scalekit multi-tenant auth write-up.

If your tool schema still carries a tenant id, add the second case:

{
  "tool": "get_customer",
  "arguments": {
    "tenant_id": "t-south",
    "customer_id": "c-900"
  }
}

Expected result: denial, with an error that states the credential is scoped to t-north and cannot read t-south. Note the difference from the first case. Here the agent named the boundary explicitly, so the error can be precise about which boundary was crossed.

A row-level analogue, for teams porting the habit, seeds matching rows and then checks the wrong identity sees nothing:

begin;
select plan(2);
insert into invoices (tenant_id, amount) values ('t-south', 100);
set local role authenticated;
set local request.jwt.claim.sub = 'u-42';
select is_empty($$ select * from invoices where tenant_id = 't-south' $$,
  'credential scoped to t-north cannot read t-south invoices');
select is_empty($$ update invoices set amount = 0 returning 1 $$,
  'credential scoped to t-north cannot modify t-south invoices');
select * from finish();
rollback;

Supabase's own RLS test example uses exactly this structure: set the role, set the claim, assert a count or assert an update returns nothing in the Supabase testing overview. Note that with RLS, a policy mismatch usually returns zero rows rather than raising an error, which is why tests assert emptiness rather than an exception in the tenant-context overview. Tool layers should do better: fail loudly, not silently.

A denial that returns a generic 403 wastes the most valuable minutes of an incident. The error should answer four questions without anyone opening a debugger.

• Who was acting? The credential's tenant and user, by id.
• What was called? Tool name.
• What was requested? The foreign resource or tenant id from the payload.
• Why was it denied? The rule that fired, in plain words.

A shape that works:

{
  "error": {
    "code": "SCOPE_DENIED",
    "message": "update_ticket denied: credential for tenant t-north cannot modify ticket tk-900 owned by tenant t-south",
    "tool": "update_ticket",
    "actor": { "tenant_id": "t-north", "user_id": "u-42" },
    "requested": { "ticket_id": "tk-900", "owner_tenant_id": "t-south" },
    "rule": "tool_scope.ticket_owner_matches_credential_tenant"
  }
}

Requirements worth asserting in the test itself, not just writing in a doc:

1. The message mentions both the actor's scope and the requested scope. A test can check the response body contains both ids.
2. The code is stable and enumerable, so dashboards and alerts can count denials by rule.
3. No data from the foreign record leaks. The error names tk-900 because the caller asked for it, but it must not include the ticket's subject line or customer name. Leaking contents through error messages is its own disclosure path, and OWASP lists sensitive information disclosure among the risks that least-privilege scoping mitigates in the Auth0 access control overview.
4. Denials are logged as audit events. Authorization decisions should be visible in logs, and the log should be legible enough to reconstruct what happened in the WorkOS access control guide.

One nuance on ambiguity: for a resource outside your scope, "denied" and "does not exist" are often better merged, so an agent cannot probe for valid ids. That is a product decision. What is not optional is that the internal log distinguishes them, because during an incident you need to know whether the target was real.

The mixed model to avoid is validating the tenant argument and treating "validated" as the whole control. A validated parameter can still be wrong; it is validated, not authorized. Supabase makes the same point about grants and policies: a missing grant raises 42501 before any policy runs, so a failure your policy should have allowed is often a grants problem in the Supabase docs. Two layers, two distinct failure modes, two distinct error codes.

A negative test that only runs when someone remembers is a comment. Wire it in.

Supabase runs its pgTAP suites with supabase test db inside a transaction that rolls back, so tests do not need a reset between runs in the Supabase testing overview. Application-level tests cannot rely on a cleaned database, so they should generate unique ids per run and avoid collisions in the Supabase testing overview. Do the same for tool tests: a fixture tenant and user per run, tagged so the audit check can find its own events.

The inventory gate is the highest-value step and the cheapest. Keep a registry of registered tools and fail CI when a new one lands without a matching negative test file. This is the same rationale behind generating RLS tests from the policy catalog and failing the build on an unprotected table: hand-written coverage gets skipped, and a wrong policy ships as a silent leak in the Supabase discussion on rlsautotest.

Two more habits that pay off:

• Test the revoked path too. A credential revoked mid-session should fail closed on the next call, not succeed because a cache still holds the old scope. Short-lived credentials and just-in-time issuance shrink that window, but a test proves the behavior in the WorkOS guide.
• Run the negative suite against the enforcement proxy, not the tool. Authorization belongs in the execution component, outside the agent's context, and unknown tools should fail closed in the OWASP cheat sheet. Testing there means the test survives a model swap, a prompt rewrite or a framework upgrade.

If you are standing this up across several tools and want a second opinion on where the boundaries belong, a free discovery call is a reasonable place to sanity-check the design before you write the suite: Quanta Consulting. And if your agent layer fans out across many providers, text, image, audio and embedding models through one OpenAI-compatible key is one less integration to secure: Quanta Inference.

A worked number makes the prioritization concrete. Suppose you have 12 tools, of which 4 write to external systems, and you run 6 negative cases per tool. That is 72 tests. At roughly 150 ms each for mocked execution plus 2 seconds of shared setup, the suite adds under 20 seconds to CI, and the 24 cases covering external writes are the ones you write first. If any of those 24 can complete a cross-tenant write, you have a defect class with no rollback.