Testing an Agent Fails Closed Without Its Group Claim

Fail closed means the system defaults to a blocked state when a check cannot be confirmed. Fail open means it continues as if the check had passed. The pattern is easiest to see in code. In the fail-open form, the authorization check raises only on an explicit denial and execution otherwise continues; in the fail-closed form, execution proceeds only when authorization is explicitly returned true, and everything else raises Authzed's explanation of fail open and fail closed. The second shape is what you want around a group claim.

For an agent, the decision rule is short:

If the groups claim is absent, null, or an empty array, the agent is authorized for nothing beyond actions that require no authorization at all, such as returning a structured error to its caller. No default group, no cached permission set, no legacy or debug mode.

Write that rule into the test file as a comment above the first assertion. It is the specification the tests enforce, and it makes code review of new tools trivial: anything that can act must appear in the denial matrix.

The reason to be strict is that the risk compounds once a credential is abused. Non-human identities, including agents, outnumber human identities by more than 90 to 1 in many organizations, and 92 percent of surveyed organizations said their legacy IAM could not effectively manage AI and non-human identity risk the Cloud Security Alliance Agent Identity Governance Framework. An agent that quietly inherits a default group on a missing claim is exactly the overprivileged case those numbers describe. The same framework recommends replacing standing agent privileges with intent-declared, time-bound, scope-limited grants, and that is only enforceable if the empty case denies.

A single assertion ("the agent returned 403") is not enough. An agent can return an error and still have done work. Split the test surface into four classes and assert each one independently.

The tool-invocation class matters because agents plan before they act, and a plausible-looking failure at the top level can hide a successfully completed tool call underneath. Mock the tool layer and assert on the mock's call log, not on the model's text. Guidance on detecting non-human identity abuse flags unusual tool calls and behavior outside the agent's expected pattern as signals, which only works if a baseline of "zero tool calls under denial" exists Sweet Security on non-human identity for AI agents.

The delegation class matters for multi-agent designs. If an orchestrator spawns sub-agents that carry their own tokens, a missing claim on the parent must not produce children that authenticate with a broader credential. The CSA framework notes that orchestrating agents spawn ephemeral sub-agents that each need scope definitions and tracked access, which makes the parent's denial the natural control point the CSA Agent Identity Governance Framework.

Worth testing explicitly: the "deny but retry" path. Some agents catch an authorization failure and retry with a different scope or a service account. Assert the retry budget is not spent re-authorizing with a broader credential. The failure mode is not a bad error message; it is a successful request under an identity nobody intended.

The refusal and the log are two halves of one test. A denial without evidence is indistinguishable from a crash, and during an incident review you need to show that the agent knew why it stopped and what it tried to reach.

Every denial entry should carry these fields:

Two design choices are worth calling out. First, log the claim state as a distinct enum. empty_array and absent fail for different reasons and lead to different fixes: the first usually means the user genuinely has no mapped groups, the second usually means an overage or a broken claim mapping. Microsoft's remedy for the latter is to restrict emitted groups to those assigned to the application or to apply a group filter Microsoft's group claims documentation. Second, never log the raw token or the full group list at this level. Log the claim state and the decision and keep the assertion out of logs.

There is a governance reason to invest here, not only a debugging one. Every agent action should be logged alongside its identity context, which is the same argument made for SCIM-managed agent identities: the record of who acted and under what entitlement is the thing that makes audit possible LoginRadius on SCIM for AI agents. If your stack emits OpenTelemetry spans, model the denial as a span event with these attributes so it joins the rest of the trace instead of living in a separate log silo.

Work in a fixture-driven harness so each case is a one-line parameter rather than a new test file. The procedure assumes the agent takes a token, extracts claims, and calls tools.

Step 1: Build four token fixtures. Same issuer, same audience, same subject, same expiry. Vary only the group claim:

A. groups = ["eng-readonly"]   (control, expect allow)
B. groups = []                 (empty array)
C. no groups key at all        (absent)
D. groups = null               (explicit null)

Add a fifth for overage if your provider models it: a token carrying only the overage indicator and no groups array. Microsoft documents the overage case as the token carrying an indication instead of the claim Microsoft's Zero Trust token guidance.

Step 2: Prove the harness can pass. Run case A first and assert the tool was called. If the control does not succeed, a later denial proves nothing. This is the most common mistake in fail-closed suites: the empty-state tests go green because every test is failing.

Step 3: Run cases B, C, and D against every refusal class. For each combination, assert the refusal, the absence of side effects, and the log entry:

def test_missing_groups_denies_tool_call(agent, fixtures, tool_mock, log_capture):
    result = agent.run(token=fixtures.groups_empty, task="look up account 4471")

    # 1. the refusal
    assert result.status == "denied"
    assert result.error.code == "authorization_failed"

    # 2. no side effects
    assert tool_mock.call_count == 0
    assert tool_mock.last_call is None

    # 3. the evidence
    entry = log_capture.single(event="authorization.decision")
    assert entry.decision == "deny"
    assert entry.reason_code == "missing_groups_claim"
    assert entry.claim_state == "empty_array"
    assert entry.action == "tool.crm.lookup"
    assert entry.trace_id == result.trace_id

    # 4. no secret material in the log
    assert fixtures.groups_empty.raw not in log_capture.raw_text

Parametrize claim_state across empty_array, absent, and null and run the same body. Keep the assertion on reason_code strict: a generic authorization_failed reason hides whether the claim was missing or the scope was wrong.

Step 4: Assert the negative on fallbacks. Configure the agent with a default group in its settings as a hostile fixture, run case C, and assert the default group was not applied. Then warm any permission cache by running case A, run case C, and assert the cached permissions were not reused. A cache keyed on subject rather than on claim content is a real bug class, and this test catches it.

Step 5: Assert the downstream boundary. Point the agent at a mock MCP server or a second mock agent and assert zero outbound requests for cases B, C, and D. If the agent retries, assert the retry budget is not spent re-authorizing with a different credential.

Step 6: Assert the error shape. The caller should receive a stable error code and a message that names the missing claim without leaking policy detail. Assert the status code (401 or 403, by your own convention) and that the body contains no group names or internal policy identifiers.

Step 7: Add the regression guard. Keep the fixtures in a shared module and fail the build when a protected action is registered without a corresponding denial test. A registry-driven check is worth more than any single test, because it stops the suite losing coverage as tools are added.

The empty-claim test is stronger when it sits inside a lifecycle test. SCIM provisioning creates, updates, and deprovisions agent identities with attributes that include authorization roles and scopes, and deprovisioning removes them from connected systems LoginRadius on SCIM provisioning. Two additions follow naturally:

• After a simulated deprovisioning, issue a token with an empty groups array and assert the agent denies. This proves revocation reaches the authorization path, not only the directory.
• After a permission change, assert the agent uses the new groups and not a cached set.

Aembit's framing is useful context: agents are non-deterministic and may take different actions on each attempt, so identity systems must enforce posture in an environment where the actor shifts context Aembit on AI agent identity. A deterministic suite around the one claim that gates everything is the cheapest way to bound that non-determinism.

One more guardrail: emergency or break-glass access should be its own explicit, logged path, not an accidental fallback for missing claims. Microsoft's emergency access guidance is instructive precisely because it keeps break-glass accounts narrow, cloud-only, and excluded from policies that would otherwise block them Microsoft's emergency access account guidance. If your agent needs an override, make it a named and audited mechanism that the test can distinguish from a bug.

Treat the empty-claim suite as the first of three authorization suites. The second covers claim manipulation: tampered signature, wrong audience, wrong issuer, expired token. The third covers scope narrowing, where groups are present but the requested action falls outside them. All three should share the fixture builder and the log assertions, so a denial looks identical to the harness no matter which check fired.

There is a timing dimension too. An assertion is valid only inside its time window, and clock skew between issuer and validator can push a valid token outside it, producing failures that look intermittent and are often reported as vague "assertion expired" errors the guide to SAML clock skew in Keycloak. That belongs in the token-manipulation suite, and it is worth keeping separate from the empty-claim suite so a clock problem never masquerades as a missing-claims result.

When the suites are ready to run against real models rather than mocks, the practical constraint becomes access to many model endpoints and consistent logging across them. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API key with prepaid US dollar pricing, which lets the denial tests run against every model your agent may use without a separate integration per provider.