When a Router Picks a Tool the Agent Cannot Call

Walk the failed call through its lifecycle. A planner or router classifies the request, picks a tool, and your orchestrator dispatches it. The executor checks the agent's credentials, calls the upstream API, and gets a 403 with an insufficient-scope error. That response goes back into the conversation as a tool result, and the model is asked to continue.

Each of those steps has a price. The cost is dominated by what gets resent, not by the size of the refusal. A typical agent context after a few turns includes a system prompt, a tool catalog, conversation history, and prior tool outputs. If that context is 6,000 tokens and the tool catalog is another 2,000, then every retry round trip sends roughly 8,000 input tokens, plus a few hundred output tokens for the reasoning and the refused tool call. At a low input price this looks trivial per call. It compounds quickly across thousands of requests.

Tool-call failures are also routine in production, not exceptional. One production write-up reports that individual tool calls fail 3 to 15 percent of the time in well-engineered systems due to timeouts, rate limits, and upstream interruptions, and that at 99 percent per-step reliability a 10-step workflow succeeds only about 90.4 percent of the time (Scalekit). Authority failures are worse than that baseline because they do not resolve on their own: the same credential will keep being refused.

Here is a worked example of the naive path. Suppose a support agent routes 20,000 tool requests a day, and 2 percent pick a tool the agent is not scoped for. That is 400 authority misroutes. Assume a naive retry loop that makes 3 attempts, escalating the final attempt to a larger, more expensive model. Assume 8,000 input tokens and 400 output tokens per attempt.

The escalation step is where the bill gets ugly. Research on agentic routing quantifies what the authors call token inflation, the ratio of true workflow cost to single-call cost. They measured inflation as high as 4.25 times for a 7B model on multi-hop question answering, and separately found that forwarding a failed reasoning chain to a stronger model reduces that model's accuracy by up to 34.8 percentage points (Fu et al., arXiv). The second finding matters here: an escalation that carries the failed attempt in context is not just expensive, it can be counterproductive, because the bigger model inherits the small model's framing of the problem.

Put a price on the daily total above. At an illustrative one dollar per million input tokens and three dollars per million output tokens, the 9.6 million wasted input tokens are about $9.60 and the 480,000 output tokens about $1.44, so roughly $11 a day, or about $330 a month, for zero value delivered. Replace those numbers with your own contracted rates; the multiplication is the point. If you run the retry loop four times a day across a bigger fleet, the wasted spend scales linearly with retry count while the failure rate does not move.

Not every 401 or 403 means "you may not call this tool." The same status code can arise from three different causes, and treating them identically is how retry storms begin. The production guide from Scalekit is explicit that a 401 from an expired access token and a 401 from a revoked refresh token look identical at the execution layer but need completely different responses.

That empty-body row deserves stress. In the Scalekit example, a Salesforce query ran against a delegated OAuth token that a user had revoked, the connector returned an empty result set without surfacing the auth failure, and the agent treated empty context as valid state and continued routing tickets. No exception was raised. From every observability surface the team had, the run looked successful. Authority failures do not always announce themselves.

A policy or capability signal reads better than an HTTP status. The NHI Management Group's FAQ on diagnosing agent tool call failures lays out the diagnostic logic: if the inputs are malformed, it is a prompt or orchestration problem; if the inputs are valid but the provider returns an explicit error, expect authorization, quota, or a similar upstream control; if request and response both look clean but the answer is empty or wrong, suspect the tool's contract or data source. A valid, well-formed tool call that is refused is almost always authorization.

The most expensive misroute is the one that fails on authority, and the cheapest way to prevent it is to make it impossible. Filter the tool catalog per agent, per task, before the router scores anything. An unauthorized tool should never be a candidate, so the router cannot pick it even by accident.

This is a server-side decision, not a prompt instruction. IBM's tool calling overview makes the control boundary explicit: the model proposes a tool call, and the application owns which tools are available, which arguments are allowed, whether the user has permission, and how errors and retries are handled. A practical comparison of function calling and structured outputs puts the same point in the loop: validate the structure, apply business rules and authorization, then execute and verify, and note that valid JSON alone does not prove success.

Here is the pattern in code. The filter runs where the tools are built, not where the prompt is written.

from dataclasses import dataclass

@dataclass(frozen=True)
class Tool:
    name: str
    required_scope: str

AUTH_ERRORS = {"insufficient_scope", "revoked", "unauthorized_agent"}

def filter_catalog(catalog: list[Tool], granted_scopes: set[str]) -> list[Tool]:
    """Only tools the agent can actually call reach the router."""
    return [t for t in catalog if t.required_scope in granted_scopes]

def handle_tool_result(result: dict, attempt: int) -> str:
    """Return 'retry', 'stop' or 'escalate_human'."""
    if result.get("ok"):
        return "done"
    code = result.get("error_code")
    if code in AUTH_ERRORS:
        # No retry, no bigger model. The grant will not change.
        return "escalate_human"
    if code == "rate_limited" and attempt < 3:
        return "retry"          # transient: back off and retry
    if code == "upstream_5xx" and attempt < 2:
        return "retry"          # transient: retry once
    return "stop"

The second function encodes the decision rule. Authorization errors map to escalation or a clear user-facing message, never to a retry. Rate limits and transient upstream failures map to bounded retries with backoff, because those are the failures where the same request can succeed. Keeping those classes apart is the difference between a retry policy and a retry storm; the Scalekit failure taxonomy splits identity and auth, connector and proxy, upstream API, and execution semantics precisely because each layer needs its own recovery path.

For multi-turn state, this filtering works best when the tool surface is explicit for the duration of a task. The Model Context Protocol models a task as a first-class authorization boundary: a task carries which agent executes, which user initiated it, which tools are in scope, and a lifetime, and permissions are deleted when the task completes (Auth0). MCP itself is versioned and negotiates per request, with servers rejecting unsupported requests with a versioned error rather than a generic failure (MCP versioning docs), which is a useful model for how an executor should report a scoped refusal.

If you are building a catalog that spans many models and modalities, the filter is only as good as the tool registry behind it. Quanta Inference gives you one OpenAI-compatible API across text, image, video, audio and embedding models, so the router tier and the executor tier can share a single key and a single billing surface while your permission filter stays in your own code.

The instinct when a small model picks an unauthorized tool is to escalate to a bigger model. That instinct is right about the model and wrong about the payload. Sending the whole failed chain to the stronger model gives it the small model's failed reasoning to build on, and the measured cost is large: the inflation-aware routing study found that forwarding a failed reasoning chain to a stronger model reduces that model's accuracy by up to 34.8 percentage points (Fu et al., arXiv). Their response was a fresh-escalation policy that discards the failed chain before routing to a stronger model.

Authority failures make this even cleaner. If the failure is a scope denial, escalating at all is a waste, because a bigger model with the same grant will produce the same well-formed call and receive the same 403. The correct move is to stop, surface a clear message, and either ask the user to grant the scope or route to a different path that the current identity is permitted to take. Reserve escalation for fit failures, where a stronger model genuinely changes the outcome, and send it a fresh prompt.

A four-stage router illustrates where the cost goes: measure token inflation across model tiers and task types, predict the retries a query will need from a pre-execution difficulty signal, score candidates by expected accuracy divided by predicted true cost, and escalate with a fresh prompt only when the chosen model inflates beyond prediction. Even in that harness, inflation for a 7B model on multi-hop QA reached 4.25 times while the frontier model sat at 1.31 times, which is the gap that authority-blind routing hides.

You cannot fix what you cannot see, and authority failures are quiet by default. The minimum viable log record for a tool call is the request, the permission context, and the response path, so an operator can tell whether the fault was prompt shape, policy, or tool chain without guessing (NHI Management Group).

Log these fields on every tool call:

• The tool name the model selected, and whether it was in the filtered catalog for this agent.
• The scope the tool requires and the scopes the agent's token actually carried.
• The token ID, so an expired or revoked credential is distinguishable from a missing scope.
• The call_id returned by the model, so the refusal is tied to a specific call in the trace, since every tool result should reference the originating call (OpenAI).
• The result class: success, transient failure, authorization failure, or silent empty.

Then read two metrics weekly. Misroute rate on authority is the share of routed calls that fail because of a missing scope or a filtered tool; target it at zero, because the filter should prevent it. Retry spend on authority is the tokens spent on retries that were introduced by an authorization failure; it is pure waste. Both go to zero as the catalog filter tightens, and neither shows up in a latency dashboard.

For teams running many agents with different grants, the operational load is in keeping scope grants and tool catalogs in sync as agents gain and lose capabilities. That is the kind of plumbing Quanta Experts addresses by packaging a working agent that reads your context and hands back finished deliverables, rather than making every team rebuild the permission filter from scratch.

Run this against a staging copy of the catalog before adding tools to production.

The order matters. Filtering first means the router never sees a candidate it cannot use, and every other check is a second line of defense. Retry discipline comes second because it controls cost. Logging comes third because it controls how fast you find the leaks you missed.