Tool Testing: How to Simulate Agent Calls in Staging

Replay without ground truth measures nothing. For every captured turn you need an expected outcome, and it is not always a single tool name. Use one of four labels:

• Correct tool plus required arguments. The canonical case.
• Acceptable set. Several tools are defensible (a general search_knowledge_base versus a narrower lookup_policy_by_id). Store a set, score any member as correct.
• No tool. The right answer is a clarifying question or a direct answer. These cases matter because agents over-call tools as often as they under-call them.
• Must fail safely. Requests that should be refused, escalated, or answered with an error the user can act on.

Producing these labels is the expensive part. Three techniques reduce the cost. First, have a strong model propose a label for each turn and have a human review only the disagreements and a random sample. Second, reuse the trace's own tool call as the label when a downstream check confirms the call produced a correct final answer; that is a weak label but a cheap one. Third, mine recovery cases from incidents: any turn followed by a tool error and then a successful retry is a labeled recovery example for free.

Set a target before you start labeling. If you have 400 captured turns and your selection accuracy is somewhere in the high 80s, the confidence interval on that estimate is roughly plus or minus 3 points at 95 percent confidence, which is tight enough to detect a regression of 5 points. With 50 cases the interval is wider than any change you care about, so invest in volume before you tune thresholds.

The harness replays a conversation prefix, forces tool choice to auto, records what the model does, and scores it. Providers differ in how strict they can be: Cohere's strict_tools mode guarantees that generated tool calls follow the schema, with no hallucinated names or parameters and every required field present (Cohere). That does not remove the need for replay, because strictness constrains the shape of a call, not whether the right tool was chosen or whether the values are correct for the user's request.

A minimal replay loop looks like this:

def replay(case, tools, client, runs=3):
    calls = []
    for _ in range(runs):
        resp = client.chat(messages=case.history, tools=tools, tool_choice="auto")
        calls.append(extract_tool_call(resp))
    return score(case, calls)

Score in three independent dimensions rather than one pass/fail:

1. Selection: did the model pick a tool in the acceptable set, or correctly call none?
2. Arguments: do the arguments validate against the JSON schema, including required fields, types, and enum membership?
3. Outcome: given the recorded tool result, does the final answer match the recorded answer on the facts you care about (order number, refund amount, date)?

Run each case at least three times. Sampling variance means a single pass per case will show flaky "regressions" every release. Report per-case pass rates, not just suite totals, so a case that swings between pass and fail is visible rather than averaged away.

What each dimension catches

Argument-level failures are the most common and the most fixable. Analysis of tool design failures notes that a parameter called user invites a name, an email, or an integer ID, and the model will guess; renaming it user_id removes the ambiguity (AI/TLDR). The same source describes Anthropic's finding that requiring absolute file paths eliminated a whole class of model errors, because the model stopped guessing the working directory.

Replay only covers inputs users actually produced. Fuzzing covers the inputs they will produce next month. Classic fuzzers are a poor fit here: the ToolFuzz authors report that running American Fuzzy Lop against an agent tool for several hours failed to find a single example that realistically represents an agent scenario, because random strings do not look like natural-language queries (ToolFuzz paper). Their method instead uses an LLM to generate queries that are natural but aimed at breaking the tool, combining fuzzing with model-based generation.

The published numbers are worth internalizing. ToolFuzz found 20 times more erroneous inputs than two prompt-engineering baselines, and in the expanded evaluation it identified 50 percent more unique errors while cutting the false discovery rate by 4.5 times. It found that all of the LangChain tools tested and the majority of Composio tools were erroneous (ToolFuzz, ICLR submission). Documentation, not model capability, was the dominant failure source.

You can build a useful fuzz layer without reproducing the paper. Generate four families of cases per tool:

• Type confusion: pass a string where an integer is declared, a list where a string is declared, null for a required field.
• Constraint violations: enum values that are plausible but wrong ("done" instead of "completed"), dates in the past when the schema requires a future date, strings below a minimum length.
• Oversized payloads: a query that returns thousands of rows, a document far larger than the context can absorb. Design analysis notes that an agent calling list_all_records and receiving 800 rows has burned most of its context window on data it will never use (AI/TLDR).
• Ambiguity probes: two tools whose descriptions overlap, with a query that could justify either. If the model splits 50/50 across runs, the descriptions are the problem.

Then feed the failing arguments back through the model rather than asserting directly. The question is not "does my validator reject a string where an integer belongs," which unit tests already answer, but "when the model's call is rejected, does it retry with a corrected value, or does it loop or give up."

Recovery is the dimension most teams skip and the one users notice. The failure mode is well documented: a 400 Bad Request with {"error": "invalid input"} is fine for a human who can read docs or try again manually, but an agent has none of those fallbacks, and the best case is that it halts while the worst is that it retries the same broken payload indefinitely (Szymon Pacholski). The fix is error messages that are prescriptive: what failed, why, and how to fix it, including the allowed values and a hint.

Test this by injecting failures in staging rather than waiting for them. For each tool that can fail, create scripted variants of the same trace:

• Return a validation error listing allowed values.
• Return a 404 for a plausible-but-absent identifier ("City 'Tokyoo' not found").
• Return a rate-limit error with a retry-after hint.
• Return a timeout with no detail.
• Return a partial success.

Then measure recovery rate: the fraction of injected failures where the agent's next action is a corrected retry, an alternative tool, or a clear message to the user, within a bounded number of steps. Cap the loop. An agent that eventually succeeds after 40 tool calls has recovered in a technical sense and failed in every practical one.

Two design patterns make recovery measurable. The first is an explicit failure-reporting tool so the agent has a structured exit path instead of looping (Pristren). The second is validation in the harness rather than the prompt: the runtime checks whether the result is empty, malformed, or out of range, and decides to retry or inject a corrective message (Pristren). A production write-up argues the same point bluntly, that the harness accounts for the bulk of agent reliability while validation logic, permission checks, and retry handling all live in orchestration code, not inside the LLM prompt (MLflow).

Recovery checklist

That last row matters. A retry on issue_refund after a timeout is a double refund. Tools should be designed to be retry-safe, idempotent where possible, with explicit failure states, and write, delete, and financial operations should sit behind approval gates rather than being retried by the model (MLflow).

Turn the three dimensions into release criteria you can defend in a review. A workable gate looks like this:

1. Selection accuracy on the labeled replay set must be at or above the last release, and no single high-traffic intent may drop by more than a set margin.
2. Schema-pass rate on generated arguments must be 100 percent for required fields and types when the provider supports strict mode; otherwise, track it and treat a drop as a blocking regression.
3. Recovery rate on injected failures must meet its floor, and unbounded-loop cases must be zero.
4. Stability: cases that pass on one run and fail on another are counted separately and reported; an increase in flaky cases is a regression even if the mean holds.

When the gate fails, resist the urge to reword prompt instructions first. The observed evidence usually points at the tool interface. Vague names, overlapping descriptions, and loose schemas produce failures that a stronger model only partially masks (Machine Learning Mastery). Anthropic's guidance is to build an evaluation, then iterate the tool against it, using an agent to help analyze results and propose improvements (Anthropic). OpenAI's practical guide takes the same shape: set up evals to establish a baseline, meet the accuracy target with the strongest available model, then swap in smaller models and confirm they hold the target (OpenAI). Your replay suite is that baseline, and it is what lets you switch models on cost grounds without guessing.

Keep the suite honest with two habits. First, refresh traces on a schedule so the eval reflects current user behavior, and retire cases for flows you no longer support. Second, hold back a portion of freshly captured turns from tuning; if you optimize tool descriptions against every case you have, you will eventually fit the test set. A held-back slice, replayed only at release time, tells you whether improvements generalize.

If you are running these suites across several models to pick a cost and quality point, a single gateway helps: Quanta Inference (https://quantaintelligence.ai/inference) exposes text models behind one OpenAI-compatible key, so the same harness can replay a suite against each candidate without rewriting clients. And if the work of building and maintaining the labeled set, the harness, and the gate is more than your team has bandwidth for, Quanta Experts (https://quantaintelligence.ai/agents) hands the whole evaluation loop to a specialist who returns finished deliverables.