AI Agent Testing Framework

The Trap of Token-Only Assertions

We built this skill because we watched too many engineers treat LLM outputs like deterministic database queries. You write a test, you assert the string matches, you move on. But the LLM is a distribution, not a function. It paraphrases. It hallucinates. It gets distracted by context window noise. When you build an agent that calls tools, the risk multiplies. The LLM might call the right tool with the wrong arguments. It might invent a tool that doesn't exist. It might get stuck in a retry loop because the schema changed or the model provider updated the weights.

Most teams fall into the trap of asserting exact text outputs. They miss the fact that the agent is burning tokens on hallucinated tool calls or returning correct-looking answers for completely wrong queries. Trajectory evaluation scores the entire execution path, not just the final token ^[8]. An AI agent hallucinating a tool call might still produce the right result by luck, but that's a fluke, not a feature you can ship. You need a framework that validates tool correctness, checks argument types, and enforces deterministic assertions on the agent's behavior. Without this, you're not testing an agent; you're testing a chatbot that happens to have a print statement attached to an API.

The gap between lab and production is wide. AgentBench represents the state of the art in agentic evaluation but assumes controlled, single-session, lab-scale execution, where tool availability is static and the environment is pristine ^[7]. In the wild, tools drift, schemas evolve, and the LLM's behavior shifts. We designed the AI Agent Testing Framework to bridge that gap. It forces you to define evals that catch silent failures before they hit users, ensuring your agent behaves predictably even when the underlying model changes.

The Engineering Tax of Silent Agent Failures

Ignoring rigorous evals costs you more than just rework. When agents operate without drift detection, you're flying blind. A 2024 field report from Anthropic warns that you scale only after solving single-agent failure modes, and investing in observability before automation is non-negotiable ^[5]. If you skip this, you pay the engineering tax in three ways: token burn, incident response, and lost trust.

Every hallucinated tool call costs money. If your agent retries a failed tool call because the LLM can't resolve a schema, your AWS bill spikes. We've seen teams spend 40% of their sprint cycles patching agent regressions that could have been caught by a single eval run. If the agent drifts and starts giving legal advice instead of routing tickets, your customer trust evaporates. Microsoft's Agent SRE principles highlight that agents in production are services that can fail, degrade, or cost too much—just like any other service ^[6]. You need guardrails that enforce compliance and quality at every step.

Silent failures are the worst kind. The agent returns a plausible answer, the user is happy, but the tool call was wrong. Maybe it wrote to the staging database instead of production. Maybe it used the wrong currency conversion rate. These errors don't trigger alerts. They accumulate. Drift detection catches these shifts early, comparing production trajectories against your baseline evals. Without it, you're gambling with every deployment. Upgrade your testing posture now, or pay for it later in PagerDuty pages and refunds.

How a Logistics Agent Burned Credits on a Hallucinated Tool

Imagine a logistics team with 200 endpoints that built an agent to handle customer shipment inquiries. The agent uses a suite of MCP tools to check balances, calculate rates, and book shipments. In the lab, the agent passes every test. The final text response matches the rubric. The team deploys.

Three days later, the billing dashboard shows a 300% spike in token usage. The agent is stuck in a retry loop. The LLM has started hallucinating a check_balance_v2 tool that doesn't exist. The agent catches the error, retries, burns tokens, and eventually returns a generic "I'm sorry" message. The team missed this because their evals only checked the final text output, not the tool trajectory. EvalView highlights that most eval tools handle single-turn well but struggle with multi-turn clarification paths and tool use across steps ^[2]. By the time they noticed, the agent had processed 15,000 unnecessary tool calls in a week.

A comprehensive evaluation framework would have caught this. AWS's prototype-to-production guide describes how to evaluate the agent systematically with custom evaluators for tool accuracy and compliance ^[4]. If the team had used a tool-calling validation layer, the eval would have failed the moment the LLM invoked check_balance_v2. Instead, they spent a week debugging a non-deterministic retry loop. This is the cost of weak testing. You need multi-turn evals that track every tool call, every argument, and every fallback.

What Changes When Evals Become Gatekeepers

Once you install this skill, your CI/CD pipeline becomes the gatekeeper for agent quality. You define evals that enforce deterministic assertions on tool calls and rubric-based scoring on outcomes. Promptfoo catches 12 issues your team misses, including silent fallbacks and argument drift. You get LangSmith tracing that maps every token to a specific tool call, giving you full observability across OpenAI, Anthropic, and Google ADK SDKs.

Evaluation strategies shift from "did the text look right?" to "did the agent use the right tools in the right order to achieve the goal?" DeepEval's approach to task completion and tool correctness becomes your standard ^[1]. You detect drift before it hits users. The run-evals.sh script fails the pipeline on regression thresholds, so you can't merge code that breaks the agent. The assert-schema.json validator ensures your eval files are structurally sound, preventing malformed tests from slipping through.

You also get the knowledge base. The eval-strategies.md reference walks you through deterministic vs LLM-as-judge tradeoffs, transcript analysis, and drift detection. The agent-failure-modes.md reference catalogs silent failures, token burning, and auth drift, so you know exactly what to look for. If you're building the agent first, check the AI Agent Builder Pack for orchestration patterns. Once built, you need the AI Evaluation Pack for deeper metrics and human-in-the-loop workflows. This skill gives you the scaffolding to ship agents that hold up under load.

What's in the AI Agent Testing Framework Pack

This is a multi-file deliverable. Every file serves a specific purpose in your testing workflow. No fluff, no boilerplate you have to delete.

• skill.md — Orchestrator skill that defines the AI Agent Testing Framework philosophy, workflow, and references all templates, references, scripts, validators, and examples.
• templates/promptfoo-evals.yaml — Production-grade Promptfoo configuration for agent evals, including deterministic assertions, LLM rubrics, and tool-calling validation.
• templates/langsmith-tracing-config.py — Python integration template for LangSmith tracing across OpenAI, Anthropic, and Google ADK SDKs with OpenTelemetry support.
• references/eval-strategies.md — Canonical knowledge on eval design: outcome verification, tool-calling validation, transcript analysis, deterministic vs LLM-as-judge tradeoffs, and drift detection.
• references/agent-failure-modes.md — Canonical knowledge on silent failures, token burning, auth drift, and runtime guards based on failure mode analysis.
• scripts/run-evals.sh — Executable workflow script that runs Promptfoo evals, aggregates results, and fails the pipeline on regression thresholds.
• validators/assert-schema.json — JSON Schema defining the strict structure for agent eval test cases, ensuring tool-calling and rubric assertions are valid.
• tests/validate-evals.sh — Validator script that checks eval files against the schema and exits non-zero on structural or assertion failures.
• examples/worked-eval.yaml — Worked example of a complex agent eval with multi-turn tool calls, fallback rubrics, and drift detection variables.

Upgrade to Pro and Ship with Confidence

Stop guessing if your agent will hold up under load. Stop patching hallucinated tool calls in production. Upgrade to Pro to install the AI Agent Testing Framework. Run the evals. Catch the drift. Ship with confidence.