AI agents

Golden tasks and evaluation harnesses

• How to Evaluate an AI Agent Before Launch
• How to Build a Golden Task Set for a Support Agent
• How to Evaluate a RAG Pipeline: Retrieval and Answer Metrics
• Setting the Tolerance Band for an Agent Equivalence Rule
• A Budget Sweep Harness: Fixtures, Metrics, a Rerun Curve
• A Fact List per Query: Turning Coverage Checks into a Test
• A Generator Version Field in the Task Contract
• Building a Prompt Evaluation Harness with Golden Tasks
• Cleaning and Labeling Support Logs Before They Grade Anything
• Fixtures That Age: Keeping a Scorecard Honest
• Held-Out Pairs That Survive a Re-Embed
• Held-Out Sets and Train/Test Splits for Prompt Iteration
• Latency Budgets for Agent Test Suites in CI
• Sizing a Pair Set From Clicks and Citations
• Splitting the Golden Set by Question Type