Security

Agent Testing and Red Teaming

• How to Do a Security Review of an AI Agent
• Adversarial Test Cases for Agents That Use Tools
• Keeping a Security Review Current as Agents Change
• Setting a Budget for Injection Testing Before Launch
• Testing a Model SDK by Its Failure Modes, Not Its Happy Path
• When the Model Picks No Tool: Scoring the Abstain Case
• When to Re-run Evals After a Prompt Edit
• When to Re-run Evals for a Tool Change, Not a Prompt Change
• A Fake Model Server for Local Development and CI
• A Model Change Log That Links to the Eval Run
• A Regression Set That Grows With Every Incident
• Building a Corpus of Malicious Documents for Agent Testing
• Building a Paired Test Set for Two Models
• Comparing Two Models on Cost and Failures
• Deciding When to Re-run an Agent's Evaluation Suite
• From No-Tool Log to Eval Case: Growing the Corpus
• Fuzzing Agent Tools with Generated Arguments
• Golden Response Set for Tool Outputs: Catching Unintended Changes
• How to Run an Agent Red Team Before It Touches Production
• How to Test Whether an Agent's Tool Permissions Actually Hold
• Rolling Back Agent Prompts and Models to a Known-Good State
• Routing Repairs to a Cheaper Model When Tools Misbehave
• Scoping an Eval Suite to the Risks Your Prompt Change Touches
• Scoring Injection Test Cases Without a Full Review Team
• Scoring Refusals Differently From Wrong Tool Calls
• Testing Your Agent's Kill Switch Before You Need It
• Turning Injection Scores Into a Launch Decision
• Turning Support Tickets Into Eval Cases After an AI Incident
• Versioning Prompts and Eval Results Together in Git
• When a Prompt Edit Breaks Other Tasks: Scoping the Eval Suite
• When to Mock a Tool and When to Hit the Real API in Tests
• Writing a Tool Response Fuzzer for Agent Inputs