Tool call errors

Failed tool calls and error handling

• Designing Error Payloads an Agent Can Recover From
• Metering Agent Retries at the Tool Boundary
• Teaching a Tool to Return a Hint With Every Error
• A Decision Table an Agent Runs Before It Calls a Tool
• A Decision Table for Tool Errors: Fix, Retry, Escalate or Abort
• A Retry Tier Table: Errors That Get a Second Call
• Counting Wrong-Type Arguments by Tool, Field and Model
• Mapping Error Codes to Runbook Links an Agent Can Follow
• All of Tool call errors

Structured output

Structured output and JSON reliability

• JSON Mode, a Schema or a Tool Call for Structured Output
• Payload Shapes Across Model Families: An Acceptance Table
• Prompting the Cheap Tier to Say When It Is Unsure
• A Hosted Platform Is Its Own Row in the Matrix
• A Repair Step That Keeps the Original Payload
• A Schema Conformance Suite That Runs Against Every Model
• A Schema Registry for Agent Prompts: Versioning and Rollback
• Handling Tool-Call Arguments a Model Sends in the Wrong Type
• All of Structured output

Test tampering

Detecting agent cheating and test tampering

• Blocking an Agent from Weakening Its Own Assertions
• A Tamper Log Schema: Recording Refused Test Changes
• Logging Test Tampering: Fields and Alerts
• Shadow-Running a Tamper Detector Against Human Review
• When an Agent Edits Its Own Checks: Blocking Test Tampering
• When an Agent Weakens an Assertion Outside the Test File
• When an Agent's Test Passes for the Wrong Reason: A Checklist

Reviewer calibration

Calibrating automated reviewer decisions

• Designing Reviewer Abstention: A Band Between Pass and Fail
• Shadow Runs: Comparing the Gate's Verdict to a Human Reviewer
• Two Judges and a Human: Three-Way Agreement as a Gate
• Two LLM Reviewers Disagree: A Tie-Breaker Pass
• When the Tie-Breaker and the Human Disagree: An Override Log
• A Calibration Log for Pass Criteria
• A Calibration Set for the Abstention Band
• Cohen's Kappa Per Criterion: A Spreadsheet and Its Limits
• All of Reviewer calibration

Dependency reliability

Dependency and lockfile reliability

• A File Manifest for Lockfile Diffs: Paths, Globs and Line Counts
• A Lockfile Diff Budget: Capping an Agent's Dependency Change
• A Second Registry Check Before an Agent Bumps a Package
• Costing a Dependency Bump in CI Minutes
• How to Read a Dependency Change an Agent Made
• Reading a Lockfile Diff: Which Entries a Reviewer Must Open
• Teaching an Agent to Read a Lockfile Before Touching Dependencies
• When an Agent Bumps a Major Version: A Pre-Merge Checklist
• All of Dependency reliability