AI coding

Caching and routing to reduce agent costs

• Choosing a Model Per Task from Pass Rates and Token Counts
• Routing Only the Hard Cases to the Expensive Model
• Testing Output Against a New Model Without Burning the Budget
• Caching and Routing: When a Cache Hit Should Still Escalate
• Hashing a Prompt Template into the Same Key
• Pricing a Cache Hit and a Cache Miss in Dollars
• Prompt Caching for Agents: What to Cache and Skip
• Routing Agent Requests to the Cheapest Model That Passes Evals
• Sampling Judge Calls Without Burning Your Budget
• Sampling Test Runs to Cut Gate Costs Without Hiding Failures
• Segment Budgets: What to Cut When a Check Costs Too Much
• Warm Caches vs Cold Starts: Costing an Agent Fleet
• What a File Denial Costs in Tokens and Retries
• When to Cache a Live Contract Run: Hit Rates and TTLs