Business

Running a model bake-off

• How to Run a Model Bake-Off: Script, Score and Sign-Off
• When to Accept a Live Call in Your Test Suite
• Building a Baseline Test Set from Production Traces
• How to Keep a Bake-Off Repo a Reviewer Can Rerun
• How to Sample Production Traffic for a Test Set
• How to Set a Cost Ceiling for a Model Evaluation Run
• Logging Two Providers Side by Side: A Cross-Run Comparison Schema
• Mocking an Agent's Tool Calls Without Mocking the Model
• What a Flaky Test Tells You About a Nondeterministic Model