Development

Stubbing and replaying model calls

• Choosing a Re-Record Cadence from Vendor Change Volume
• Recording Vendor Responses for Replay in CI
• Splitting Tests by Model Dependency: Stub, Sandbox, Live
• Stubbing an Embedding API in CI Without Losing Fidelity
• Stubbing Model APIs So CI Quickstarts Stay Deterministic
• A Determinism Contract With Every Model Vendor
• A Fixture Inventory That Names Every Owner
• A Fixture Linter: Catching Drift Before It Breaks CI
• A Mock Provider That Keeps the Tool-Call Shape Honest
• A Replay Harness for Failed Agent Runs: Fixtures and Clocks
• Asserting Retrieval Quality Without Calling the Model
• Comparing Two Prompt Revisions on the Same Recorded Inputs
• Pinning a Model Version Beside a Recorded Fixture
• Redacting a Recorded Payload Without Losing the Assertion
• Replaying Recorded Embeddings to Reproduce a Retrieval Bug
• Replaying Recorded Model Responses to Make a Suite Deterministic
• Replaying Recorded Traffic to Tune a Gate Without Risk
• Stubbing a Tool-Calling Model Without Losing the Call Shape
• Tests That Assert an Agent Switches Approach After an Error
• When a Recorded Response Contains a Time or a Random Id
• Which Agent Tool Calls to Stub in CI and Which to Run