Building on AI models: prompts, context and costs
Writing prompts
Writing prompts that hold up
- A Provider-Agnostic System Prompt: What to Abstract, What to Keep
- How to Write a Skill File for a Coding Agent Your Team Will Share
- Prompt Sections You Can Cut Without Breaking the Output
- How to Test Whether a Prompt Still Works After a Model Swap
- How to Turn Logged Support Failures into Prompt Fixes
- How to Write a Cost-Aware System Prompt That Routes Itself
- Structuring Prompt Output So Tests Can Assert on It
- What to Change in Prompts When a Model Swap Shifts Output
- All of Writing prompts
Sharing and versioning prompts
Sharing and versioning prompts
- How to Turn a Team's Saved Prompts into a Shared Prompt Library
- Regression Tests for Prompts Without a Live Model Call
- A Section Ledger for a Long Prompt: Labels, Owners, Dates
- A Versioning Scheme for Prompts That Shows What Changed
- How to Set Permissions on a Shared Prompt Library
- Logging Which Skill File Version Produced a Live Output
- Reading a Test Failure: Prompt Bug or Model Change?
- Versioning the Shared Prompt Core: A Changelog Reviewers Follow
- All of Sharing and versioning prompts
Token costs and context
Token costs and context budgets
- A Context Budget for an Agent: What to Load and When
- A Failover Rule for Context Overflow: Split, Truncate or Escalate
- A Reason-Check Classifier That Runs on Every Ticket
- A Token Budget per Prompt Section: Where the Spend Actually Sits
- Agent Seats vs Human Seats: Budgeting Concurrency and Rate Limits
- Caching Model Outputs: What to Store and When to Invalidate
- Cost per Task vs. Cost per Token: Which Metric to Dashboard
- Forecasting Token Costs per Team Before a Support Agent Rollout
- All of Token costs and context
Model confidence and calibration
Model confidence and calibration
- Model Confidence Score vs. Your Own Classifier
- Setting a Confidence Threshold from a Calibration Curve
- Why a Model's 0.8 Is Not 80 Percent Right
- A Confidence Score the Agent Can Act On
- A Drift Check for a Confidence Gate Without New Labels
- A Job Aid for the Operator in the Middle Band
- A Moving Threshold: Recalibrating When Input Distribution Shifts
- How to Set a Confidence Threshold for an Agent's Tool Calls
- All of Model confidence and calibration
Routing across models
Routing requests across models
- A Test Suite for Precedence Rules: Cases That Catch Drift
- A/B Testing Model Routing Rules Without Breaking Production
- Combining Two Signals Without Double-Counting: A Lift Test
- How to Tag and Route Inbound Questions by Topic Without an LLM
- Routing Requests by Cost and Quality: Rules You Can Audit
- Small Model First, Larger on Retry: Routing by Tier
- Ticket Signals That Predict a First-Pass Failure
- When Two Routing Rules Match the Same Request
- All of Routing across models
Fallbacks and retries
Fallbacks and retries
- When the Second Retry Climbs: A Ladder for Three Model Tiers
- Reading 409 and 5xx Responses Before You Trust a Retry
- Routing Around a Stopped Model: A Fallback Chain
- The Hard Stop for a Retry Loop That Never Ends
- The Retry Budget: Capping Second Passes per Tier
- The Retry Budget: How Many Second Passes Before Cheap Loses
- The Retry Prompt: What to Add, Cut and How to Test It
- When a Fallback Model Fails: Writing the Chain and the Stop Rule
- All of Fallbacks and retries