Routing tasks between models

Routing LLM tasks between models and code

• Classifying Requests Before They Reach the Router
• Routing Logic for LLM Tasks: Rules, Thresholds and Fallbacks
• Signals a Router Can Read Before Any Model Call
• When to Let the Model Choose the Tool vs. When to Route in Code
• Config Format for Routing Rules: Fields, Defaults, Validation
• Logging the Cap Decision per Call
• Placing the Router: Gateway, Service or Prompt
• Routing by Data Sensitivity: Where a Prompt May Go
• All of Routing tasks between models

Model selection and fallbacks

Model selection and fallback chains

• A Canary Process for Model Version Upgrades
• Choosing the Model for a Tool-Using Agent: A Comparison Method
• Detecting Model Drift When Your Provider Updates a Model
• Fallback Hops for a Routed Call: Order, Triggers and Cost
• Handling a Provider Model Retirement in Code, Config and Tests
• Pinning Model Aliases in CI: A Suite That Fails on Drift
• Pricing a Fallback Tier: Cost per Overrun and Break-Even Volume
• When an Alias Points to a Deprecated Model: A Failover Test

Latency and cost routing

Latency and cost based routing

• Routing by Output Length: A Budget the Model Cannot See
• Routing Conflicts by Latency: When a Slow Source Deserves a Human
• Routing by Latency: When a Slow Class Deserves a Smaller Model
• When to Recompute the Cut Point After a Mix Shift

Confidence and tie-breaks

Confidence, classification, and tie-breaks

• Routing Conflicts by Cost: Which Precedence Calls Need a Human
• A Drift Detector for Classifier Inputs: Three Cheap Metrics
• Calibrating a Small Classifier: From Raw Scores to a Cut Point
• Calibrating a Support Agent's Confidence Score
• Confidence Scores the Router Can Trust: Calibration
• Labeling by Exception: Router Overrides as a Training Set
• Measuring a Classifier's Misfire Rate and Its Cost
• Routing by Answer Confidence: Where the Cut Point Lives
• All of Confidence and tie-breaks

Routing decision logs

Routing logs, decision records, and audits

• A Decision Log for Routing: What to Record and Who Reviews It
• A Trace Schema for Routing: Model, Reason and Fallback Hop
• Comparing a Router and a Model-Chosen Run on Logs
• Logging the Route Decision: Fields That Prove Where a Case Went
• Measuring a Router in Production: Misfire Rate and Its Cost
• Routing Between Prompt Versions and Models at Runtime
• What to Log When an Agent Picks a Tool: Fields for Debugging