When Two Routing Rules Match the Same Request

Give every rule two things: a declared scope (how much of the request it constrains) and a precedence value. Then compute the winner from data rather than from file order.

The comparison order matters. Sort candidates by explicit priority first (highest wins), then by scope weight (narrower match wins), then by age (older rule wins), then by rule ID as a last resort so two rules created in the same instant still produce a stable answer. The rule ID step is not decoration: it guarantees a total ordering, which is what makes the outcome testable.

A small reference implementation:

from dataclasses import dataclass

@dataclass(frozen=True)
class Rule:
    id: str
    priority: int        # explicit human override
    scope: int           # number of constrained fields, 1..5
    created_at: int      # epoch seconds, used as tie-break
    matches: callable

def winner(rules, request):
    candidates = [r for r in rules if r.matches(request)]
    if not candidates:
        return None
    candidates.sort(key=lambda r: (-r.priority, -r.scope, r.created_at, r.id))
    top = candidates[0]
    if len(candidates) > 1:
        log_overlap(request, candidates)   # every match, not just the winner
    return top

Two details carry most of the value. First, scope is computed from the rule's condition set, not typed by a human, so nobody can widen a match and keep a narrow-looking score. Second, the overlap log records all candidates, which is the raw material for the test matrix in the next section. This is the same discipline observability guides recommend for model routing: log the matched condition, the selected route, and the routing reason alongside cost and latency, so the decision is auditable after the fact (Inworld).

One caveat worth writing into the design doc: explicit priority is a footgun if it becomes the only mechanism. Once three rules share the top priority and the scope weights are equal, you are back to relying on age, which is nobody's intent. Reserve explicit priority for a small, named set of overrides and review it whenever a PR touches it.

A tie-break is deterministic when the same rule set and the same request always produce the same winner, regardless of load order, file order or machine. Test for that property directly rather than trusting it.

Three properties are worth asserting as invariants in your test suite:

1. Totality. For any request and any non-empty matching set, exactly one winner is returned.
2. Order independence. Shuffling the rule list does not change the winner.
3. Additivity. Adding a rule that does not match the request never changes the winner for that request.

Property two is the one that catches real bugs. A quick property test with shuffled inputs:

import random

def test_order_independence(rules, requests):
    for req in requests:
        expected = winner(rules, req)
        for _ in range(20):
            shuffled = rules[:]
            random.shuffle(shuffled)
            assert winner(shuffled, req) == expected

Property three is subtler and often forgotten. Teams add rules constantly, and a new rule that silently captures requests it should not is the most common production surprise. Asserting that unmatched additions are inert gives you a cheap regression net around every routing change.

The reason to invest here is operational, not academic. If you cannot attribute a request to the decision that produced it, you cannot calculate what that route costs. Route-selection telemetry is the same primitive cost attribution depends on: without per-request records of provider, model and matched route, spend attribution across shared infrastructure defaults to an allocation assumption rather than a measured number (Mavvrik). A surprising bill and a surprising route have the same root cause, which is a decision nobody recorded.

Shadow mode runs production traffic through a candidate path without letting it answer the user, so you can compare behavior safely (Christopher GS). It is genuinely useful, and it has a blind spot that matters here: sampling live traffic will eventually show you the common overlaps and may never show you the rare ones. The overlap between "enterprise tenant in region B" and "prompt over the long-context threshold" might fire once a week. A two-week shadow run tells you nothing about it.

Shadow testing compares distributions and catches regressions in aggregate, while a pair-by-pair matrix catches specific interactions. The two answer different questions, and you need both.

The matrix is the piece most teams skip. Build it mechanically. During config load, compute which rule pairs can both match at least one synthetic request, then generate one test per pair with an expected winner recorded in a fixture file. Reviewing that fixture is a code review anyone can do, including whoever owns the product rule, because it reads as a sentence: "the regulated-tenant rule beats the region rule for request X."

A worked example makes the size concrete. Suppose you have 40 routing rules and the static analysis finds 26 overlapping pairs. That is 26 test cases and 26 fixture lines, maybe 150 lines of generated YAML. If you instead sample 1% of live traffic through shadow mode, you are likely to observe the five or six highest-traffic pairs and almost none of the rest. Twenty tests generated in an afternoon beat a month of sampling that misses the interaction that matters.

The reverse also holds: when the overlap log shows a pair colliding that is not in your matrix, that is a signal your static pair detection missed a condition (often an access control or feature flag that is not visible to the analyzer). Add the pair, add the test, and check whether the analyzer needs to read one more field.

Run this in order. It takes minutes once the tooling exists.

• Does the new rule have a declared scope computed from its conditions, not typed?
• Does any existing rule pair with it? If yes, is that pair in the fixture file with an expected winner?
• Does the invariant suite still pass, including order independence and additivity?
• Does the overlap log field carry all matching rule IDs, so the winning decision is reconstructable from a single log line?
• Has the owner of any rule that lost a contested pair signed off on the changed winner?

The last item is the one teams forget. A test that asserts the wrong winner passes forever. The fixture only has value when the person who owns the business rule confirms the expected outcome, and that confirmation should be recorded next to the fixture line, not buried in a chat thread.

Once every request carries a matched-rule record, the routing decision becomes a dimension in your cost data. That is more useful than it sounds. Cost-per-request framing matters because a single AI workflow touches model calls, retrieval, orchestration and infrastructure, and those signals land in different places with no automatic link back to the workflow that triggered them (Mavvrik). A route ID is a cheap way to establish that link.

If routing changes move 30% of a workload onto a different path, you want the cost delta attributed to that decision rather than absorbed into a monthly average. PostHog's documentation on calculating LLM costs shows how per-model token counts feed a cost model (PostHog docs), and LiteLLM's pricing calculator exists precisely to forecast spend per request volume before a route goes live (LiteLLM docs). Both become far more accurate when the router emits a route label with every call.

The measurement gap is not hypothetical. In a survey of 500 finance leaders conducted for DoiT, 83% expected a return on AI investment within 12 months and only 15% could calculate AI ROI without significant bottlenecks (DoiT). Attribution is the bottleneck in most of those cases, and a routing decision that nobody logged is one of the smaller but more fixable pieces of it.

When you are consolidating routes across several providers, the plumbing itself is worth simplifying. Quanta Inference exposes text, image, video, audio and embedding models through one OpenAI-compatible API key, which reduces the number of provider-specific branches your routing logic has to carry (Quanta Inference).