How to Route Contract Review Between a Cheap and a Strong Model

A clause library is not a folder of boilerplate. Each entry is a decision record that tells a reviewer what to use, what to concede and when to stop. Entries that omit those fields push the decision back to a human, which is the cost the library exists to remove, as Aavenir's guide to clause library adoption argues. The same source notes that the preferred position is accepted only in the minority of negotiations, so fallbacks carry most of the workload.

For a cheap model those fields become the instruction set. The minimum viable entry for automated screening has six parts:

Two design choices matter more than the rest. First, put the escalation trigger in the library, not in the router. A clause type like limitation of liability should be able to force escalation regardless of how confident the cheap model is, which is the distinction Ironclad draws between a clause library for governance and a playbook for review: the playbook holds the review-time positions. Second, keep the number of positions small. Bind Legal's structure guide organizes entries by function (liability, confidentiality, data and privacy, termination) and recommends multiple versions with fallback positions, which is exactly the shape a classifier needs.

A library built this way also supports the comparison pass described in Glean's workflow write-up, where each clause outcome is classified clearly rather than left as a free-text comment. Classification is what makes a severity score possible.

The threshold is the only real tuning parameter, and it should be set empirically rather than by taste. The sampling procedure has five steps and takes a few days, not a few months.

1. Build a probe set. Pull a few hundred clauses from executed contracts, stratified by clause type and by how often that type was redlined. Do not sample random clauses only: liability, indemnity, data protection and termination clauses redline far more often than notices and severability, and Aavenir recommends prioritizing clauses by past redlines and escalation friction rather than by a generic textbook list.
2. Run both models on every clause. Record the cheap model's severity score and the strong model's verdict.
3. Define "changed the outcome" precisely. A strong model changes the outcome when it moves the clause across your decision boundary: accepts what the cheap model rejected, rejects what it accepted, or changes the recommended fallback.
4. Plot override rate by severity band. You are looking for the band below which overrides are zero or near zero.
5. Set the threshold at the first band with a non-zero override rate, then subtract one band for safety.

A representative result from a 400-clause probe looks like this. Read the numbers as an illustration of the shape, not as a benchmark.

In that shape, the threshold sits at 3. Bands 1 and 2 stay with the cheap model; bands 3 and 4 escalate. That routes about 15 percent of clauses to the strong model, and the single override in band 2 tells you to spot-check low bands on a sample rather than route them all.

The decision rule for the threshold itself: pick the highest band where the strong model's override rate is under your tolerance, then escalate everything above it. If you cannot tolerate any silent error in a clause type, put that type on the mandatory escalation list in the library and stop negotiating with yourself about scores.

Escalation rate is a budget line, so it deserves engineering attention. Four techniques keep it low without hiding risk.

Batch escalated clauses per contract. One strong-model call carrying five escalated clauses, each with its library entry and the cheap model's note, costs less and reads better than five separate calls. The model sees related positions (a liability cap and its indemnity carve-out) in one context, which is where cross-clause interactions live.

Deduplicate across a portfolio. If you are reviewing a set of near-identical supplier agreements, the same escalated clause text will appear repeatedly. Hash the clause text plus the library entry ID and reuse the strong model's verdict. Teams reviewing high volumes of similar paper get most of their saving here.

Cap per contract and route the overflow to a human. A contract that escalates 30 clauses is not a routing problem, it is a negotiation. Send it to a reviewer with the full cheap-model report attached rather than paying for 30 strong calls that will each say "non-standard."

Use the model only for the decision, not the draft. Ask the strong model for a verdict and a one-sentence rationale against the library entry. The redline itself comes from the pre-approved fallback text, which is what CobbleStone's buyer's guide describes as recommending clauses from the organization's own approved library rather than generating language. Shorter outputs are cheaper and easier to audit.

One structural warning: clause-level retrieval is the right primitive, not chunk-level. Future AGI's build guide walks through a real failure where a retriever surfaced a chunk straddling payment terms and limitation of liability, and the model grounded its answer in the wrong section. If your cheap model is fed chunks rather than whole clauses, its severity scores are noise and the threshold is meaningless.

Instrument three counters per 100 contracts, and report them together. Cost alone hides quality regressions; quality alone hides the bill.

# Decision rule as code: cheap screens, strong judges, threshold from sampling
SEVERITY_THRESHOLD = 3          # set by probe: first band with non-zero override
MANDATORY_ESCALATION = {"liability.limitation", "ip.assignment"}

def route(clause, library_entry):
    screen = cheap_model.score(clause, library_entry)   # returns severity 1-4
    escalate = (
        screen.severity >= SEVERITY_THRESHOLD
        or library_entry.clause_type in MANDATORY_ESCALATION
        or screen.confidence < 0.7
    )
    if not escalate:
        return {"decision": "accept", "tier": "cheap", "severity": screen.severity}
    verdict = strong_model.review(clause, library_entry, must_be_short=True)
    return {"decision": verdict, "tier": "strong", "trigger": screen.severity}

To size the numbers, get your own price and latency for both endpoints from the provider's current price list, then apply them to your measured call counts. The arithmetic a legal operations lead should present looks like this, with placeholder units:

Fill in your own cost per call from the provider's published rate and the table becomes a budget. The latency target matters as much as cost: a routing pipeline that takes 40 minutes has failed even if it is cheap, because the reviewer is waiting. Run the cheap tier in parallel across clauses, and run the strong tier as a single batched call per contract.

Track two quality counters alongside: overrides (strong model disagreed with the cheap model) by severity band, and missed deviations found in human spot checks. If overrides in band 2 start climbing, your library changed under you and the threshold needs resampling. If missed deviations appear in band 1, your parser is splitting clauses and the fix is upstream.

Volume changes the economics of tooling. Once you are routing thousands of contracts a month, one API key across text, embedding and audio models simplifies both billing and the audit trail, which is what Quanta Inference is built for.

The same 100-contract review, run under three designs, shows why the threshold matters more than the model choice. The pattern below is drawn from the probe figures above, applied to a 12,000-clause volume.

The threshold design and the strong-only design produce the same decisions on every clause that matters, because the escalation triggers cover exactly the bands where the strong model ever overrides. The difference is in the denominator. Whole-contract escalation is the trap: it looks disciplined, but it sends clean clauses to the expensive tier because of a header-level judgment, and it gives you no per-clause record to audit.

The practical consequence is that you should defend the threshold in writing. Put the probe table in your legal operations documentation, note the date and the library version, and resample when the library changes materially or when override rate in low bands rises.

Two situations call for a different design.

First-contact or high-value paper. If a contract type appears once a quarter and the counterparty is the company's largest customer, route the whole document to a reviewer. The per-clause saving on a handful of contracts does not pay for the machinery, and the escalation path is short anyway.

Clause types where the library has no position yet. Band 4 in the probe carries the highest override rate because there is nothing to compare against. That is not a routing problem, it is a library gap. Track band 4 volume by clause type and use it to decide what to write next, the way Carta's term sheet guidance treats economic rights and control rights as distinct families of terms that need separate negotiation logic.

When you are ready to build the pipeline itself (ingestion, clause segmentation, routing, a reviewer-facing report) rather than reason about it, a specialist can scope it with you through Quanta.