Compaction Triggers: When a Fold Breaks the Answer

The simplest trigger anyone writes is a counter: after N turns, summarize or drop everything older than the last M turns. The OpenAI-side demo of agent memory shows exactly this configuration in a side-by-side agent harness, where trimming is set to fire at the sixth turn while keeping the most recent three turns in context (agent memory demo). It works as a demo. It is a poor production rule on its own.

The trouble is that turn boundaries do not align with subtask boundaries. A numbered list of ten questions counts as ten turns and may be one subtask. A single turn containing a 40,000-token file read is one turn and a whole subtask. A counter sees neither.

If you insist on a counter, make it a secondary guard rather than the primary rule. Fire the count-based fold only when a subtask-closing signal also fires: the agent produced a final answer to a sub-question, or a plan step completed. On its own, turn count is a proxy for something you can measure directly.

The token milestone is the trigger nearly every harness implements, because the context window has a hard ceiling and providers publish it. Haystack's compaction hook is a clean reference: it starts compacting at 70% of the model's context window and asks the compactor to reduce the context to roughly 40%, configured explicitly as compact_at=0.7, compact_to=0.4 against a declared window size (Haystack compaction docs). The numbers are yours to choose; the pattern (threshold up, target down) is the important part.

Two rules turn this from a crude guard into a decision.

First, read the limit from the provider documentation rather than guessing, and subtract the room you need for the response. If the model accepts a window of size W and your largest expected completion is C tokens, the trigger threshold belongs at (W minus C) times your chosen fraction, not at W.

Second, and more important: when the threshold trips, do not fold immediately if the agent is mid-subtask. Mark the context as "over budget" and defer the fold to the next clean boundary, defined as the end of a plan step or a completed tool-and-reason cycle. If the run cannot continue without folding right now, fold the least load-bearing material first. That means old tool output, not the instructions at the top of the window.

There is a harder reason to prefer boundary-aligned folds. Even within a nominal window, models attend less precisely to material buried in the middle of a very long context, and recall of early context degrades well before the window is full (context engineering for multi-turn agents). Waiting for the hard ceiling means running for a long stretch in the degraded zone. A threshold well below the maximum, crossed at a clean boundary, buys accuracy as well as space.

The third family is the one that decides most incidents: a tool result whose payload is large, or whose exact content drives the next call. Here the broadcast rule is simple. Archive the raw payload to durable storage, put a short retrievable reference in context, and keep the recent results inline (Haystack compaction docs).

The failure mode of folding these too early is concrete and ugly. Consider an agent that calls a deployment tool, receives a payload containing a build ID, a container tag, and a rollback command, then folds that payload into a sentence like "deployment succeeded." The next step asks the agent to roll back. It now has no build ID. A model with no exact string will frequently produce one that looks right, and an ID that looks right in the log is one of the more expensive kinds of wrong.

The inverse failure is equally real. Tool output can spike unpredictably; a file read or an API response can return thousands of tokens in one call, and a handful of those turns will crowd out the reasoning the agent actually needs (context engineering for multi-turn agents). When a single tool result dominates the window, folding it is right. Just do it as offload, not deletion.

Take a coding agent with a declared window of 200,000 tokens, a maximum expected completion of 8,000 tokens, and a trigger fraction of 70%, following the shape of the Haystack example (Haystack compaction docs).

1. Usable budget: 200,000 minus 8,000 for the response leaves 192,000 tokens. The trigger fires at 0.70 times 192,000, which is 134,400 tokens.
2. Mid-run, the counter reads 131,000 tokens as the agent waits on a test-suite run. It is under the trigger; nothing happens.
3. The test tool returns 9,500 tokens of log output, putting the total at 140,500. The trigger trips, but the agent is mid-subtask: it has not yet read the failures. Defer.
4. The agent reads the log, isolates three failing tests, and writes a fix. The subtask closes at roughly 146,000 tokens.
5. Fold now. Summarize the reasoning, offload the 9,500-token log to storage with a reference, keep the three failing test names verbatim, and target 40% of the usable budget, about 76,800 tokens.

Notice what the fold preserved: the exact test names, because the next step depends on them, and the plan. What it discarded: the log body, which the model had already distilled into a diagnosis, and is retrievable if the fix fails. A turn counter would have fired somewhere in the middle, around step 3, at the worst possible moment.

The same arithmetic applies to cost. Naive agent loops rebill the entire prior conversation on every call, so a 20-step loop can consume more than ten times the tokens a per-step estimate suggests (agent loop token costs). Boundary-aligned folds cut that curve without touching the strings the run depends on.

A trigger is a hypothesis about what the run can afford to forget, and hypotheses need tests. Evaluate the retriever and the generator separately so a failing trace points at a cause (RAG evaluation guide). For compaction, the equivalent is a three-part check after every fold:

• Constraint recall: did every rule and exact string that must survive survive? Compare the pre-fold and post-fold context against a list you maintain.
• Task completion: does the run still finish? Task completion and step efficiency are standard agent metrics (LLM evaluation metrics).
• Cost and latency: what did the fold actually buy in tokens and milliseconds?

This is where the published evidence should shape your expectations. A production evaluation of native compaction in the Responses API reported an input-token reduction of about 86% with no measurable change to aggregate evaluation score (SentinelLABS compaction evaluation). A separate study of 20 production agent configurations found that a widely used compaction prompt preserved 53% of safety rules after one round and 10% after five, and named the effect the compaction cliff (The Compaction Cliff). Both numbers are real. The difference is that the first measured task score on a workload where summaries sufficed, and the second measured retention of exact constraints, a property summaries degrade with each round.

Repeated folding is the case most teams never test. If your agent folds five times in a long session, measure the output of the fifth fold against the first, not against the original context, which is the comparison a single-shot evaluation will quietly make.

None of this needs a model call. A fold decision is a handful of cheap checks on the message array, and separating the trigger from the strategy keeps it testable (Haystack compaction docs). The sketch below is deliberately plain: it encodes the table above and nothing more.

def should_fold(state, window, max_completion, fraction=0.70):
    budget = window - max_completion
    over = state.tokens > fraction * budget
    if not over:
        return False
    # token milestone crossed: wait for a clean boundary
    if not state.subtask_closed:
        return "defer"
    # exact strings needed by the next call must survive
    if state.next_step_needs_exact and not state.pins:
        raise ValueError("pin required strings before folding")
    return True

Two habits keep this honest. First, maintain a pins list: the exact strings the next action will use, lifted out of tool payloads and kept verbatim through the fold. Second, log every trigger decision with the reason (token milestone, tool result, turn count) and the tokens saved. After a few weeks you will know which trigger actually fires and which is decoration.

If you are running several models and want one place to route both the main agent and any summarizer the fold uses, Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible key (Quanta Inference). Keep the summarizer cheap and the main loop on the model that reasons best.