Compacting Agent Memory Without Losing the Facts That Matter

Before you write a summarizer prompt, decide what the summarizer is not allowed to touch. In practice, five categories belong in a verbatim block that gets copied, not generated:

The AWS write-up on AgentCore long-term memory makes the extraction side of the same point: a memory system has to distinguish the meaningful from the chatter, so "I'm vegetarian" belongs in long-term storage while "hmm, let me think" does not in the AWS deep dive. It also has to merge related facts over time rather than duplicate or contradict them, which is exactly why the verbatim block needs stable keys.

A practical rule: anything a downstream tool, a person or an auditor could check character by character goes verbatim. Anything that only serves to explain how you got here can be summarized.

Compaction triggers should be boring and predictable. Two work well:

1. Task close. The agent finishes a unit of work (a ticket, a build, a research question). Everything from that unit is now closed and can be folded.
2. Time window. At the end of a day or a shift, fold everything older than the cutoff, regardless of whether it formally closed.

Do not compact mid-task unless the budget forces it, and when it does, treat it as a checkpoint rather than a close: keep the current task's entries raw and fold only the older, finished segments.

The schema is what makes the output checkable. A fixed set of fields means you can diff two folds, count missing fields and replay questions against the result. Something like this works for most operational agents:

segment_id: task-2026-0412
closed_at: <timestamp>
summary: >
  Three-sentence narrative of what happened and why it ended.
decisions:
  - text: "Use Postgres for the audit table; DynamoDB rejected on query flexibility"
    rationale: "..."
identifiers:
  order_id: "ORD-88213"
  ticket: "INC-4471"
error_strings:
  - "ECONNREFUSED 10.4.2.11:5432"
commitments:
  - "Deliver migration script by Friday review"
permissions:
  - "Read only on prod; no schema writes"
open_threads: []
source_range: [entry_1041, entry_1198]

Three properties matter. The schema is fixed, so every segment yields comparable records. The source_range field points back to the raw entries, so a fold is auditable and reversible. And open_threads is explicitly empty for a closed segment, which forces the summarizer to notice anything unfinished instead of quietly dropping it.

Mem0's guide to chat history summarization makes the complementary case that memory formation (selectively storing key facts) beats blanket summarization, with reported reductions in token cost of 80 to 90% in this Mem0 library article. The fixed schema is how you get the formation part right: the summarizer is told which fields to fill, not asked to "compress this nicely."

For a broader treatment of what belongs in durable memory versus the active window, the distinction between semantic memory (stable facts about a user, team or project) and working memory (the current scratchpad) is worth internalizing from Mem0's semantic memory guide. Folding a closed segment mostly produces episodic records with a few semantic facts lifted out of them.

Suppose a support agent serves 400 conversations a day, each around 30 turns, and each turn plus tool output averages 450 tokens. That is about 5.4 million tokens of raw history per day. After seven days you are holding roughly 37.8 million tokens of raw entries, most of it resolved chatter.

Now fold at task close. Assume every resolved conversation collapses to a 150-token record under the schema above (a summary, two decisions, three ids, one error string, a couple of commitments). Resolved conversations are 85% of the volume. Daily cost drops from 5.4 million to about 0.81 million tokens of folded records plus 0.81 million of raw entries for the conversations still open, so roughly 1.6 million tokens, a 70% reduction. Keep the raw entries archived with their source_range intact, so the fold is reversible.

The numbers are illustrative, but the shape is the point: the savings come from folding closed work aggressively, and the safety comes from the schema, the archive and the verification step. Compare that with a uniform summarization pass over everything, where the study above suggests the rules you most need are the ones most likely to erode after several rounds per the compaction cliff paper.

A separate field report is useful for calibration. SentinelLABS ran compaction on an automated malware analysis harness and reported an 86% cut in input tokens with no measurable change to the aggregate evaluation score in their write-up. Their workflow is exactly the closed-segment case: each binary analysis ends, so it can be folded. Where they would have had to be careful is any standing rule from an earlier case that still applied.

A fold you have not checked is a hypothesis. Two checks turn it into a fact.

Replay questions. Before compaction, write a short set of questions whose answers live in the segment: "Which database did we pick and why?", "What was the customer's order id?", "Which host refused the connection and on which port?", "What did we promise the reviewer?". After compaction, run each question against the compacted store, with no access to the raw entries. Score exact matches on identifiers and error strings, and judge the rest for correctness. The Deep Memory Retrieval task in the MemGPT evaluation is the same idea in miniature: answer a narrow question about earlier sessions and compare to a gold answer explained in this alphaXiv breakdown.

Fact diff. Extract the structured fields from the pre-compaction snapshot and from the folded record, then diff them key by key. Identifiers, error strings, commitments and permissions have to match exactly. Decisions have to be present, even if reworded. Anything in the snapshot that is missing from the fold gets flagged with its source_range, so you can look at the source and decide whether its absence matters.

Automate both checks. A fold that fails any exact-match row should not be promoted to the live store.

The failure path is as important as the success path. When a check fails, do not patch the summary by hand. Restore the source entries, mark the fold as rejected, and re-run with a stricter rule. That keeps the system deterministic and gives you a record of which rule was too loose.

The stricter rule usually takes one of three forms:

• Widen the verbatim block. Move a category (say, all tool error outputs, not just the last one) out of the summmarizer's control entirely.
• Shrink the segment. Split a long task into subtasks so each fold covers less ground. The decomposition operator in Knowledge Triage exists for exactly this: split a topic too large to compact safely, and replicate in-scope rules across the partitions per the compaction cliff paper.
• Pin rules ahead of relevance. When retrieving folded records back into context, return applicable constraints first, then relevance-ranked content. The same study reports 100% recall@50 for this approach against 73% for the best single-shot retriever in the same paper.

Keep every fold as a versioned record with its checks attached. Over a few weeks you get a map of which categories your summarizer actually handles and which ones need to be pinned. That map is more valuable than the compression ratio.

For agents that also need to consolidate facts across sessions, rather than just fold closed tasks, the extract, consolidate, store and retrieve loop described in Mem0's long-term memory guide covers the merge step and its failure modes in the Mem0 library. Folding is the per-segment version of that loop, with a tighter scope and a verifiable output.

It helps to be clear about what compaction does and does not solve. It does not replace retrieval, and it does not replace the working context. It sits between them: the active window handles the current task, the folded records and the archive handle everything else, and retrieval decides what comes back in.

A comparison of options makes the tradeoffs concrete:

For most operational agents, folding under a fixed schema is the sweet spot. It gives you a checkable artifact, an audit trail, a rollback path, and predictability, without the engineering cost of a full typed triage system.

If you are running this across many agents and models, one practical friction point is model access: folding prompts, replay question generators and diff judges each want a different model. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API key, prepaid and priced in dollars (https://quantaintelligence.ai/inference), which keeps the verification pipeline from becoming a procurement project. If you would rather have the whole memory review designed and run for you, a Quanta specialist can build it with you (https://quantaintelligence.ai/).