Summarizer Prompts That Preserve Numbers, Names and Negations

The single biggest prompt fix is structural. Instead of "summarize this chunk," ask for a fixed record. Every chunk summary fills the same five slots, and every slot must be present. When a slot has no content, the model writes the literal token none. That gives you three properties you cannot get from free prose: empty slots cannot be invented, missing keys are visible in the output, and downstream parsing is trivial.

Two rules make the schema bite. First, quantities are copied with their unit and never normalized: if the source says "about 40 to 50 ms," you do not write "45 ms," and you do not strip the hedge, because the hedge is the fact. Second, negations travel with their claim. "The vendor does not support SAML" is one item. Splitting it into "the vendor supports SAML" plus a separate "not" note is how a reviewer approves something the source forbade.

Actors matter more than they look. Models are fluent in generic roles, so a name like "Priya Raghavan" drifts to "the product lead" within one fold. Once it drifts, the next fold cannot tell whether two commitments belong to the same person or two different ones, and the final summary reads as if one owner agreed to contradictory deadlines. Copying the proper noun is cheap; reconstructing it later is impossible.

The remaining section on chunk boundaries explains why the schema is per chunk rather than per document, and why that choice decides whether your fold can be audited at all.

Order matters in practice: instructions first, then examples, then the numbered source, then the output skeleton. A model that reads the text first tends to treat everything after it as commentary and answers in prose. The production summarization guidance from Galileo makes the same point about delimiters, noting that separating instruction from content prevents the model from treating document content as part of the instruction the Galileo production guide. The same guide observes that models routinely disregard word-count instructions, which is why structural constraints outperform length targets.

So do not ask for "a concise summary." Ask for five keys and nothing else. A working skeleton:

You are a record extractor, not a writer. For CHUNK 7 only, fill every slot.
Copy verbatim: numbers with units, proper nouns, negators (not, except,
unless, only, no), and modals (may, must, should, will).
Do not round, convert units, expand abbreviations or replace names with roles.
If a slot has no content, write exactly: none

Output:
ACTORS: ...
QUANTITIES: ...
DATES: ...
COMMITMENTS: ...
NEGATIONS: ...

The phrase "not a writer" does real work. Without it, models default to smoothing and hedging because that is what their instruction-tuning rewarded. With it, and with the slot list directly above the text, output shape stabilizes across chunks, which is what lets you diff two folds and see what changed.

Delimiters also protect against a quieter bug: a chunk that contains instruction-like text ("ignore previous instructions," or simply a policy document written in the imperative). Numbered source lines inside a fenced block, with an explicit statement that the block is data and never instruction, costs one sentence and removes an entire class of failure.

Demonstrations steer behavior more reliably than rules alone; few-shot prompting works by conditioning the model on the format and label distribution you show it, and format itself carries signal even when labels are imperfect the few-shot prompting guide. For folding, the useful unit is not a label but a keep/drop pair. Show two or three, each short, each targeting a failure you have actually seen.

A pair that teaches the number rule:

• Keep: "Latency budget is 250 ms at p99, with a hard floor of 180 ms."
• Drop: "The team felt the latency target was reasonable and broadly achievable."

A pair that teaches the negation rule:

• Keep: "Do not enable retries on POST /orders; duplicate charges are not reversible."
• Drop: "Retries were discussed at length and the team has thoughts."

A pair that teaches the modality rule:

• Keep: "Legal may require a second review before the vendor's DPA is signed."
• Drop: "Everyone is aligned and excited about the vendor timeline."

The dropped lines are not random. They are hedges, praise, and restated context, the three categories that eat token budget without carrying a checkable claim. Naming those categories explicitly in the prompt ("drop praise, hedges, and restated context; keep only checkable statements") lets the model generalize past your three examples.

One caution from the same research applies: few-shot examples do not rescue tasks that need multi-step reasoning. If two chunks conflict, examples will not teach the model to resolve it; you need an explicit rule, which is the next section.

Folding many chunks into one record creates a new failure: the record looks unanimous when the source was not. Three chunks may cite three budgets, and a smooth fold will average them or pick one silently. The fix is procedural, not stylistic.

Number every line in every chunk (c7L14), require each folded item to carry its line reference, and write the conflict rule directly into the prompt: if two chunks disagree on a quantity, date, or commitment, emit both with their references, flagged CONFLICT, and never reconcile without a source that resolves it. This mirrors the provenance-preserving design the ablation paper points to as the reason near-verbatim units beat abstraction: retrieval accuracy tracks how far the representation departs from the source the ablation study.

The idea that different content needs different retention policies has independent support. Knowledge Triage, a framework that classifies each item and routes it through a type-specific policy, reports TypeCompact preserving 2 to 4 times more safety rules than the strongest single-shot LLM compactor at every ratio, with 96% recall over five rounds the Knowledge Triage paper. You do not need their classifier to borrow the principle: a rule and a status update should not be compressed at the same rate.

A worked example, with numbers you can check against the sources above. Suppose chunk 3 says the migration window is "6 hours, starting 02:00 UTC," chunk 9 says "four hours from midnight UTC," and chunk 12 says "the window is not yet approved." Under the conflict rule you emit three items with references and one flag. Under a paraphrasing prompt you get "a roughly six-hour maintenance window," and the approval gap disappears. The second output is shorter, reads better, and is wrong twice.

The ablation result does not mean folding is pointless. It means extraction should augment verbatim text, not replace it: a union store of chunks plus artifacts matched chunks on both benchmarks, while artifacts alone forfeited the gap the ablation study. That gives a decision rule rather than a slogan.

The cost side is real and measured. SentinelLABS evaluated compaction against its automated malware-analysis harness and found input tokens cut by about 86% with no measurable change to the aggregate evaluation score the SentinelOne evaluation. That is the case for folding aggressively on the volume content in the left column. It is not a case for folding the right column, which is exactly where the 14.0% versus 91.0% constraint-answering gap lives.

If you are running several models to compare fold quality, or routing text and embedding work through different providers, Quanta Inference (https://quantaintelligence.ai/inference) puts text, image, video, audio and embedding models behind one OpenAI-compatible key with prepaid USD pricing.

Writing the prompt is half the job. The other half is a check that catches the failures you cannot see by reading. Build a small labeled set first: take 20 chunks from your own corpus and list, by hand, every number, name, negator and modal in each one. That list is your recall baseline. The constraint-question probe in the ablation paper is the same idea at smaller scale, and 20 chunks is enough to expose a rounding habit.

Then run three checks.

1. Recall. For each chunk, count how many labeled facts appear in the fold. Any missing quantity or negator is a prompt bug, not a model bug, until proven otherwise.
2. Fidelity. Diff every number and proper noun in the fold against the source string. A changed digit or a shortened name fails.
3. Trace. Confirm every folded item carries a line reference, and that at least one deliberately seeded conflict is flagged rather than resolved.

Track these as rates, not vibes. Arize's walkthrough of summarization troubleshooting makes the case for computing metrics over prompt-response pairs and finding the specific inputs where quality breaks, rather than judging from a handful of good outputs the Arize guide. The same discipline applies: a fold prompt without a per-chunk fact recall number is a guess dressed as a system.

Two small implementation details save time. Log the fold output as structured JSON so the diff is mechanical, and keep the previous fold's record in context so the model can mark "unchanged" instead of restating every slot. If you are maintaining per-project or per-customer fact sets across sessions, the distinction between durable facts and session state is well covered in the writing on semantic and working memory the semantic memory guide.

If you would rather hand the whole pipeline to someone who has built these, Quanta Experts (https://quantaintelligence.ai/agents) is the route where an expert reads your context and returns finished deliverables rather than a prototype.

A fold prompt cannot fix a source that never contained the fact, and it cannot fix a retriever that never surfaced the chunk holding the number. Before tuning further, check which problem you have. The memory-systems survey literature separates retrieval failures from representation failures, and conflating them leads teams to rewrite prompts that were never the bottleneck the agent memory architecture guide. If the fact is absent from the chunk set, no amount of schema will produce it.

A second wrong-tool case is long-horizon sessions where compaction runs repeatedly. Each round compounds loss: 53% preservation after one round becomes 10% after five in the production configuration measured by the compaction study the compaction study. The fix there is architectural, keeping rules verbatim and outside the compaction path, not a better-sounding instruction. If your rules must survive twenty folds, store them as pinned entries and let only the narrative content compact.

The third case is a document low on checkable claims: meeting notes that are mostly sentiment, market commentary, exploratory drafts. A five-slot schema on that material produces a record full of none and a fold barely shorter than the input. Extractive pre-filtering before abstractive synthesis is the standard hybrid pattern for that content, and it is cheaper than forcing an extraction contract onto prose that has nothing to extract the Galileo production guide.