What to Prune When a Context File Outgrows Its Token Budget

Ask two questions per section, in order.

Does removing it change output? Delete the section, run the eval, compare against the noise band. If the pass rate holds, the section is not doing work: drop it. If the rate falls outside noise, the section matters and you move to question two.

Can the same instruction be stated in fewer tokens with the same effect? Compress it, run the eval. If the rate holds within noise of the pre-rewrite run, rewrite and keep the shorter version. If it falls, keep the original wording.

The discipline that matters is serial application. Bundling ten cuts into one commit and seeing the rate hold proves the ten together were unnecessary, not which five were. You will keep a bloated section because a good cut in the same batch masked its contribution.

There is a deliberate asymmetry here. Dropping is cheap to test and reversible; keeping costs tokens on every request. Every token in a persistent context file loads on every request regardless of relevance, so the file should be as small as possible Matt Pocock's AGENTS.md guide. That guide cites an instruction budget of roughly 150 to 200 instructions followed with reasonable consistency by frontier thinking models, and fewer for smaller or non-thinking ones. That is a ceiling on instruction count, not just characters.

The last row is the one people skip, and it is the commonest reason pruning campaigns stall. A section that produces different verdicts on consecutive runs is a coin flip, not a finding.

Non-determinism survives temperature zero. Outputs can vary across calls due to hardware numerics, decoding implementations, concurrency or upstream model changes the evaluation-driven iteration paper. That paper also reports a blunt warning: replacing task-specific prompts with generic "improved" rules moved extraction pass rate from 100% to 90% and RAG compliance from 93.3% to 80% for one local model, while instruction-following improved. Quality dimensions trade against each other.

So do not ask whether a section helps "generally." Sections help specific behaviors, and a change that improves format adherence can quietly break extraction. Split the eval into per-behavior buckets even if you report one headline number. When a removal drops the aggregate by three points, the buckets tell you whether you lost formatting or correctness. Those are not worth the same tokens, and your stopping rule needs that distinction.

Take a repository context document with 40 headed sections at 7,850 tokens: architecture notes, conventions, exact commands, known gaps, file paths, directory listings and a decision log. A useful context document covers architecture, domain model, patterns and conventions, file paths with purpose, exact commands and known gaps the CONTEXT.md discussion. That list suggests most sections should be dense, not narrative.

The eval was 60 tasks drawn from real work: add an endpoint following existing patterns, thread a field through the data layer, fix a failing test with the project command. Baseline on the untouched file was 51 of 60, or 85.0%. A second run also scored 51 of 60, so the noise band was plus or minus two tasks.

Applying the rule took 16 deletion tests and 9 rewrite tests.

The surprise was the discoverable categories. Six directory trees and five stack summaries cost 1,920 tokens, and removing them moved the rate from 51 to 52 of 60. Within noise, so the honest reading is no effect. But it matches research finding that LLM-generated context files reduced task success by 2 to 3% while raising cost over 20%, and that the same content improved performance by 2.7% once all repo documentation was stripped first, meaning it is redundant rather than useless Addy Osmani's summary.

The file landed at 3,710 tokens, a 52.7% reduction. Pass rate moved from 51 of 60 to 54 of 60, or 90.0%. All three gained tasks came from buckets where a rewrite removed a conflicting instruction, not from a deletion. Deletions bought tokens; rewrites bought accuracy.

Two sections earned specific attention. The style section held seven rules across 900 tokens, several contradicting each other on error handling; compressed to four imperatives at 430 tokens, the endpoint bucket went from 8 of 12 to 10 of 12. The architecture section lost its narrative of why the system was built that way, which no task depended on, and kept the invariants new code must respect.

The stopping point came at the eleventh deletion. Removing a cascade delete rules section cost two tasks, and the rewrite attempt cost the same, so the content was irreducible. Those 140 tokens were worth more than they cost.

Deletion is mechanical. Rewrite is where judgment enters.

Convert prose to imperatives. "We generally try to keep controllers thin and put logic in services, though there are exceptions in legacy billing" becomes "Controllers delegate to services. Exception: billing/legacy keeps logic in controllers." Fewer tokens, exception preserved. Context engineering practice points at clarity and precision in instructions, with structured organization and format optimization as supporting moves Packmind's context engineering practices.

Replace discoverable description with pointers. The distinction that matters separates what the agent finds by reading the repo from what it cannot: tooling gotchas, non-obvious conventions, landmines. Everything else is noise Addy Osmani. A context file briefs the assistant rather than introducing the project, which is exactly the non-discoverable layer the CONTEXT.md discussion.

Delete instructions the model no longer needs. Capability moves, and guidance required by an earlier generation is often dead weight. Build rules files gradually rather than front-loading, because what you once had to include may no longer be necessary Birgitta Böckeler on context engineering for coding agents. Your eval decides "no longer necessary," which is why the double baseline matters.

For the bookkeeping, parse the file into sections so token-per-section accounting is mechanical rather than a manual audit. A Markdown analysis library extracts headers, sections, code blocks and lists programmatically the markdown-analysis package, which gives you the table above for your own file.

from mrkdwn_analysis import MarkdownAnalyzer

print(MarkdownAnalyzer("CONTEXT.md").count_elements("header"))
print(MarkdownAnalyzer("CONTEXT.md").analyse())

Dropping a section from the always-loaded file does not mean deleting the knowledge. Tools, MCP servers and skills are context interfaces the agent loads when it decides they matter, and each one you configure consumes space, so choosing them deliberately is part of the same budget problem Birgitta Böckeler.

The routing is simple. Exact build and test commands stay loaded: short, non-discoverable, constantly used. Long runbooks become a skill. Deep domain detail becomes a referenced document read when the task touches the data layer. Decision logs move to a record consulted only when someone asks why.

Staleness is why this matters. An analysis of over 3,000 GitHub projects found 28.9% of the most popular projects currently contain at least one outdated code element reference, and 82.3% were outdated at least once in their history the outdated references study. A human fills the gap with skepticism; an agent reading stale text on every request has stale context poisoning its work Matt Pocock. Moving high-volume, slow-changing material out of the always-loaded path shrinks the surface that can rot.

To catch drift mechanically rather than only through your eval, a CI step that compares docs against merged diffs and opens a follow-up pull request is workable, with the caveat that prompt tuning is ongoing and the pattern is not production-grade without care Dosu's documentation drift workflow. Drift detection covers facts; the eval covers behavior.

A pruned file regrows. The loop is predictable: the agent does something you dislike, you add a rule, and after enough cycles you have a ball of mud Matt Pocock.

Three habits hold the line. Give the file a token ceiling, and require every new section to fit inside it or displace an existing one through the same eval process. Keep the eval set in version control beside the file, so a proposed rule arrives with evidence. Review composition as well as size: a small file that is all style rules and no landmines is badly allocated. The five-theme taxonomy of what developers actually write in these files, covering conventions, guidelines, project information, LLM directives and examples, is a useful checklist for spotting a missing category the empirical study of developer-provided context.

The case in one sentence: excessive or unoptimized context produces more complex and less accurate responses along with higher cost and latency the same study. Context is a finite resource with diminishing marginal returns, and recall accuracy degrades as token count grows, across models Anthropic's context engineering guide. The guide frames this as an attention budget every new token depletes. Pruning spends that budget on instructions that change output.

If the eval runs across several models, route the harness through one interface so verdicts are attributable to the context file rather than to plumbing. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible key, keeping the harness stable while the model underneath changes.

Once the file is pruned and the eval is green, the remaining decision is which dropped sections should become on-demand skills rather than deleted knowledge. That routing work is what Quanta Skills covers: install a skill into a coding agent, or hand the job to an expert that reads your context and returns finished deliverables.