Auditing an Inherited Repo's Context Files

Start by enumerating the files. The set is larger than most people expect:

• README.md and any README files in subdirectories
• CONTRIBUTING.md
• AGENTS.md, CLAUDE.md, or whatever agent instruction file the repo uses
• .cursorrules and equivalent editor rule files
• .github/ISSUE_TEMPLATE/*, PULL_REQUEST_TEMPLATE.md
• docs/ pages that describe setup or architecture
• Makefile targets, package.json scripts, and CI config (these are context too, because they teach people and agents what to run)

A quick enumeration on a cloned repo:

# Markdown and agent context files, excluding dependencies
find . -path ./node_modules -prune -o \
  \( -name '*.md' -o -name '.cursorrules' -o -name 'AGENTS.md' \) \
  -print | sort

# Where do the docs actually live?
git log --format= --name-only -- '*.md' | sort | uniq -c | sort -rn | head -30

If you need machine readable structure, a library like markdown-analysis parses headers, sections, code blocks, links and tables out of Markdown files and returns line numbers with each element markdown-analysis on PyPI. That gives you a stable section inventory to reference in your report.

Then assign ownership. Git blame tells you who last touched a line, not who owns the file, so do two passes:

# Who edited this file most recently, and who edited it most often?
git log -1 --format='%h %an %ad %s' --date=short -- README.md
git shortlog -sn -- README.md

# Whose lines still survive in the current file?
git blame --line-porcelain README.md | grep '^author ' | sort | uniq -c | sort -rn

The decision rule:

• If the author of the last meaningful edit is still on the team, that person is the owner and gets the report first.
• If they have left, the owner defaults to whoever last edited the file and is still reachable, otherwise the current maintainer of the module the file describes.
• If nobody qualifies, the file is orphaned, and saying so in the report is itself a finding.

Some repos already encode ownership. GitHub's CODEOWNERS file assigns owners per path and requests their review automatically when a pull request touches those paths About code owners. Check it before you guess:

cat .github/CODEOWNERS 2>/dev/null || cat CODEOWNERS 2>/dev/null || cat docs/CODEOWNERS 2>/dev/null

If CODEOWNERS exists but does not cover the context files, that is a one line fix you can propose in the report. If it does cover them, you still need to check that the named owners are current team members, since an inherited repo often has CODEOWNERS entries pointing at people who left with the agency. Note that team based review has its own noise controls, such as limiting notifications to requested members and auto assignment of reviewers, which matter when you escalate a stale file to a team Managing code review settings for your team.

This is the highest yield pass and the easiest to do mechanically. Extract every code block, command, package name, service name and URL from the context files, then check each against the repo and the outside world.

Extraction can be automated. The markdown-code-extract tool scans a Markdown file for code blocks and writes their contents to real files, with the stated use case of validating code examples in a README and avoiding the divergence that copy and paste produces markdown-code-extract. A PowerShell oriented approach parses code fences and inline code out of Markdown to test documentation examples, and its author makes the case that untested examples teach readers bad habits Parse Code from Markdown Files.

You do not need a full test harness for the first pass. A grep sweep over every command-like line gets you most of the way:

# Every fenced command in the docs, deduplicated
awk '/^```/{f=!f; next} f' README.md CONTRIBUTING.md AGENTS.md 2>/dev/null \
  | grep -Ei '^(npm|pnpm|yarn|pip|poetry|uv|docker|cargo|go|make|./)' \
  | sort -u

Classify every command you find into one of four buckets. This is the comparison that decides what goes in the report:

The worked example that shows up in inherited repos constantly: the README says npm start, and the project actually uses pnpm. You can prove this in seconds by comparing the lockfile to the documented command:

ls -1 pnpm-lock.yaml package-lock.json yarn.lock 2>/dev/null
grep -n '"start"' package.json
grep -rn 'npm start\|npm run\|yarn ' README.md CONTRIBUTING.md docs/ 2>/dev/null

If pnpm-lock.yaml is the only lockfile and the README says npm start, you have a finding with a quote, a path, and a one line fix. Do the same for Python (poetry versus pip versus uv), for Docker (compose file name), for make targets that no longer exist, and for any service name in the docs that does not appear in the dependency manifest or the infrastructure config.

Two rules keep this step bounded:

1. Only mark a tool dead if you can state how you checked. "Not present in package.json, and grep -ri <tool> . outside node_modules returns nothing" is enough.
2. Do not rewrite the docs during the audit. You are producing a list for the authors, not a pull request. Mixing the two makes the report harder to accept.

The most embarrassing class of finding is a context file that argues with the runtime. An AGENTS.md that says "always write tests before code" while the system prompt says "keep changes minimal and do not add tests unless asked" will produce an agent that obeys whichever instruction lands in the window with more weight on that turn. The outcome is inconsistent behavior that nobody can reproduce, because the conflict lives in prose rather than in code.

The mechanics matter here. A system prompt and the context files both occupy the model's context window, alongside the user's request, conversation history, tool outputs and the model's own response, and once the window is full, earlier content is truncated or dropped without warning What Is a Context Window?. That means the order and length of your context files directly affect which instructions survive. A 600 word AGENTS.md that repeats the system prompt pushes useful information out of the budget on every single call.

What counts as a contradiction worth flagging:

• A coding standard in AGENTS.md that the linter or formatter config forbids (tabs versus spaces, semicolons, import order).
• A stated test command in CONTRIBUTING.md that differs from the CI workflow's actual command.
• A stated Node or Python version in the README that differs from .nvmrc, .python-version, or the CI matrix.
• An instruction to use a tool the runtime prompt already forbids, or that forbids one the runtime prompt requires.
• A documented directory layout that no longer matches the tree.
• A documented branch or release process that differs from what the workflows actually do.

The check for the last two is quick:

# Does the documented directory list still exist?
for d in src lib app packages services; do [ -d "$d" ] && echo "OK  $d" || echo "GONE $d"; done

# What does CI actually run?
grep -rn 'run:' .github/workflows/*.yml | head -40

For the prompt side, read the runtime system prompt the product actually sends, not the vendor's public release notes. Anthropic, for example, publishes system prompts for its web and mobile interfaces and notes that those updates do not apply to the API, which is exactly the kind of assumption that produces a wrong context file System prompts. If your product sends its own system prompt, diff it against AGENTS.md by hand and list each line where the two disagree.

One caution on telling the authors "the system prompt should change". In a controlled local experiment, replacing task specific prompts with generic "improved" rules improved instruction following but dropped extraction pass rate from 100% to 90% and RAG compliance from 93.3% to 80% for one model When "Better" Prompts Hurt: Evaluation-Driven Iteration for LLM Applications. The lesson for your report: do not recommend deleting a context file line because it looks redundant. Recommend a change and say how it will be measured, whether by a prompt evaluation suite with defined quality criteria What is prompt evaluation? or a token tracking check that shows the before and after cost per call How to track LLM token usage.

The report is the deliverable. Keep it to one page, and structure every finding the same way so a reader can verify or dismiss it in under a minute.

An example finding, filled in:

README-03 (README.md line 42, owner: J. Okafor, last edited 3 commits before handover) Quote: "Run npm start to boot the dev server." Evidence: package.json has a start script; the only lockfile is pnpm-lock.yaml; npm start fails without a package-lock.json. Impact: new contributors install the wrong tool and get a dependency tree that differs from CI. Fix: replace with pnpm install && pnpm start, or delete the section if setup moved to CONTRIBUTING.

Label the strength of each finding rather than asserting everything is a defect. An evidence ladder with four levels (claimed, observed, reproduced, measured) is a good scaffold, and its author draws a useful line: a scanner finding is not a verified defect until a reviewer has checked the resolved version, the reachable path and the credible consequence Inherited Codebase Audit: An Evidence-Based Guide. The same discipline applies here. "I read the README" is claimed. "I ran the command and it failed with this output" is observed.

Then decide how to hand it over. If the owner is still on the team, a pull request that contains only the report (as a Markdown file) plus a paragraph of context gets the review conversation going in the normal workflow, and reviewers can comment on specific lines or suggest exact changes Pull request reviews. If the author has left, send the report to the current maintainer and name the orphaned files explicitly, because an unowned context file will rot again. When you do file fixes, remember that review decisions are informational unless branch protection requires approval, so say plainly which findings you consider blocking Approving a pull request with required reviews.

A useful closing section for the report is a short list of files to delete outright. The stored context finding that context files can reduce success while raising cost by more than 20% gives you a defensible reason to recommend deletion of any section that names a dead tool, without proposing a replacement Evaluating AGENTS.md: Are Repository-Level Context Files Effective?. Minimal beats comprehensive.

If the audit surfaces a larger question, such as whether the team has enough context engineering skill to keep agent instructions current across several products, that is a people problem rather than a file problem, and it is the kind of thing an expert engagement is built for Quanta Experts. For everything you can fix yourself with a diff and a report, stay in the repo.

The audit fails when it turns into a rewrite. Set explicit bounds before you open the first file.

• File count. Stop enumerating at the files that describe setup, architecture, contribution and agent behavior. Ignore changelogs, release notes and generated API docs on the first pass.
• Time box. One pass per step. If a command requires a paid account or a machine you do not have, record it as unverified and move on.
• Depth. Quote and test commands and setup steps. Do not fact check prose that explains business history unless it contradicts code.
• Output. One page. If the findings exceed a page, rank by impact and send the top ten.

What to hand to the authors, in order:

1. The file inventory with an owner or "orphaned" next to each file.
2. Dead tool and dead command findings, with evidence.
3. Contradictions with the system prompt and build scripts, quoted on both sides.
4. A short list of deletions you recommend.
5. A note on which findings are unverified and what would verify them.

The reason to keep it short is not politeness. The inheritance problem is that nobody holds the full picture, and a long report gets filed rather than read. A repo with no tests to prove its docs still work is exactly the setting where stale references pile up unnoticed for years Detecting Outdated Code Element References in Software Repository Documentation. Your job is to make the drift visible once, with evidence, so the team can decide what to do about it.