A Stale Section Detector: Comparing Context Files to Code

Not every line in a context file can be checked. Before writing a parser, decide which reference classes are machine-resolvable in your repo. The table below is the checklist I use.

Two categories are deliberately out of scope. Prose conventions ("prefer thin controllers") have no resolvable target, and external URLs rot on someone else's schedule. Detect what you can verify locally and leave the rest to a human review, which is the same split Cursor's own guidance draws when it tells teams to review rules like code and remove stale commands, because an agent cannot reliably infer which obsolete rule the team intended to ignore Cursor context management.

The gain from doing this is not marginal. Context files need to stay small to be useful: many teams describe rules that explain where they apply, what the agent must or must not do, and which command verifies the result, and keeping each rule focused makes it cheap to load and easy to verify Cursor context management. A detector turns "keep it focused" from advice into a check that runs on every change.

The extractor reads the context file once, line by line, and emits candidate references with their line numbers. Line numbers are non-negotiable: a report that says "handlers.ts is missing" without a line number forces a search through a 400-line file.

Three rules keep the extractor honest.

Skip fenced code blocks by default. A context file often contains example commands and sample diffs that are illustrative, not operative. Parse fenced regions separately and tag those references as low confidence. Tooling exists for exactly this split: a markdown parser can return each code block with its language and start line so you can treat prose and code differently markdown-analysis on PyPI, and there are dedicated projects that extract fenced code blocks from markdown to real files for validation markdown-code-extract.

Match paths with a strict shape, not a loose regex. Require a path to either contain a slash and a known extension, or start from a known top-level directory such as src/, app/, packages/ or docs/. This avoids matching e.g. and v1.2.3 as filenames. Skip anything inside backticks that also appears in the file's own ignore list.

Treat backticked spans as the primary reference source. Authors who write good context files wrap identifiers in backticks. Parse inline code spans first, then fall back to a bare-word scan for known symbols.

A minimal Python extractor looks like this:

import re
from dataclasses import dataclass
from pathlib import Path

PATH_RE = re.compile(r"^(?:\./)?[\w.-]+(?:/[\w.-]+)*/[\w.-]+\.\w{1,5}$")
SYMBOL_RE = re.compile(r"^[a-z_][a-z0-9_]*\(\)$|^[A-Z][A-Za-z0-9]+$")
SCRIPT_RE = re.compile(r"^(?:npm|pnpm|yarn)\s+run\s+([\w:-]+)$")
FLAG_RE = re.compile(r"^(?P<cmd>[\w.-]+)\s(?P<flags>(?:--[\w-]+|\s*-[a-zA-Z]\s*)+)$")

@dataclass
class Ref:
    kind: str          # path | symbol | script | flag
    raw: str
    line: int
    in_code_block: bool = False

def extract(md_path: Path) -> list[Ref]:
    refs, in_fence = [], False
    for lineno, line in enumerate(md_path.read_text().splitlines(), start=1):
        if line.lstrip().startswith("```"):
            in_fence = not in_fence
            continue
        for span in re.findall(r"`([^`\n]+)`", line):
            span = span.strip()
            kind = None
            if PATH_RE.match(span):
                kind = "path"
            elif SCRIPT_RE.match(span):
                kind = "script"
            elif FLAG_RE.match(span):
                kind = "flag"
            elif SYMBOL_RE.match(span):
                kind = "symbol"
            if kind:
                refs.append(Ref(kind, span, lineno, in_fence))
    return refs

Run it against your context file and count by kind before you write a single resolver. If 90% of references are paths, you may not need the full AST path yet.

Each resolver takes a Ref and returns a status of ok, missing or skipped, plus a detail string. Keep the schema uniform so the reporter does not need per-kind logic.

Path resolver. Join the reference to the repository root and check existence. Handle two wrinkles: paths that are intentionally generated at build time (mark them skipped via an allowlist glob), and paths that resolve relative to a subdirectory. If your context files live at both the root and in nested directories, resolve relative to the nearest enclosing context file first, then to the repo root.

Symbol resolver. Parse the language's files into an AST and collect top-level and exported names once, then do a set lookup. Walking the tree a single time and building a symbol index is far cheaper than searching per reference, and the symbol index is the piece that catches renames, the most common form of drift in context files that describe functions.

Script resolver. Read package.json, pyproject.toml or Makefile and check that the named script or target exists. This catches the case where a context file says "run pnpm test:e2e" and the key was renamed to test:e2e:ci six months ago.

Flag resolver. For each distinct CLI command in the file, run the command with --help once, capture stdout, and check that every referenced flag appears. Cache the output per command per run. If the binary is not installed in CI, mark the reference skipped rather than failed; a detector that fails on a missing dependency will be disabled within a week.

The decision rule that keeps the tool usable: report missing as a hard failure only for paths, symbols and package scripts. Report flags and code-block references as warnings. Paths and symbols are things the repo owns; CLI flags belong to third-party tools that change on their own release cadence.

Suppose a mid-sized repository has four context files (CLAUDE.md, AGENTS.md, and two .cursor/rules/*.mdc files) totaling 600 lines. A first scan yields 412 backticked references:

Forty-two missing references across four files. Assume each takes four minutes to investigate and fix, either by editing the path or deleting the dead paragraph. That is under three hours of one-time cleanup. The steady-state cost is what matters: after cleanup, the detector blocks the pull request that introduces the next broken reference, so the file cannot silently accumulate a backlog again. Without the check, the same team would plausibly reintroduce a few percent of drift per quarter, and each stale path keeps consuming tokens on every agent request in between.

The second number worth tracking is the size of the file after deletions. If the cleanup removes 60 lines of dead references, the context file is 10% smaller, and every request carries that much less irrelevant text. That is the argument to bring to a skeptical reviewer: the detector does not just find bugs, it keeps the instruction budget from leaking.

Treat the detector like a linter: one command, non-zero exit on failure, machine-readable output as an artifact. A GitHub Actions job needs nothing exotic.

name: context-drift
on: [pull_request]
jobs:
  stale-sections:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Detect stale references
        run: python tools/check_context.py --format sarif --out context.sarif
      - name: Upload results
        if: always()
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: context.sarif

SARIF matters because it puts each finding on the exact line in the pull request diff, which is the difference between a maintainer fixing it in ten seconds and a maintainer ignoring the build. If you run GitLab, the same script can emit a JUnit XML file through the artifacts:reports:junit keyword, which GitLab renders as a test report in the merge request GitLab CI/CD YAML reference.

The report format below is the one I recommend: one block per missing reference, with context line first.

CLAUDE.md:118  MISSING path    src/auth/handlers.ts
  -> no file at src/auth/handlers.ts; nearest match: src/auth/session/handlers.ts
AGENTS.md:44   MISSING symbol  buildSessionToken()
  -> not exported anywhere under src/; similar: issueSessionToken()
AGENTS.md:71   WARNING flag    pytest --maxfail
  -> pytest --help does not list --maxfail (did you mean --maxfail=1?)

Three details in that output do most of the work. The nearest-match suggestion turns a rename into a one-word edit. The warning severity keeps the build green for third-party flags. And the file:line prefix makes the finding actionable without context.

Two operational rules. First, run the detector on pull requests, not only on merge to main; the pull request is where the person who broke the reference is still looking. Second, add a suppression comment (<!-- context-drift: ignore --> on the line, or a glob allowlist for generated paths) so the check never becomes a blocker people work around by deleting the whole file. Pattern-guided drift tooling in the wild takes the same shape: the practical advice is to review rules like code and remove stale commands, because an agent cannot infer intent Cursor context management.

If your team also runs agent-written follow-up automation, note that automated doc updates are not a production-grade substitute for a deterministic check: prompt-tuned pipelines need ongoing tuning and carry no memory between runs How to Catch Documentation Drift. The detector and the LLM reviewer are complementary: the LLM catches semantic drift ("this paragraph still describes the old auth flow"), the script catches reference drift with certainty and at near-zero cost.

The detector is code, and it can go stale too. Three habits keep it alive.

Version the extractor with the context files. Put it in tools/ next to the files it validates, and review parser changes in the same pull requests that change the file format. If your team moves from .cursor/rules .mdc files to a plain AGENTS.md, the parser changes with it, and the check keeps working rather than silently matching nothing.

Measure coverage, not just failures. Emit a count of references by kind and a ratio of resolved to skipped on every run. A sudden drop in extracted references usually means the parser broke, not that the file got clean. Alert when extracted counts fall by more than half between runs.

Pair it with a periodic human audit. Reference checking cannot judge whether a paragraph is still true. Schedule a quarterly pass where someone reads each context file end to end and deletes anything that is not load-bearing. The instruction budget argument applies here: a focused file with a dozen high-value lines outperforms a long one, and file paths are the first thing to go stale A Complete Guide To AGENTS.md.

For teams that would rather not maintain the parser, the general pattern (extract references, resolve, fail the build) is the same one that documentation-drift pipelines use, and it composes with whatever arena your agents run in. If you are standardizing across several agent tools, the expert skill library at Quanta (https://quantaintelligence.ai/skills) is one place to keep the instructions and the checks in the same place rather than per-tool copies.