Reviewing an Agent Diff at a Fixed Time Cost: A Triage Order

A human developer usually knows what they touched, because they were present while touching it. An agent does not work that way. It may edit many files across several steps, recover from failed commands, and run tasks that branch and backtrack. The diff you receive is the residue of a search process, not a record of intent.

That changes what a reviewer should look for. In a human diff, a surprising file is usually an accident. In an agent diff, a surprising file is often a workaround: the agent hit a failure, added a migration, pinned a dependency, or flipped a flag to get past it. Those artifacts cluster in the first three passes precisely because they are the levers an agent reaches for when the obvious logic path is blocked.

Practical guidance for agent-assisted development makes the point from the author side: before a substantial change, check the working tree with git status and git diff so an unfinished task does not get mixed into the new one, and after the agent finishes, inspect the complete change to confirm every file belongs to the task and no credentials or build artifacts crept in (ESP-IDF workshop). The reviewer's four passes are the receiving end of that discipline. If the author did not clean the change, the reviewer will find the mess in passes one through three.

The behavioral guidance now circulating as a shared skill offers a yardstick you can apply during review. The Karpathy guidelines ask an agent to state assumptions, make surgical changes, remove only the orphans its own edits created, and treat every changed line as traceable to the request (SKILL.md). If the schema pass shows a new column the ticket never asked for, the agent violated the guideline and you can stop.

If you would rather tighten the authoring side than catch it at review, installing that kind of behavioral skill into the agent is the cheaper fix. The library that sits behind every Quanta expert is available for exactly that, so you can install a skill into your coding agent or put an expert on the job.

Read first the files that define persistent shape: migration files, ORM models, type definitions that cross a boundary, protobufs, and API schemas. This pass is cheap because those files are short relative to the logic they unblock, and it is expensive to get wrong because a schema change is the hardest thing in the diff to reverse after merge.

Three checkpoints, in order:

1. Is every schema change traceable to the request? For each added or altered column, table, index or field, name the sentence in the task that requires it. If you cannot, that is a fail.
2. Is the change reversible? A migration that drops a column, renames one, or narrows a type needs a backfill or a two-step deploy. Check whether the agent produced one, or whether it assumed the deploy is atomic.
3. Does the change respect existing constraints? Nullability, uniqueness, foreign keys and default values are where an agent's model of the domain shows. A non-nullable column added to a populated table with no default will fail on deploy.

Worked example. A task reads "allow users to save a preferred currency." The diff adds a preferred_currency column, then also adds a currencies reference table and an index, and drops users.locale. The first additions might be defensible; the drop is out of scope and is a one-way door. Stop at pass 1. You have spent about six minutes and avoided an hour. The response is not "please fix the logic," it is "split this: currency column only, and explain the locale removal in a separate change."

That early exit is the whole point of the order. You are not trying to be thorough on a diff that is wrong in kind. You are finding out cheaply whether thoroughness is worth buying.

Next, filter the diff down to manifests and lockfiles, then read only those hunks. Dependency review is the second-cheapest pass and the second most likely to invalidate a change, because a new package or a widened version range expands what runs in production and what your team now has to track.

For Node projects, the lockfile is the place to look: it pins exact versions, records integrity hashes, and captures the full dependency tree, which is why it belongs in version control for applications (OneUptime). The counter-rule matters just as much. Lockfiles are for top-level projects, not for packages that are themselves dependencies of other packages, where committing one makes maintainers resolve versions their users will never see and breaks the assumption that the published package works against the ranges it declares (dev.to). So the first dependency question is which kind of repository you are reviewing.

# See what the agent actually added or widened
git diff --stat main...HEAD -- '**/package.json' '**/package-lock.json' '**/pyproject.toml' '**/requirements*.txt' '**/go.mod' '**/Cargo.toml'

Use --stat first. If the manifest diff is small but the lockfile churn is enormous, the cause is usually one new package pulling a large transitive tree, not many deliberate additions.

An agent that runs an install to make its tests pass will happily update a lockfile as a side effect. That is the most common dependency failure you will see, and it is invisible in a logic-first read because the code looks fine.

Check the size mechanically too. A dependency-free GitHub Action can fail a pull request once its diff crosses a line threshold, counting additions and deletions from the files API, with glob patterns for ignored paths such as **/*.lock and a bypass label for exceptions (pr-size-check). Excluding lockfiles from the count is the right default: they belong in the dependency pass, not in the line budget you spend on logic.

Then read environment variables, feature flags, config files, defaults and secret handling. This pass is cheap in lines and disproportionately likely to hide a silent production failure, because the agent's session had an environment and your deployment has a different one.

The checkpoints:

• Every new environment variable referenced in code is defined, defaulted or documented. Grep the diff for the variable name and count the definitions.
• Defaults fail safe. A new feature flag that defaults to on, with no rollout plan, is a fail.
• No secrets, credentials or build artifacts. A basic review discipline for agent-generated changes is to confirm every changed file belongs to the task, that generated files are intentional, and that no credentials were added (ESP-IDF workshop).
• Flag lifecycle is stated. Who removes the flag, and when? A flag with no owner is permanent configuration debt.
• Config changes match the deploy target. If staging and production read different config sources, a change that touches only one is incomplete.

# Environment variables added by the diff
git diff main...HEAD | grep -E '^\+.*(process\.env|os\.environ|getenv|ENV\[)'

For each name that grep returns, search the repository by hand for a definition in an environment template, a config schema, or a deployment manifest. A variable used in code and absent from every template will take down the process at startup, and no amount of logic review will catch it.

Configuration review also explains why some agent changes look correct in the transcript and fail in CI. The agent ran the code with its own environment; the diff carries the code but not the environment. Reviewing configuration is reviewing that gap.

Only now read function bodies, control flow, error handling and tests. This is where most of the review minutes belong, and by this point you know the change is worth them.

Work through it in a deliberate order rather than top to bottom:

1. Tests first. Read the tests the agent added or changed before the implementation. They tell you what the agent believed the requirement was, and a test asserting the wrong behavior is the clearest possible signal. Reviewing tests first is a practice experienced reviewers recommend for exactly this reason, and starting with the file that has the biggest change is a common way to find the theme of the diff (Pratik's Pakodas).
2. The largest hunk next. It usually carries the center of gravity of the change and orients you for the rest.
3. Error paths and boundaries. Agents are strongest on the happy path. Look at what happens on timeout, empty input, and partial failure.
4. Scope discipline. Every changed line should trace to the request, and orphans created by the change should be removed while pre-existing dead code is left alone (Karpathy Guidelines). Nits go in as prefixed comments the author can skip, which keeps attention on decisions that matter.

A review model that starts with context and business flow, and that weighs the return on investment of each comment before leaving it, is the right frame for this last pass. Business-flow errors can force a full rewrite, and a comment that turns a small task into a large one should move to the backlog instead of blocking the merge (Medium). A template that asks the author what the worst outcome could be, how well the change is covered by tests, and what the monitoring plan is gives the reviewer the context these passes assume (Workflow86).

Because agents can produce very large diffs, the logic pass is also where splitting pays. Stacked pull requests, where each change builds on the previous one, with schema first, then the service layer, then endpoints, then frontend integration, keep every diff reviewable and independently testable (DeployHQ). If pass 4 keeps failing because the diff cannot be held in your head, the fix is a smaller diff, not a longer review.

A reading order saves time only when the diffs arriving at your queue are small enough to survive it. Three practices help.

Ask the agent for a narrower change. A spec with an explicit goal, a list of files it may edit, and a stop condition produces fewer diffs that die in pass 1 than a conversational request does (SurePrompts). The same idea governs the agent's instructions: tell it what it owns, what its tools are, and when to stop, because the failure modes that matter are wrong actions, runaway loops and context overflow (The AI Engineer).

Isolate concurrent tasks. Worktrees let each agent task run in its own working directory against the same repository history, so parallel tasks do not contaminate each other's uncommitted changes (Nx). Smaller, isolated diffs are what make a four-pass review finish in minutes rather than an hour.

Gate the size, not the reviewer. A reviewer who has to negotiate over a 1,500-line diff has already lost the budget argument. Set the threshold in CI, ignore generated paths and lockfiles, and require a label to bypass (pr-size-check). The bypass exists so exceptions are visible, not so they become routine.

When the volume of agent work outgrows what your team can read, a useful first reader is not a person: route the diff through a model that summarizes it in the same four-pass shape before a human opens it. Quanta Inference puts text, image, video, audio and embedding models behind one OpenAI-compatible API key, prepaid and priced in US dollars, so a summarization step and the rest of your pipeline share one integration: Quanta Inference.