Reviewing AI-Generated Pull Requests: What Changes When the Author Is an Agent

For most of the history of code review, the scarce resource was the code. Someone had to sit down and write the thing, and review was the cheap step at the end. That ratio has inverted. An agent can open a pull request in eight minutes that takes forty minutes to read properly, and it can do that several times before lunch.

The uncomfortable data point is what teams do about it. A 2026 study of AI-generated pull requests on GitHub found that most agent-authored PRs receive no review at all — and when they are reviewed, the review is largely conducted by other agents rather than humans, with human involvement showing up as "agent steering" rather than standalone evaluation. Meanwhile DORA's 2025 research found that AI adoption now correlates with higher delivery throughput and higher delivery instability at the same time, with 30% of developers reporting little or no trust in AI-generated code. More code, moving faster, trusted less, reviewed less.

The instinct is to review agent PRs harder — same process, more diligence. That doesn't scale, and it isn't even aimed at the right target, because agent PRs fail differently than human PRs do. This is a guide to reviewing them well: what actually goes wrong, the order to look for it in, and how to move most of the work earlier so that less of it lands on a reviewer at 5pm.

Agent PRs fail differently#

A human contributor and a coding agent produce different defect distributions, and knowing the difference tells you where to spend attention.

A human who doesn't understand the codebase writes code that looks uncertain. Variables are named vaguely, the structure is awkward, there's a comment saying "not sure if this is the right place for this." The uncertainty is legible, and reviewers have spent twenty years learning to read it.

An agent that doesn't understand the codebase writes code that looks finished. Consistent naming, tidy structure, tests that pass, a well-formatted description. The confidence of the output carries no information about whether the agent had the right context, because the agent generates confident-looking code either way. Every visual signal reviewers use to allocate attention has been decoupled from the thing it used to predict.

That last row deserves emphasis. On a human PR, the description is weak evidence but honest evidence. On an agent PR, the description is frequently a restatement of the prompt rather than a summary of the diff, which makes it actively misleading as a review shortcut. Read the diff before the description, or you will anchor on a summary of something that didn't happen.

Where the review effort should actually sit#

The mistake teams make is treating "review the agent's PR" as one undifferentiated block of human attention. It isn't. Most of what needs checking is mechanical, and the mechanical parts should never reach a person.

flowchart TD
  A["Agent writes the change"] --> B["Pre-PR self-review: conventions, docs, new surface"]
  B -->|must-fix findings| A
  B --> C["PR opened"]
  C --> D["Automated gates: tests, types, lint, security scan"]
  D -->|fails| A
  D --> E["Human review: intent, boundaries, risk, taste"]
  E -->|changes requested| A
  E --> F["Merge"]

Everything in the first two boxes is work a human should never do on an agent PR. Whether the code compiles, passes tests, satisfies the linter, and follows a written convention are all questions with mechanical answers — and mechanical answers should come back in ninety seconds from a check, not in six hours from a colleague.

What's left for the human is the part with no mechanical answer: is this the right change, does it belong here, what does it break that nothing tests, and is anyone going to understand it in March. That's a smaller job than "review this PR," and it's a job a person is uniquely good at. The goal of everything below is to protect that box by shrinking the ones before it.

A review order that catches agent-shaped defects#

When an agent PR does reach you, read it in this order. The sequence matters — it front-loads the cheap questions that let you reject early.

1. Size and scope first, before any code. How many files, how many concerns? An agent asked to fix a bug in the webhook handler that also touched nine unrelated files did something you didn't ask for. This is the cheapest possible rejection and it should be nearly reflexive. A PR that does two things is not one PR; send it back before you read a line of it.

2. What did it delete? Deletions are where the real risk lives, and they're the easiest thing to skim past in a large diff. Agents remove code they judged unreachable, drop a branch that looked redundant, and delete tests that failed. Review every deletion deliberately. "The tests pass now" is entirely compatible with "the test that would have caught this is gone."

3. Does this duplicate something? The single most common agent defect is a clean, correct, well-tested reimplementation of a utility that already exists forty lines away in another file. Nothing in your pipeline catches this — it compiles, it passes, it lints, and it quietly doubles the surface area of the codebase. This is one of the highest-value things a human reviewer contributes, because it requires knowing what's already there.

4. Do the tests actually assert anything? Agents are good at producing tests that pass and less reliable at producing tests that would fail if the code were wrong. Look for assertions that restate the implementation, mocks so complete that no real logic executes, and the classic assertion that a result is merely defined. A useful check: mentally break the implementation and ask whether any test would notice.

5. Boundaries and blast radius. Did it touch auth, permissions, migrations, billing, secrets, or anything that talks to a third party? Agents don't have a sense of which lines are load-bearing, so a schema migration and a copy change arrive in the same diff with the same tone. If your repository has areas that require a second reviewer, agent PRs are exactly the case that rule exists for — and if the agent has tools that can act on other systems, MCP security risks is worth reading before you widen what it can reach.

6. Only now: does the description match the diff? Compare them explicitly. Where they disagree, the diff is true and the description is a hypothesis. A mismatch is usually the most informative thing in the whole PR, because it means the agent's model of what it did diverged from what it did.

7. Did the docs move with the code? If the change altered behavior, an API surface, a command, or an operational contract, the documentation describing it is now wrong. Not "will need updating later" — wrong, right now, and being served to the next agent that asks about this area. More on why that compounds below.

The defect class nobody catches by reading harder#

Steps 1 through 7 catch things that are visible in the diff. The category that gets through is the one where the code is correct with respect to what the agent knew, and what the agent knew was wrong.

An agent implements a rate limiter using a Redis pattern your team abandoned last quarter. The code is clean. The tests pass. The reviewer, who joined in June, has no reason to think anything is off. It ships, and now there are two rate-limiting patterns in the codebase, one of which is officially deprecated in a document nobody reads. Next month a different agent greps the codebase, finds the new instance of the old pattern, and treats it as the house style.

Reading the diff harder cannot catch this, because there is nothing in the diff to catch. The defect entered upstream, in the agent's context, and the PR is just where it became visible. Our field guide to context engineering breaks this down as a clash: two sources in the agent's context disagree — a doc says X, the code says Y — and the model picks one with no way to know which is authoritative. Often it picks the doc, because docs read like statements of intent.

This is documentation drift with a much shorter fuse. The gap between what your docs say and what your code does used to cost you a confused new hire who would eventually ask someone. Now it costs you a confident pull request, written in minutes, that encodes the wrong pattern into the codebase where the next agent will find it and copy it. The feedback loop that used to run in weeks runs in hours, and it runs unattended.

Which reframes the review problem. If a defect class is invisible at review time, the answer is not a better reviewer. It's fixing the input, then verifying it earlier.

Moving the work left#

Three shifts, in rough order of payoff.

Make the always-on context correct and small. Your AGENTS.md or CLAUDE.md is read on every task, so an error in it is an error in every PR the agent opens. Audit it against reality on a schedule — the test command, the directory map, the boundaries. Keep it short enough that auditing it is a ten-minute job, and push everything larger into something the agent queries on demand rather than carries in every window. We covered that tradeoff in managing AI agent context without the bloat and the Skill vs. MCP server decision.

Have the agent review its own change before it opens the PR. This is the highest-leverage change available, because the agent has the full file contents in context at the moment it finishes editing — the cheapest possible time to catch a convention breach is before the commit, not after a human has read the diff. Moxie Docs exposes this as an MCP tool, moxie.review_change: the agent passes the proposed new state of each changed file, and gets back a severity-ranked verdict — clean, warnings, or must-fix — covering convention breaches, docs the change makes factually false, net-new API surface added without documentation, and doc references broken by deletions. It's read-only, and each finding carries a location and a fix, so the agent can correct and re-run until the verdict comes back clean. The PR a human eventually sees is the one that already passed.

Put the mechanical checks in CI, not in a person's head. Conventions your team actually enforces should fail a check, not generate a review comment. Moxie Docs posts an advisory Conventions check-run on pull requests and keeps its findings in a single edit-in-place comment rather than a new one on every push. Whether a given rule is advisory or blocking is your call; what matters is that the answer arrives automatically and identically every time, instead of depending on which reviewer got assigned.

The through-line: an agent's PR is only as good as the context it worked from, and that context is only as good as the docs behind it. Moxie Docs indexes your GitHub repository into living, source-cited documentation, serves your conventions, docs, gaps, and verified commands to coding agents over MCP, and opens reviewable Cleanup PRs when the docs fall behind the code — it proposes, you merge, never the reverse. You can see what the first index turns up on your own repository on the free plan: one repository, no card required. For the authoring side of the same loop, ship cleaner PRs with agentic AI covers giving the agent good context before it writes anything.

Measuring whether it's working#

Review quality is famously hard to measure, but agent PRs give you a few signals that are cheap to watch and hard to game.

• Share of agent PRs merged with no human comment. If this is climbing toward the study's finding, your review process is nominal. This is the headline number.
• Time-to-first-review on agent PRs vs. human PRs. A widening gap means reviewers are avoiding them, usually because they're too large. Fix the size, not the reviewers.
• Reverts and follow-up fixes within seven days. The clearest evidence that review isn't catching what it should — and, per DORA, the instability that AI throughput tends to buy you.
• Duplicate-implementation catches. Count them. A rising count means your agents can't see what already exists, which is a retrieval problem, not a review problem.
• Must-fix findings at self-review time. If you run a pre-PR check, its findings are a free readout on which conventions your agents keep missing — and therefore which ones are absent from context or written ambiguously.
• Docs changed per behavior change. If behavior changes ship without doc changes, you're accruing documentation debt at agent speed, and every future agent PR gets a little worse.

None of these need a dashboard. A ten-minute look at the last twenty merged PRs will tell you most of it.

Frequently asked questions#

How should reviewing AI-generated pull requests differ from reviewing human ones?#

Shift attention away from style and mechanics, which automation handles better, and toward the things an agent has no judgment about: whether the change duplicates existing code, whether deletions were safe, whether tests assert anything real, and whether the change belongs in this PR at all. Also read the diff before the description, since agent PR descriptions often restate the prompt rather than summarize the change.

Can AI review AI-generated code?#

Partially, and it's worth doing for the mechanical layer — an agent reviewer is good at conventions, missing tests, obvious bugs, and undocumented new surface, and it never gets tired on the twentieth PR of the day. What it can't do is decide whether the change was the right thing to build, or notice that it conflicts with a decision the team made in a meeting. Use agent review to shrink the human's job, not to replace it. The failure state is a pipeline where agents write, agents review, and no human ever formed an opinion.

Why do AI-generated PRs take longer to review?#

They're often larger, they touch more files than the task required, and they carry none of the usual signals about where the author was unsure. A human PR tells you where to look; an agent PR is uniformly confident, so a reviewer has to give the whole diff even attention. The fix is upstream — smaller, single-concern PRs — not more reviewer stamina.

Should agent-authored pull requests be merged automatically?#

Not on anything load-bearing. Automated gates should be able to reject without a human, which is where the time savings actually are, but a merge is an assertion that someone decided this change was right. For low-risk mechanical categories — dependency bumps, formatting, generated files — auto-merge on green checks is reasonable. For anything that changes behavior, keep a human in the loop.

What's the single highest-leverage change for a team drowning in agent PRs?#

Cut PR size, then move convention and documentation checks to before the commit. Most review pain is a size problem wearing a quality costume: a 400-line diff spanning four concerns is unreviewable no matter how good the reviewer is. Constrain the agent to one concern per PR, and have it self-review against your conventions before it opens anything.

The short version#

Review didn't get harder because agents write bad code. It got harder because agents write plausible code at a volume that overwhelms the human step, and because the signals reviewers evolved to read — hesitation, awkwardness, a hedging comment — don't exist in agent output. The response isn't to read harder. It's to let machines answer the mechanical questions before a person sees the PR, to keep human attention on scope, deletions, duplication, and blast radius, and to fix the upstream context so the defect class no reviewer can see stops being generated in the first place. Teams that keep up aren't reviewing faster. They're reviewing less, because less of what reaches them needed a human at all.