Agent review said "good to go" (and missed it)
智能体审查说“可以合并”（却漏掉了问题）
Several agent review passes approve a change that a human then finds broken, unnecessary or architecturally wrong. More passes from similar models don't add independent checks.
Applies to: Claude Code · Cursor · Codex · Copilot agent
Symptom
Every bot left comments or approved, yet a human finds a breaking change, a duplicated subsystem or thousands of lines that shouldn't exist. Reviews mix real bugs with false positives, find something new on every run, or comment on an empty diff.
Why it happens
Thorsten Ball's post asked whether humans really catch what three models checking four times each would miss, and many replies gave concrete counterexamples. @MilesDigitek had agents miss the same architectural mistake four times in a row: same-family reviewers are "one review sampled four times". @joshmanders has colleagues finding issues all the AI bots had stamped "Good to go". @BradleySkuse ran several Claude, Codex, Cursor, Copilot and Bugbot review cycles, then a doc-only update made Bugbot report a critical issue, and a colleague's AI review found a dozen things, two of them breaking. @paolocargnin stopped about 6k lines of useless code that had passed many review cycles, including agents. @octavianslash asked Claude to review a PR that duplicated his framework's middleware system, and it waved it through. @DonutsWithData sees bugs on every run mixed with many false positives, and higher-level issues missed. @michalkuncio notes self-review never converges. @danfarrdotcom: "my agents reviewed an empty PR once. one suggested a better title".
Example
Illustrative, combining reports from the thread.
textreviewer-1 (model A): LGTM, 2 nits
reviewer-2 (model A, new seed): LGTM, suggests better title
reviewer-3 (model B): 1 critical (false positive), 3 nits
human: this adds a second middleware dispatcher next to the existing one. Reject.
Copy-paste fixes
Pick the files for the agents you use. Snippets are in English so they work in any repo.
Claude Code: add to CLAUDE.md
CLAUDE.md## Treat agent review as one signal, not a gate
- Never report a change as "reviewed" because review agents approved it. Say which reviewers ran,
on which model, and what they did not check (architecture, whether the change is needed).
- For review, use a fresh context that sees only the request and the diff, and if possible a
different model family from the one that wrote the code.
- Every finding needs file:line, severity and a way to verify it. Drop findings you can't verify.
- Stop after one fix round per finding. If a new round only finds new low-severity items,
report them instead of fixing them, and hand the decision to a human.
- Always ask: does this change need to exist, and does it reuse what is already in the codebase?
Codex, Copilot coding agent and others: add to AGENTS.md
AGENTS.md## Treat agent review as one signal, not a gate
- Never report a change as "reviewed" because review agents approved it. Say which reviewers ran,
on which model, and what they did not check (architecture, whether the change is needed).
- For review, use a fresh context that sees only the request and the diff, and if possible a
different model family from the one that wrote the code.
- Every finding needs file:line, severity and a way to verify it. Drop findings you can't verify.
- Stop after one fix round per finding. If a new round only finds new low-severity items,
report them instead of fixing them, and hand the decision to a human.
- Always ask: does this change need to exist, and does it reuse what is already in the codebase?
Cursor: project rule
.cursor/rules/agent-review-rubber-stamp.mdc---
description: "AgentFails: Agent review said 'good to go' (and missed it)"
alwaysApply: true
---
## Treat agent review as one signal, not a gate
- Never report a change as "reviewed" because review agents approved it. Say which reviewers ran,
on which model, and what they did not check (architecture, whether the change is needed).
- For review, use a fresh context that sees only the request and the diff, and if possible a
different model family from the one that wrote the code.
- Every finding needs file:line, severity and a way to verify it. Drop findings you can't verify.
- Stop after one fix round per finding. If a new round only finds new low-severity items,
report them instead of fixing them, and hand the decision to a human.
- Always ask: does this change need to exist, and does it reuse what is already in the codebase?
Fresh-context review skill (subagent reviews the diff, not the chat)
Follows the habit Anthropic describes in its code-review post: verify first, then have a subagent review the diff with fresh context. Named review-diff to avoid clashing with built-in or popular /code-review skills.
.claude/skills/review-diff/SKILL.md---
name: review-diff
description: Review the current diff with fresh context before a PR. Use after /verify passes.
---
# Review the diff
Start a subagent that has NOT seen this conversation. Give it only:
- the original request (one paragraph, verbatim),
- the output of `git diff origin/main...HEAD`,
- the names of the test/build commands.
Ask it to report, with file:line for each finding:
1. **Wrong feature**: does the diff do what the request asked? Quote the request line each change serves.
2. **Bugs**: logic errors, missing edge cases (empty, null, boundary), unhandled errors,
unawaited promises, invented APIs or packages, swallowed exceptions.
3. **Overreach**: changes no request line needs (refactors, renames, formatting, dependency or
config edits, public API changes, weakened tests).
4. **Evidence gaps**: claims in the PR description that no command output supports.
Then fix real findings, or list them under "Known issues" in the PR. Do not argue with the reviewer
in the PR text; either fix or explain in one line.
Shared /verify skill for Claude Code
Commit it to the repo so the whole team (and every agent session) runs the same checks. Replace the commands with your project's real ones.
.claude/skills/verify/SKILL.md---
name: verify
description: Verify the current change before calling it done. Use before every commit, PR, or "done" message.
---
# Verify
Run every step and paste the real output. Never summarize a step you did not run.
1. **Scope**: run `git diff --stat` and `git status --short`. For every changed file, say which part
of the request needs it. Any file you cannot justify gets reverted, or listed under
"Unrequested changes" for the user to decide.
2. **Build and types**: `npm run build` (or the project's build command).
3. **Tests**: `npm test`. Report pass/fail counts. If a test was edited, skipped or deleted,
say so explicitly with the reason.
4. **Behavior**: exercise the changed behavior once for real (a curl, a CLI run, a screenshot)
and show what happened.
5. **Report** in this format:
- Requested: <one line>
- Changed files: <list, each with its reason>
- Checks run: <command -> result>
- Not done / unsure: <list, or "none">
Sources and credit
- @MilesDigitek on X: the same architectural mistake missed four times; "one review sampled four times"
- @joshmanders on X: issues that every AI bot stamped "Good to go"
- @BradleySkuse on X: many AI review cycles, then a critical issue and two breaking changes found later
- @paolocargnin on X: ~6k lines of useless code that passed review cycles, including agents
- @octavianslash on X: Claude review waved through a duplicate middleware dispatch layer
- @DonutsWithData on X: bugs on every run, mixed with false positives, higher-level issues missed
- @michalkuncio on X: self-review finds new issues every time you ask
- @danfarrdotcom on X: agents reviewed an empty PR; one suggested a better title
- @thorstenball on X: "Why are you still using pull requests?" (the thread)