有 Bug

智能体审查说“可以合并”(却漏掉了问题)

Agent review said "good to go" (and missed it)

多轮智能体审查都批准了一个改动,结果人一看就发现它有问题、多余或架构不对。同类模型多审几遍,并不等于多了几次独立检查。

适用于: Claude Code · Cursor · Codex · Copilot agent

症状

每个机器人都留了评论或批准了,结果人还是发现了破坏性改动、重复的子系统,或者成千上万行本不该存在的代码。审查把真 bug 和误报混在一起,每次运行都能找出新东西,甚至会对空 diff 发表意见。

为什么会这样

Thorsten Ball 在帖子里问:三个模型各审四遍都发现不了的问题,人真能发现吗?很多回复给出了具体反例。@MilesDigitek 的智能体连续四次漏掉同一个架构错误:同一家族的审查者是“同一次审查采样了四遍”。@joshmanders 的同事发现过所有 AI 机器人都标了“Good to go”的问题。@BradleySkuse 跑了好几轮 Claude、Codex、Cursor、Copilot 和 Bugbot 的审查,后来只改了文档,Bugbot 却报出一个严重问题;同事用 AI 再审一遍,又找出十几处,其中两处是破坏性改动。@paolocargnin 拦下了大约 6k 行无用代码,它们已经通过了包括智能体在内的多轮审查。@octavianslash 让 Claude 审查一个重复实现了框架中间件系统的 PR,Claude 直接放行。@DonutsWithData 说 AI 审查每次都能找到 bug,但混着大量误报,还会漏掉更高层的问题。@michalkuncio 指出自我审查永远不会收敛。@danfarrdotcom:“我的智能体审查过一个空 PR,其中一个还建议换个更好的标题。”

示例

示意,综合了串里的几条报告。

text
reviewer-1 (model A): LGTM, 2 nits
reviewer-2 (model A, new seed): LGTM, suggests better title
reviewer-3 (model B): 1 critical (false positive), 3 nits
human: this adds a second middleware dispatcher next to the existing one. Reject.

复制即用的修复

按你用的智能体挑选文件。规则片段用英文写,放进任何仓库都能用。

规则

Claude Code:加入 CLAUDE.md

CLAUDE.md
## Treat agent review as one signal, not a gate
- Never report a change as "reviewed" because review agents approved it. Say which reviewers ran,
  on which model, and what they did not check (architecture, whether the change is needed).
- For review, use a fresh context that sees only the request and the diff, and if possible a
  different model family from the one that wrote the code.
- Every finding needs file:line, severity and a way to verify it. Drop findings you can't verify.
- Stop after one fix round per finding. If a new round only finds new low-severity items,
  report them instead of fixing them, and hand the decision to a human.
- Always ask: does this change need to exist, and does it reuse what is already in the codebase?
规则

Codex、Copilot 编码智能体等:加入 AGENTS.md

AGENTS.md
## Treat agent review as one signal, not a gate
- Never report a change as "reviewed" because review agents approved it. Say which reviewers ran,
  on which model, and what they did not check (architecture, whether the change is needed).
- For review, use a fresh context that sees only the request and the diff, and if possible a
  different model family from the one that wrote the code.
- Every finding needs file:line, severity and a way to verify it. Drop findings you can't verify.
- Stop after one fix round per finding. If a new round only finds new low-severity items,
  report them instead of fixing them, and hand the decision to a human.
- Always ask: does this change need to exist, and does it reuse what is already in the codebase?
Cursor 规则

Cursor:项目规则

.cursor/rules/agent-review-rubber-stamp.mdc
---
description: "AgentFails: Agent review said 'good to go' (and missed it)"
alwaysApply: true
---

## Treat agent review as one signal, not a gate
- Never report a change as "reviewed" because review agents approved it. Say which reviewers ran,
  on which model, and what they did not check (architecture, whether the change is needed).
- For review, use a fresh context that sees only the request and the diff, and if possible a
  different model family from the one that wrote the code.
- Every finding needs file:line, severity and a way to verify it. Drop findings you can't verify.
- Stop after one fix round per finding. If a new round only finds new low-severity items,
  report them instead of fixing them, and hand the decision to a human.
- Always ask: does this change need to exist, and does it reuse what is already in the codebase?
技能

全新上下文的审查技能(子智能体只看 diff,不看聊天记录)

沿用 Anthropic 代码审查文章里的做法:先验证,再让一个上下文干净的子智能体审 diff。命名为 review-diff,避免和内置或流行的 /code-review 技能重名。

.claude/skills/review-diff/SKILL.md
---
name: review-diff
description: Review the current diff with fresh context before a PR. Use after /verify passes.
---

# Review the diff

Start a subagent that has NOT seen this conversation. Give it only:
- the original request (one paragraph, verbatim),
- the output of `git diff origin/main...HEAD`,
- the names of the test/build commands.

Ask it to report, with file:line for each finding:
1. **Wrong feature**: does the diff do what the request asked? Quote the request line each change serves.
2. **Bugs**: logic errors, missing edge cases (empty, null, boundary), unhandled errors,
   unawaited promises, invented APIs or packages, swallowed exceptions.
3. **Overreach**: changes no request line needs (refactors, renames, formatting, dependency or
   config edits, public API changes, weakened tests).
4. **Evidence gaps**: claims in the PR description that no command output supports.

Then fix real findings, or list them under "Known issues" in the PR. Do not argue with the reviewer
in the PR text; either fix or explain in one line.
技能

团队共享的 /verify 技能(Claude Code)

把它提交进仓库,让整个团队(以及每个智能体会话)跑同一套检查。把命令换成你项目里真实的命令。

.claude/skills/verify/SKILL.md
---
name: verify
description: Verify the current change before calling it done. Use before every commit, PR, or "done" message.
---

# Verify

Run every step and paste the real output. Never summarize a step you did not run.

1. **Scope**: run `git diff --stat` and `git status --short`. For every changed file, say which part
   of the request needs it. Any file you cannot justify gets reverted, or listed under
   "Unrequested changes" for the user to decide.
2. **Build and types**: `npm run build` (or the project's build command).
3. **Tests**: `npm test`. Report pass/fail counts. If a test was edited, skipped or deleted,
   say so explicitly with the reason.
4. **Behavior**: exercise the changed behavior once for real (a curl, a CLI run, a screenshot)
   and show what happened.
5. **Report** in this format:
   - Requested: <one line>
   - Changed files: <list, each with its reason>
   - Checks run: <command -> result>
   - Not done / unsure: <list, or "none">

来源与致谢

相关条目

每周翻车

每周一个真实的智能体翻车案例和它的修复。免费、简短、不刷屏。