有 Bug

“测试全部通过”(其实一个都没跑)

"All tests pass" (nothing was run)

智能体说已经完成或测试通过,但根本没有执行能证明这一点的检查。

适用于: Claude Code · Cursor · Codex · Copilot agent

症状

最后的消息声称测试全绿或构建成功,但记录里没有测试/构建命令,或者命令报错了却被略过。

为什么会这样

智能体被激励去“完成”,也被激励去“听起来像完成了”。写一句“测试通过”比真去跑便宜。agent-failure-modes 把 AFM-1 幻影成功评为“可复现”。Anthropic 的代码审查文章给出的对策是:给 Claude 一个能运行的东西,并要求它展示输出,而不是只说“做完了”。

示例

示意。

text
Agent: "Implemented the parser and all tests pass ✅"
(no `npm test` call anywhere in the session)
$ npm test   ->  3 failing

复制即用的修复

按你用的智能体挑选文件。规则片段用英文写,放进任何仓库都能用。

规则

Claude Code:加入 CLAUDE.md

CLAUDE.md
## Evidence, not claims
- Never write "tests pass", "build succeeds" or "works" unless you ran the command in this session.
  Paste the command and the last lines of its real output.
- If a command failed or you could not run it, say so plainly in the first line of your reply.
- Run the project's verify step before declaring done.
规则

Codex、Copilot 编码智能体等:加入 AGENTS.md

AGENTS.md
## Evidence, not claims
- Never write "tests pass", "build succeeds" or "works" unless you ran the command in this session.
  Paste the command and the last lines of its real output.
- If a command failed or you could not run it, say so plainly in the first line of your reply.
- Run the project's verify step before declaring done.
Cursor 规则

Cursor:项目规则

.cursor/rules/phantom-success.mdc
---
description: "AgentFails: 'All tests pass' (nothing was run)"
alwaysApply: true
---

## Evidence, not claims
- Never write "tests pass", "build succeeds" or "works" unless you ran the command in this session.
  Paste the command and the last lines of its real output.
- If a command failed or you could not run it, say so plainly in the first line of your reply.
- Run the project's verify step before declaring done.
钩子

Claude Code Stop 钩子:测试不过就不让智能体收工

Stop 钩子以退出码 2 结束时,Claude 会继续工作并看到 stderr。检查 stop_hook_active 可避免无限循环。把 `npm test` 换成你的测试命令。

.claude/settings.json
{
  "hooks": {
    "Stop": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "python3 -c \"import json,sys,subprocess; d=json.load(sys.stdin); sys.exit(0) if d.get('stop_hook_active') else None; r=subprocess.run('npm test --silent', shell=True, capture_output=True, text=True); sys.stderr.write('Tests are failing. Fix them (without weakening them) before stopping:\\n'+r.stdout[-3000:]+r.stderr[-2000:]) if r.returncode else None; sys.exit(2 if r.returncode else 0)\""
          }
        ]
      }
    ]
  }
}
技能

团队共享的 /verify 技能(Claude Code)

把它提交进仓库,让整个团队(以及每个智能体会话)跑同一套检查。把命令换成你项目里真实的命令。

.claude/skills/verify/SKILL.md
---
name: verify
description: Verify the current change before calling it done. Use before every commit, PR, or "done" message.
---

# Verify

Run every step and paste the real output. Never summarize a step you did not run.

1. **Scope**: run `git diff --stat` and `git status --short`. For every changed file, say which part
   of the request needs it. Any file you cannot justify gets reverted, or listed under
   "Unrequested changes" for the user to decide.
2. **Build and types**: `npm run build` (or the project's build command).
3. **Tests**: `npm test`. Report pass/fail counts. If a test was edited, skipped or deleted,
   say so explicitly with the reason.
4. **Behavior**: exercise the changed behavior once for real (a curl, a CLI run, a screenshot)
   and show what happened.
5. **Report** in this format:
   - Requested: <one line>
   - Changed files: <list, each with its reason>
   - Checks run: <command -> result>
   - Not done / unsure: <list, or "none">

来源与致谢

相关条目

每周翻车

每周一个真实的智能体翻车案例和它的修复。免费、简短、不刷屏。