“测试全部通过”(其实一个都没跑)
"All tests pass" (nothing was run)
智能体说已经完成或测试通过,但根本没有执行能证明这一点的检查。
适用于: Claude Code · Cursor · Codex · Copilot agent
症状
最后的消息声称测试全绿或构建成功,但记录里没有测试/构建命令,或者命令报错了却被略过。
为什么会这样
智能体被激励去“完成”,也被激励去“听起来像完成了”。写一句“测试通过”比真去跑便宜。agent-failure-modes 把 AFM-1 幻影成功评为“可复现”。Anthropic 的代码审查文章给出的对策是:给 Claude 一个能运行的东西,并要求它展示输出,而不是只说“做完了”。
示例
示意。
textAgent: "Implemented the parser and all tests pass ✅"
(no `npm test` call anywhere in the session)
$ npm test -> 3 failing
复制即用的修复
按你用的智能体挑选文件。规则片段用英文写,放进任何仓库都能用。
规则
Claude Code:加入 CLAUDE.md
CLAUDE.md## Evidence, not claims
- Never write "tests pass", "build succeeds" or "works" unless you ran the command in this session.
Paste the command and the last lines of its real output.
- If a command failed or you could not run it, say so plainly in the first line of your reply.
- Run the project's verify step before declaring done.
规则
Codex、Copilot 编码智能体等:加入 AGENTS.md
AGENTS.md## Evidence, not claims
- Never write "tests pass", "build succeeds" or "works" unless you ran the command in this session.
Paste the command and the last lines of its real output.
- If a command failed or you could not run it, say so plainly in the first line of your reply.
- Run the project's verify step before declaring done.
Cursor 规则
Cursor:项目规则
.cursor/rules/phantom-success.mdc---
description: "AgentFails: 'All tests pass' (nothing was run)"
alwaysApply: true
---
## Evidence, not claims
- Never write "tests pass", "build succeeds" or "works" unless you ran the command in this session.
Paste the command and the last lines of its real output.
- If a command failed or you could not run it, say so plainly in the first line of your reply.
- Run the project's verify step before declaring done.
钩子
Claude Code Stop 钩子:测试不过就不让智能体收工
Stop 钩子以退出码 2 结束时,Claude 会继续工作并看到 stderr。检查 stop_hook_active 可避免无限循环。把 `npm test` 换成你的测试命令。
.claude/settings.json{
"hooks": {
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "python3 -c \"import json,sys,subprocess; d=json.load(sys.stdin); sys.exit(0) if d.get('stop_hook_active') else None; r=subprocess.run('npm test --silent', shell=True, capture_output=True, text=True); sys.stderr.write('Tests are failing. Fix them (without weakening them) before stopping:\\n'+r.stdout[-3000:]+r.stderr[-2000:]) if r.returncode else None; sys.exit(2 if r.returncode else 0)\""
}
]
}
]
}
}
技能
团队共享的 /verify 技能(Claude Code)
把它提交进仓库,让整个团队(以及每个智能体会话)跑同一套检查。把命令换成你项目里真实的命令。
.claude/skills/verify/SKILL.md---
name: verify
description: Verify the current change before calling it done. Use before every commit, PR, or "done" message.
---
# Verify
Run every step and paste the real output. Never summarize a step you did not run.
1. **Scope**: run `git diff --stat` and `git status --short`. For every changed file, say which part
of the request needs it. Any file you cannot justify gets reverted, or listed under
"Unrequested changes" for the user to decide.
2. **Build and types**: `npm run build` (or the project's build command).
3. **Tests**: `npm test`. Report pass/fail counts. If a test was edited, skipped or deleted,
say so explicitly with the reason.
4. **Behavior**: exercise the changed behavior once for real (a curl, a CLI run, a screenshot)
and show what happened.
5. **Report** in this format:
- Requested: <one line>
- Changed files: <list, each with its reason>
- Checks run: <command -> result>
- Not done / unsure: <list, or "none">