"Fully migrated" (it wasn't)
“全部迁移完了”（其实没有）
The agent reports a sweep (rename, migration, API swap) as complete when it only covered the occurrences it happened to see.
Applies to: Claude Code · Cursor · Codex · Copilot agent
Symptom
"Updated all 8 call sites." There were 9; the ninth lives in a shell string, a config file, or a file past a search-result cutoff, untouched and unmentioned.
Why it happens
Search output gets truncated, and dynamic references (strings, templates, configs) don't show up in symbol search. The agent treats "everything I found" as "everything". agent-failure-modes calls this AFM-12 Coverage Overclaim.
Example
Illustrative.
bash# Agent: "Renamed getUser -> fetchUser everywhere."
$ rg -n "getUser" .
scripts/backfill.sh:12: node -e "require('./dist/api').getUser(process.argv[1])"
Copy-paste fixes
Pick the files for the agents you use. Snippets are in English so they work in any repo.
Claude Code: add to CLAUDE.md
CLAUDE.md## Sweeping changes need a count, before and after
- For any rename, migration or "replace everywhere" task, first run a repo-wide search and report the
total count and the list of locations (include strings, configs, scripts and docs).
- After the change, re-run the same search and show it returns zero (or explain each remaining hit).
- Never say "all" or "everywhere" without that before/after evidence. If a search output was truncated, say so.
Codex, Copilot coding agent and others: add to AGENTS.md
AGENTS.md## Sweeping changes need a count, before and after
- For any rename, migration or "replace everywhere" task, first run a repo-wide search and report the
total count and the list of locations (include strings, configs, scripts and docs).
- After the change, re-run the same search and show it returns zero (or explain each remaining hit).
- Never say "all" or "everywhere" without that before/after evidence. If a search output was truncated, say so.
Cursor: project rule
.cursor/rules/coverage-overclaim.mdc---
description: "AgentFails: 'Fully migrated' (it wasn't)"
alwaysApply: true
---
## Sweeping changes need a count, before and after
- For any rename, migration or "replace everywhere" task, first run a repo-wide search and report the
total count and the list of locations (include strings, configs, scripts and docs).
- After the change, re-run the same search and show it returns zero (or explain each remaining hit).
- Never say "all" or "everywhere" without that before/after evidence. If a search output was truncated, say so.
Shared /verify skill for Claude Code
Commit it to the repo so the whole team (and every agent session) runs the same checks. Replace the commands with your project's real ones.
.claude/skills/verify/SKILL.md---
name: verify
description: Verify the current change before calling it done. Use before every commit, PR, or "done" message.
---
# Verify
Run every step and paste the real output. Never summarize a step you did not run.
1. **Scope**: run `git diff --stat` and `git status --short`. For every changed file, say which part
of the request needs it. Any file you cannot justify gets reverted, or listed under
"Unrequested changes" for the user to decide.
2. **Build and types**: `npm run build` (or the project's build command).
3. **Tests**: `npm test`. Report pass/fail counts. If a test was edited, skipped or deleted,
say so explicitly with the reason.
4. **Behavior**: exercise the changed behavior once for real (a curl, a CLI run, a screenshot)
and show what happened.
5. **Report** in this format:
- Requested: <one line>
- Changed files: <list, each with its reason>
- Checks run: <command -> result>
- Not done / unsure: <list, or "none">