A fix for a crash it never reproduced
修了一个从没复现过的崩溃
The agent adds a guard or branch "to prevent" a failure it never managed to trigger, so nobody knows whether the fix addresses the real cause, and reviewers skim past the admission in the description.
Applies to: Claude Code · Cursor · Codex · Copilot agent
Symptom
The PR description says something like "could not reproduce, but this should prevent it". There is no failing test or captured input before the change, and no proof after. The new branch has never executed.
Why it happens
@roulerdelaroute describes a Claude PR that added a code branch to stop a bad packet from causing a crash, while it "hadn't managed to repro that bad packet". More than eight review agents missed that line in the description and nitpicked code that had never run. Writing a plausible guard is easy; proving the cause takes a reproduction, and without one the fix is a guess that reads like a solution.
Example
Illustrative, based on @roulerdelaroute's description.
diffPR: "Prevent crash on malformed packet. Note: could not reproduce the bad packet."
function parsePacket(buf) {
+ if (buf.length < HEADER_LEN + 2) return null; // guess at the cause, never executed
const len = buf.readUInt16BE(HEADER_LEN);
...
Copy-paste fixes
Pick the files for the agents you use. Snippets are in English so they work in any repo.
Claude Code: add to CLAUDE.md
CLAUDE.md## Reproduce before you fix
- Before fixing a bug, reproduce it: a failing test, a captured input, or a command that shows
the failure. Paste the failing output.
- If you cannot reproduce it, say so in the first line of your reply and do not present a change
as the fix. Offer it as a hypothesis, with what evidence would confirm it.
- After the fix, run the same reproduction and paste the passing output.
- Reviewers: check the description for "could not reproduce" or "should prevent" before reading the code.
Codex, Copilot coding agent and others: add to AGENTS.md
AGENTS.md## Reproduce before you fix
- Before fixing a bug, reproduce it: a failing test, a captured input, or a command that shows
the failure. Paste the failing output.
- If you cannot reproduce it, say so in the first line of your reply and do not present a change
as the fix. Offer it as a hypothesis, with what evidence would confirm it.
- After the fix, run the same reproduction and paste the passing output.
- Reviewers: check the description for "could not reproduce" or "should prevent" before reading the code.
Cursor: project rule
.cursor/rules/fix-without-repro.mdc---
description: "AgentFails: A fix for a crash it never reproduced"
alwaysApply: true
---
## Reproduce before you fix
- Before fixing a bug, reproduce it: a failing test, a captured input, or a command that shows
the failure. Paste the failing output.
- If you cannot reproduce it, say so in the first line of your reply and do not present a change
as the fix. Offer it as a hypothesis, with what evidence would confirm it.
- After the fix, run the same reproduction and paste the passing output.
- Reviewers: check the description for "could not reproduce" or "should prevent" before reading the code.
Shared /verify skill for Claude Code
Commit it to the repo so the whole team (and every agent session) runs the same checks. Replace the commands with your project's real ones.
.claude/skills/verify/SKILL.md---
name: verify
description: Verify the current change before calling it done. Use before every commit, PR, or "done" message.
---
# Verify
Run every step and paste the real output. Never summarize a step you did not run.
1. **Scope**: run `git diff --stat` and `git status --short`. For every changed file, say which part
of the request needs it. Any file you cannot justify gets reverted, or listed under
"Unrequested changes" for the user to decide.
2. **Build and types**: `npm run build` (or the project's build command).
3. **Tests**: `npm test`. Report pass/fail counts. If a test was edited, skipped or deleted,
say so explicitly with the reason.
4. **Behavior**: exercise the changed behavior once for real (a curl, a CLI run, a screenshot)
and show what happened.
5. **Report** in this format:
- Requested: <one line>
- Changed files: <list, each with its reason>
- Checks run: <command -> result>
- Not done / unsure: <list, or "none">