Verdict
Yes
Yes, before fixing a typical bug, write a test with the input from the report, run it on the unfixed code, and see it fail for the reason the report describes.
Why
- Blast radiususers
- Change frequencyregularly
- Detectabilityeventually
- Reversibilitywith-effort
- Test costmoderate
Yes, write the failing test first, because a regression test that has never failed proves nothing about the bug. My typical case is a customer-reported bug in an application the team changes monthly, so Blast radius is users and Change frequency is regularly. Detectability is eventually, because the bug passed review and CI and a customer found it days later. Reversibility is with-effort, a hotfix plus a repair script, and Test cost is moderate, about an hour, because the report holds the input. Rule R11 gives Test, and writing the test before the fix costs nothing extra.
| When | Decision | Why |
|---|---|---|
| The bug overpaid refunds to customers | Test mandatory: a failing test for the reported refund and its boundary amounts, reviewed by a second person | Blast radius rises to money and Reversibility to costly, because overpaid refunds need claw-backs |
| The bug let one account read another account's records | Test mandatory: first a failing test that the other account's request is denied, then the fix | Blast radius rises to safety-or-legal and Detectability to never, because a leaked record raises no error |
| The bug is a 500 error on a read-only page that users reported within hours | Test minimally: one failing test with the request that crashed the page | Detectability falls to same-day and Reversibility to trivial, because a read-only page stores nothing |
| The fix changes a type so that the compiler rejects the code that caused the bug | Do not write a regression test; the compile error on the unfixed code is the failing check | Detectability falls to immediately and Reversibility to trivial, because the build fails before the merge |
| The bug is a race between two workers under load, and reproducing it in a test takes days | Test it differently: fix the race from the logs and alert on the trace it leaves, such as duplicate rows | Test cost rises to heavy, while Detectability stays eventually and Reversibility with-effort |
What breaks if you don't test
A test written after the fix can pass on the unfixed code too, because its data misses the case from the report. The fix can also land in the wrong place: a test written from your guess checks the function you changed, while a test written from the report stays red until the real cause is fixed. With no test at all, the bug returns at the next refactor, and the customer who reported it finds it again.
What you lose if you over-test
A strict failing-test-first rule costs the most when reproduction is the hard part. A race between two workers can take days to reproduce, while the logs already show the cause. During an outage, restore service first, then write the test and run it against the commit before the fix. A reproduction that uses sleep() to hit a timing window fails at random in CI, and the team learns to rerun red builds.
How to test
Treat the report as the red step of the red, green, refactor cycle, at the lowest level that reproduces it, usually a unit or integration test in CI:
- Copy the input from the report: the request, the file, the account settings, the time of day.
- Run the test on the unfixed code. It must fail on the assertion that matches the report, not on an import error.
- If the cause is unclear, give the test to
git bisect run, which finds the commit that introduced the bug. - Fix the code until the test passes, and commit both together, with the ticket number in the test name.
Procedure and references
When the answer changes
- The bug moved money or showed one customer's records to another.
- Reproducing the bug needs days of setup, such as timing under load.
- The fix makes the compiler reject the mistake.
Real incident + Code example
The fix that missed the streak bug
On a language-learning web app I worked on, a learner in Tokyo lost a 212-day streak although she practised every day. A colleague suspected isSameDay(), which compared UTC dates, switched it to the learner's timezone, and wrote a unit test afterwards, which passed. Four days later a learner in Sydney sent the same report. The nightly job that reset streaks never called isSameDay(): it took "yesterday" as a UTC date, so sessions at 10:00 on Monday and 08:30 on Tuesday, Tokyo time, fell on one UTC day and left Tuesday empty. The second fix started from this test:
import { expect, it } from "vitest";
import { runStreakReset } from "./streaks";
import { createTestDb } from "./test-db";
// Ticket LRN-418: a learner in Tokyo practised daily and lost her streak
it("keeps a Tokyo learner's streak across the UTC day boundary", async () => {
const db = await createTestDb();
const learner = await db.addLearner({ timezone: "Asia/Tokyo", streak: 212 });
await db.addSession(learner, "2026-03-02T10:00:00+09:00"); // Monday
await db.addSession(learner, "2026-03-03T08:30:00+09:00"); // Tuesday
await runStreakReset(db, new Date("2026-03-04T00:10:00Z"));
expect((await db.getLearner(learner)).streak).toBe(212);
});
It failed on the released code, isSameDay() change included, so the first fix had missed the cause. A script rebuilt 64 streaks from the session log.
Related questions
- Should I write a test for every bug fix?Yes
- Should I write tests before code?Code under test: Yes
- Should I commit failing tests?Code under test: Yes
- Should I use TDD?Code under test: Yes
- Should I add tests to legacy code?Yes
FAQ
- Should I write a test to prove that deleting code fixes a bug?
Yes, write a test for the behaviour that deleting the code restores, and see it fail while the code is still there. The test checks what users get, so it fails again if a revert or a merge conflict brings the deleted lines back.
- Why should a regression test fail before the fix?
A regression test must fail before the fix because a test that passes on the unfixed code cannot detect the bug. A failure on the assertion that matches the report shows that the test reaches the cause.
- Should I commit the failing test before the fix?
Commit a failing regression test to the main branch only together with its fix, so the build never stays red for a known bug. If the fix must wait, mark the test as an expected failure, such as pytest's
xfail(strict=True), which fails the build once the bug is fixed.