Do you need tests before an AI refactor?

Yes, you need tests before an AI refactor of untested code that customers use, because the assistant's mistakes compile and return plausible values. Characterization tests with one row per branch fail when a result changes.

Can the AI write the characterization tests before it refactors?

Yes, an AI can write characterization tests before a refactor, because their expected values are what the code returns today. Check that each branch has a row, and commit the tests first.

How do I stop an AI agent from changing tests during a refactor?

To stop an AI agent from changing tests during a refactor, commit the tests first, deny the agent edits to the test folder, and check before merge that the folder has no diff. A prompt instruction alone fails when the agent cannot make a test pass.

Is code review enough to check an AI refactor?

No, code review alone does not catch an AI refactor that changes a result, because a reviewer of hundreds of moved lines checks structure, not output. Run characterization tests after each step instead.

Should I write tests before letting AI refactor code?

Verdict

Yes

Yes, write tests before an AI assistant refactors code that has none: characterization tests at the code's entry point that record today's result for each branch, committed before the refactor and kept out of the assistant's reach.

Why

Blast radiususers
Change frequencyregularly
Detectabilityeventually
Reversibilitywith-effort
Test costmoderate

Yes, write tests before an AI assistant refactors code that has none. My typical case is a coding agent restructuring an untested backend module that customers use. Blast radius is users and Change frequency is regularly, because the module changes monthly. Detectability is eventually, because code the assistant broke by mistake still compiles and returns plausible values. Reversibility is with-effort, since records saved in between need a repair script, and Test cost is moderate, about an hour to choose inputs that reach each branch, so rule R11 gives Test.

When the decision changes
When	Decision	Why
The assistant refactors code that calculates charges, refunds or invoice totals	Test mandatory: pin today's amounts and boundary values before the refactor, and have a second person review the tests	Blast radius rises to money and Reversibility to costly, because a wrong charge ends in refunds
The assistant refactors the check that decides who may read a record	Test mandatory: pin an allowed, a denied and a cross-account request for each role before the refactor	Blast radius rises to safety-or-legal and Detectability to never, because a leaked record raises no error
CI tests the assistant cannot edit already assert the result through the public interface for each branch and boundary value, including zero and missing inputs, with data that gives each branch a different result	Do not add tests first; have the assistant run the existing suite after each step	Detectability moves to immediately and Reversibility to trivial, because a changed result fails the suite before the merge
The assistant refactors a prototype that only you run and will delete	Do not test the prototype; write tests when you rebuild it for users	Blast radius falls to none, because only you run the code and bear a failure
The assistant refactors a rule that archives idle projects, which changes a few times a year	Test minimally: one characterization test of the main path before the refactor	Change frequency falls to rarely, so a failure has fewer chances to happen
One long function mixes database calls, the clock and outside services, so a test needs days of setup	Test it differently: run the old and new code on the same production read requests and alert when results differ	Test cost rises to heavy, while Detectability stays eventually and Reversibility with-effort

What breaks if you don't test

An assistant asked to tidy a module rewrites more lines than a person would, merging conditions and moving defaults into helpers, and each rewrite can drop a branch. A reviewer of hundreds of moved lines reads names and structure and runs nothing. With no test to fail, the agent reports success, and a customer notices weeks later.

What you lose if you over-test

An assistant told to "add unit tests" writes one test per function, with mocks that assert internal calls. The refactor moves those functions, the tests fail, and the agent rewrites them to fit the new structure, so they check nothing about behaviour.

How to test

At an entry point the refactor keeps, record today's result for inputs that reach each branch, including zero and missing values. These are characterization tests; Jest's test.each keeps one row per input.
Break the code once on purpose and confirm a test fails.
Commit the tests and deny the agent edits to them, in Claude Code with an Edit(tests/**) deny rule.
Let the assistant refactor in small steps, running the tests after each. Before merge, git diff <tests-commit> -- tests/ must print nothing.

When a test needs days of setup, GitHub's Scientist compares old and new code on production read requests.

Procedure and references

When the answer changes

Existing tests already assert each branch and boundary value through the public interface, with data that gives each branch a different result.
The refactor touches money or an access check.
The code is a prototype that only you run.

Real incident + Code example

The buffer of zero that became fifteen

On a room booking product I worked on, an agent moved scattered defaults in the availability module into one helper, across 23 files. One line changed from settings.bufferMinutes ?? 15 to settings.bufferMinutes || 15. MDN notes that || replaces any falsy value, including 0. Spaces with a buffer of 0 got a 15-minute gap after every booking, and back-to-back slots vanished. Nothing errored, and a space manager asked support nine days later. Our later test fails on that line:

import { freeSlots } from "../src/availability";

const room = { opens: "09:00", closes: "12:00", slotMinutes: 60 };
const bookings = [{ start: "09:00", end: "10:00" }];

// Expected starts recorded from the code before the refactor
test.each([
  [0, ["10:00", "11:00"]],
  [undefined, ["10:15"]],
  [30, ["10:30"]],
])("bufferMinutes %p gives free starts %p", (bufferMinutes, expected) => {
  const starts = freeSlots({ ...room, bufferMinutes }, bookings).map((s) => s.start);

  expect(starts).toEqual(expected);
});

Against the || line the 0 row returns ["10:15"] and fails.

Sources

Should I write tests before refactoring?Yes
Should AI write unit tests?Yes
Should I use TDD with AI coding assistants?Code under test: Yes
Should I test AI-generated code?Yes
Should I add tests to legacy code?Yes

FAQ

Do you need tests before an AI refactor?: Yes, you need tests before an AI refactor of untested code that customers use, because the assistant's mistakes compile and return plausible values. Characterization tests with one row per branch fail when a result changes.
Can the AI write the characterization tests before it refactors?: Yes, an AI can write characterization tests before a refactor, because their expected values are what the code returns today. Check that each branch has a row, and commit the tests first.
How do I stop an AI agent from changing tests during a refactor?: To stop an AI agent from changing tests during a refactor, commit the tests first, deny the agent edits to the test folder, and check before merge that the folder has no diff. A prompt instruction alone fails when the agent cannot make a test pass.
Is code review enough to check an AI refactor?: No, code review alone does not catch an AI refactor that changes a result, because a reviewer of hundreds of moved lines checks structure, not output. Run characterization tests after each step instead.