Verdict
Yes
Yes, let AI draft unit tests for code that needs them, but take every expected value from the requirement, not from the code, and read each assertion before you merge.
Why
- Blast radiususers
- Change frequencyregularly
- Detectabilityeventually
- Reversibilitywith-effort
- Test costmoderate
Yes, let an AI assistant draft unit tests, and take every expected value from the requirement yourself. Who types a test does not change whether the code needs one: for a business rule in a web application, Blast radius is users and Change frequency is regularly, because customers see wrong results and rules change about once a month. Detectability is eventually, because a wrong rule returns a plausible value, and a test derived from the code asserts that same value. Reversibility is with-effort, because records saved under a wrong rule need a repair script. Test cost is moderate: the draft takes a minute, but checking each expected value against the requirement takes about an hour per module, and rule R11 gives Test.
| When | Decision | Why |
|---|---|---|
| The function computes a price, a discount or an invoice total | Test mandatory: you write the expected amounts from the pricing rules; the AI writes only the setup | Blast radius rises to money and Reversibility to costly: a wrong charge ends in refunds |
| The function decides which records a user may read | Test mandatory: you write the allowed and denied cases, and a second person reviews the tests | Blast radius rises to safety-or-legal and Detectability to never: a leaked record shows no error |
| The code is a one-line wrapper with one argument, and its callers' tests in CI use data that tells the wrapped method apart from its siblings | Do not test the wrapper; rely on its callers' tests, even if an AI writes one in seconds | Detectability moves to immediately and Reversibility to trivial: the callers' tests fail in the same CI run |
| The code is a one-time data migration | Test it differently: rehearse the migration on a copy of production data and diff the result | Change frequency moves to once: a unit test would never run again |
| The function returns text generated by a language model | Test it differently: score sampled output against an evaluation set of real prompts | Test cost rises to heavy, because a unit test cannot pin output that differs on every run |
What breaks if you don't test
Without tests, a wrong rule reaches customers, and the first report is a support ticket days later. With tests an assistant derived from the code, the same bug ships with a green check, because each test asserts what the buggy code returns. Swizec Teller makes this argument in Why you shouldn't use AI to write your tests.
What you lose if you over-test
An assistant makes tests nearly free to type, so the suite grows faster than anyone reads it. In my practice, an assistant asked to "write unit tests for this class" returns one test per method, getters included, and verifies internal mock calls. Each refactor then breaks tests that check no requirement, and regenerating a red test erases its signal.
How to test
Test the main path and the riskiest boundary values in CI, and keep the expected values your own:
- Before you prompt, write the case list from the ticket: inputs and expected results, boundaries included.
- Give the assistant the case list and the function signature, and let it write the setup and assertions.
- Read every assertion against your list, and delete tests of internal calls or getters.
- Break the code on purpose, or run a mutation tester such as PIT for Java or Stryker for JavaScript and C#, and confirm that the tests fail.
Procedure and references
When the answer changes
- The code computes money or decides access: a second person reviews every expected value.
- The output comes from a language model: an evaluation set replaces unit tests.
- You are about to refactor untested legacy code: pin its current outputs first, as in tests before an AI refactor.
Code example + Counterexample
A test that asserts the bug
# Requirement: search runs for queries of 3 or more characters.
def should_search(query: str) -> bool:
return len(query.strip()) > 3 # bug: skips "cat"
# Derived from the code: passes, and asserts the bug.
def test_skips_three_characters():
assert should_search("cat") is False
# Written from the requirement: fails, and finds the bug.
def test_runs_for_three_characters():
assert should_search("cat") is True
An assistant that sees only the function reads > 3 and writes the first test. The second test needs the requirement, which the code lacks.
The counterexample is a refactor of legacy code, where the current output is the behaviour you want to keep, so a test derived from the code is correct: Michael Feathers calls it a characterization test. Test cost stays moderate, because I still read every pinned value, and Detectability stays eventually, so the framework still gives Test. Let the assistant pin the outputs for real inputs, refactor against those tests, and fix each pinned bug later in a separate change.
Related questions
FAQ
- Should I let AI write my tests?
Yes, let AI write the code of your unit tests, but write the cases and expected values yourself from the requirement. A test whose expected value the AI read from the implementation passes on the bug it should catch.
- Can I trust AI-generated unit tests?
Trust an AI-generated unit test after you have checked each expected value against the requirement and seen the test fail on broken code. A mutation tester such as PIT or Stryker breaks the code on purpose and reports each break that no test catches.
- Does high coverage from AI-generated tests mean my code is tested?
No, coverage from AI-generated tests shows which lines ran, not whether any assertion checks a requirement. A suite generated from the code can reach full line coverage while every assertion repeats the current output, bugs included.