Verdict
Yes
Yes, test AI-generated code by the rules you apply to code you type yourself: tests for the main path and the likeliest edge cases in CI, with expected values taken from the requirement.
Why
- Blast radiususers
- Change frequencyregularly
- Detectabilityeventually
- Reversibilitywith-effort
- Test costmoderate
Yes, test AI-generated code by the rules you apply to your own code. Who typed it does not change the stakes: for a feature that customers use, Blast radius is users, and Change frequency is regularly, because its rules change about once a month. Detectability is eventually, as for hand-written business logic, because an assistant's mistakes compile and return plausible values. Reversibility is with-effort, since records saved by wrong code need a repair script, and Test cost is moderate, about an hour to write cases from the ticket, so rule R11 gives Test.
| When | Decision | Why |
|---|---|---|
| The AI-generated code computes prices, discounts or invoice totals | Test mandatory: write the expected amounts from the pricing rules yourself | Blast radius rises to money and Reversibility to costly: a wrong charge ends in refunds |
| The AI-generated code decides who may read a record, or pastes request input into a SQL string instead of a bound parameter | Test mandatory: test allowed, denied and hostile input, and have a second person review the tests | Blast radius rises to safety-or-legal and Detectability to never: a leaked record or an injected query raises no error |
| The AI writes a script that renames files on your own machine | Do not test the script; check its output once | Blast radius falls to none: a failure affects nobody but you |
| The AI writes a one-time migration of production data | Test it differently: rehearse it on a copy of production data and diff the result | Change frequency falls to once: a test in the suite would never run again |
| The AI-generated code renders a page that stores nothing and fails with a 500 error that users report | Test minimally: one test that loads the page with realistic data | Detectability moves to same-day and Reversibility to trivial: users report the error that day, and a revert fixes it |
What breaks if you don't test
Untested AI-generated code fails where the assistant guessed something the prompt did not say: a time zone, a boundary, a rounding mode. The guess holds on the developer's machine and for the prompt's sample data, so customers meet the failure first. Security flaws are quieter: Pearce and colleagues had GitHub Copilot complete 89 scenarios built around high-risk CWE weaknesses, and about 40 percent of its 1,689 programs were vulnerable (Asleep at the Keyboard?).
What you lose if you over-test
An assistant makes test code nearly free to type, so a coverage target for generated code fills the suite with tests of getters and internal mock calls. Each refactor breaks tests that check no requirement. Tests the assistant derived from its own code also give false confidence, because each asserts what the code already returns, bugs included.
How to test
Test the main path and the likeliest edge cases in CI:
- Before you prompt, write the cases from the ticket: inputs, expected results, boundaries.
- Add a case for each assumption the assistant made that you did not state, such as time zone, locale or empty input.
- Break the code on purpose, or run a mutation tester such as Stryker, and confirm that a test fails.
When the answer changes
- The code computes money, decides access, or pastes request input into SQL strings.
- The code runs once, such as a data migration.
- The code runs only on your machine.
Code example + Real incident
The trial that ended a day early
On a subscription product I worked on, an assistant wrote this function for the trial end date:
// Requirement: show the trial end date exactly as the API sends it.
export function formatTrialEnd(isoDate) {
return new Date(isoDate).toLocaleDateString("en-US");
}
// The test from the requirement, run in a time zone behind UTC:
test("keeps the API date in New York", () => {
process.env.TZ = "America/New_York";
assert.equal(formatTrialEnd("2026-10-01"), "10/1/2026"); // got "9/30/2026"
});
JavaScript reads a date-only string as midnight UTC, as the MDN Date reference documents. In Kyiv, midnight UTC is 3 a.m. on October 1, so the page looked right on the developer's laptop and in review. In New York, midnight UTC is 8 p.m. on September 30. Customers in the Americas saw a trial end date a day earlier than the reminder email, and we learned of it from a support ticket two weeks later. The fix was one argument, { timeZone: "UTC" }.
Related questions
FAQ
- Do you need to test code written by AI?
Yes, code written by AI needs the tests the same code would need if you had typed it. Take the expected values from the requirement, because an assistant that writes both code and tests repeats its own assumptions.
- Does AI-generated code need more tests than code I write myself?
No, AI-generated code needs the same tests as hand-written code with the same stakes. Add one case for each assumption the assistant made that your prompt did not state, such as time zone or rounding.
- Can the AI that wrote the code also write its tests?
Yes, an AI can write the test code if you supply the cases and expected values from the requirement. Tests an assistant derives from its own code pass on its own bugs.
- Is code review enough for AI-generated code?
No, code review alone misses bugs in AI-generated business logic, because an assistant's mistakes compile and read well, and a reviewer sees the code but not the input that breaks it. Review the code and run tests written from the requirement.