AI and LLM products

Yes, let AI draft unit tests for code that needs them, but take every expected value from the requirement, not from the code, and read each assertion before you merge.

Yes, test minimally: before each model switch, run one recall@10 comparison of the current and the candidate model on 100 real queries labelled with the documents that answer them, instead of choosing by a public leaderboard score or writing unit tests of the vectors.

Evaluate RAG retrieval separately: label 50 real questions with the documents that answer them, and check recall at 5 in CI on every chunking, embedding or document change.

Yes, mock LLM calls in tests of your own code: a fake model client that returns replies recorded from the real model, including malformed, refused and cut-off replies, tests prompt building, parsing and error handling in CI.

Code under test: Yes

Yes, test a vibe-coded app that customers use with end-to-end tests of its main flows, run on every push and after every prompt session against expected results you write yourself, because a prompt session can rewrite a file the prompt did not name and make the app save bookings with a plausible wrong date that no error shows and no customer reports for weeks.

Yes, test AI-generated code by the rules you apply to code you type yourself: tests for the main path and the likeliest edge cases in CI, with expected values taken from the requirement.

Test RAG chunking: unit tests that no source text is lost, no chunk exceeds the embedding model's token limit and tables stay whole, plus the labelled retrieval questions on every change to chunk size, overlap or splitting rules.

Yes, give each data science function that a recurring report reruns one test on a small hand-built table whose expected result you worked out by hand.

Test the code that runs LLM tool calls: in CI, feed each tool handler scripted calls with valid, malformed and out-of-range arguments, and assert what it stores and which error it returns to the model.

Test an MCP server that customers connect to: in CI, call each tool through an in-memory MCP client with valid, ambiguous and invalid arguments, and assert what the tool stores and which error it returns.

Test structured outputs from an LLM: unit-test the code that parses each output with recorded replies, including a refused reply, a cut-off reply and values the schema allows but your rules reject, and compare extracted fields with labelled inputs in CI.

Yes, use TDD with an AI coding assistant for business rules: write the failing tests yourself with expected values from the requirement, watch them fail, commit them, and let the assistant write code until they pass without editing those tests.

Code under test: Yes

Yes, write tests before an AI assistant refactors code that has none: characterization tests at the code's entry point that record today's result for each branch, committed before the refactor and kept out of the assistant's reach.

Yes, unit test the feature transforms, data splits and serving code of a machine learning pipeline, including a parity test that feeds one raw record through the training and serving feature code.

Yes, tests of code that writes, filters and deletes chunks in a vector store should run against the same vector database as production, started in a container or its local emulator, with the production index type and distance metric and small hand-written vectors.

An LLM app that users see needs evals: an evaluation set of 50 real inputs with a check for each, run on every prompt or model change, in place of unit tests that assert the reply text.

Yes, keep human review of LLM outputs as a weekly sample: a person grades 30 production outputs against the eval rubric and checks the automated grader against those grades, instead of approving every output before users see it.

Code under test: Test it differently

Yes, retest before the switch, with an evaluation set instead of unit tests: run 50 real inputs on the old and the new model version, and switch only when no case that passed on the old version fails on the new one.

Code under test: Test it differently

Check an AI agent's tool choices and answers with an evaluation set of real tasks and traces of production runs, because the model's replies vary between runs.

Test for hallucinations differently: check answers to real questions against your own documents in an evaluation set on every prompt, retrieval or model change, instead of unit tests that assert reply text.

Test a prompt differently: score every prompt or model change against an evaluation set of real inputs, and read a sample of production replies, instead of asserting exact reply text in unit tests.

Use an LLM as a judge in your evaluation set only for qualities code cannot check, such as tone or completeness, and trust its grades only after they match a person's pass and fail grades on 30 labelled replies; check facts and formats in code.

Code under test: Test it differently

Yes, run LLM evals in CI, as a separate job that starts when the prompt, retrieval code or model version changes: it fails when a code-checked case passes on the main branch and fails on the pull request, and it reports graded scores for the reviewer to read before the merge.

Code under test: Test it differently