Should you unit test your prompts?

No, do not unit test prompt replies with exact assertions; score them against an evaluation set of real inputs instead. Unit test the code around the prompt: the template, the schema check and the parser.

Should I test a prompt again when the model version changes?

Yes, run the full evaluation set before you move a prompt to a new model version, because a new version can follow the same wording differently. Pin the model version in configuration so that a switch happens only when you choose it (Should I retest my app after an LLM model upgrade?).

Can a unit test check the output of a language model?

Yes, a unit test can check structured model output: it validates a JSON reply against a schema or compares a label from a fixed list. For free text, an exact expected string fails on the next run, so check required facts in an evaluation set instead.

Should I test prompts?

Verdict

Test it differently

Test a prompt differently: score every prompt or model change against an evaluation set of real inputs, and read a sample of production replies, instead of asserting exact reply text in unit tests.

Why

Blast radiususers
Change frequencyregularly
Detectabilityeventually
Reversibilitywith-effort
Test costheavy

Test a prompt differently: run an evaluation set of real inputs on every prompt or model change, and read a sample of production replies. For a prompt behind a product feature, Blast radius is users and Change frequency is regularly, because customers read the replies and the team rewords the prompt about once a month. Detectability is eventually, because a worse reply is still fluent text and raises no error. Reversibility is with-effort: a revert takes one deploy, but stored replies need repair. Test cost is heavy, because the reply differs on every run, so a useful check needs graded real inputs, and rule R9 gives Test it differently.

When the decision changes
When	Decision	Why
You run the prompt by hand in a chat window for your own work	Do not test the prompt; read each reply before you use it	Blast radius falls to none: a bad reply reaches only you
The prompt's reply quotes prices or grants refunds	Test mandatory: evaluation cases for every pricing and refund rule gate each release	Blast radius rises to money and Reversibility to costly
The prompt answers questions from customer records	Test mandatory: evaluation cases that try to pull another customer's data run on every change	Blast radius rises to safety-or-legal and Detectability to never: a leaked record reads like a normal answer
The prompt sorts support tickets into a fixed list of queues	Test: assert the expected queue for labelled examples in CI, with a minimum pass rate	Test cost falls to moderate: the output is a label from a fixed list, which a test compares exactly; Detectability stays eventually
The reply goes to customers as an email that no person reads first	Test: run the evaluation set in CI, and block the release on a failed case	Reversibility rises to impossible: a sent email cannot be recalled
The prompt runs once to tag an archive of old documents	Test it differently: rehearse on a sample of 200 documents and check the tags by hand	Change frequency moves to once: an evaluation set would never run again

What breaks if you don't test

A prompt edit that fixes one case can quietly break others. The model follows the new wording, drops a detail that the old wording asked for, and every reply still reads well. A new model version does the same without any edit on your side. Agents or customers notice the missing detail weeks later.

What you lose if you over-test

Unit tests that assert the exact text of a reply break with the next model version, and at any temperature above zero they fail on some runs. The team reruns them until they pass, then stops reading them. An evaluation set on every commit costs time and API calls: 300 cases at 2 seconds per call add 10 minutes to a sequential CI run.

What to do instead

Collect 50 real inputs from your logs, plus every input that produced a bad reply before.
For each input, write checks: exact match for labels and fields, a "contains" check for facts such as an order number, and a rubric that a person or a second model scores.
Run the set on every change to the prompt, the template or the model version. A case that passed before and fails now is a regression.
Validate structured replies against a JSON schema in code, as Gabriel Anhaia recommends, and unit test the code that fills the template.
Log the prompt version with each production reply, and read a weekly sample.

Anthropic's guide to building evaluations describes grading methods. If users type text into the prompt, add the injection cases from OWASP LLM01.

Procedure and references

When the answer changes

The reply quotes prices, grants refunds, or reads personal data: the evaluation set becomes mandatory.
Replies reach customers before any person reads them: the set becomes a CI gate.
The prompt returns one label from a fixed list: exact assertions work as ordinary tests.

Real incident + Code example

The summary that lost the order number

On a support tool I worked on, the model summarised each new ticket for the agent. A colleague shortened the prompt to get briefer summaries, checked five tickets by hand, and shipped. The shorter prompt no longer asked for the order number, and the model began to leave it out of long tickets. Agents looked up orders by hand for two weeks before anyone traced the problem to the edit. One check over real tickets would have failed on the first run:

# load_cases and summarize are the project's own functions.
CASES = load_cases("evals/ticket_summary.jsonl")  # real tickets from logs

def check(case, summary):
    problems = []
    for order_id in case["order_ids"]:
        if order_id not in summary:
            problems.append(f"missing order {order_id}")
    if len(summary.split()) > 80:
        problems.append("longer than 80 words")
    return problems

def run(prompt_version):
    failed = {}
    for case in CASES:
        summary = summarize(case["ticket"], prompt_version)
        if problems := check(case, summary):
            failed[case["id"]] = problems
    return failed  # compare with the previous version's failures

The check accepts any wording and fails only on a missing fact or an overlong summary.

Sources

dev.to/gabrielanhaia/the-single-unit-test-every-llm-prompt-should-have-18d4

Should I test AI agents?Test it differently
Should AI write unit tests?Yes
Should I write unit tests for machine learning code?Yes
Should I test AI-generated code?Yes

FAQ

Should you unit test your prompts?: No, do not unit test prompt replies with exact assertions; score them against an evaluation set of real inputs instead. Unit test the code around the prompt: the template, the schema check and the parser.
Should I test a prompt again when the model version changes?: Yes, run the full evaluation set before you move a prompt to a new model version, because a new version can follow the same wording differently. Pin the model version in configuration so that a switch happens only when you choose it (Should I retest my app after an LLM model upgrade?).
Can a unit test check the output of a language model?: Yes, a unit test can check structured model output: it validates a JSON reply against a schema or compares a label from a fixed list. For free text, an exact expected string fails on the next run, so check required facts in an evaluation set instead.