Verdict
Yes
Test every input validation rule your application declares, with one accepted and one rejected input per rule plus the request bodies that real clients send; do not test that the validation library's own checks work.
Why
- Blast radiususers
- Change frequencyregularly
- Detectabilityeventually
- Reversibilitywith-effort
- Test costtrivial
Test the input validation your application declares, with one accepted and one rejected input per rule. The typical case is an endpoint that checks a signup or checkout request against a schema in pydantic, Zod or Bean Validation. Blast radius is users, because a rule that is too strict turns customers away and one that is too loose stores wrong data. Change frequency is regularly, since request rules change with most features, and Detectability is eventually: a rejected request looks like validation doing its job. Reversibility is with-effort, because bad records need a repair script, and Test cost is trivial, because a table of inputs covers a schema in minutes, so rule R11 gives Test.
| When | Decision | Why |
|---|---|---|
| The validation rejects a zero or negative quantity on an order line, so no line can lower the order total | Test mandatory: quantities of 1, 0 and -1, and the order total after each | Blast radius rises to money and Reversibility to costly: an order shipped at a lower total needs manual recovery |
| The validation is an allowlist that stops a request from setting fields such as `role` or `account_id` | Test mandatory: send `role` and `account_id` and assert that neither field changes | Blast radius rises to safety-or-legal and Detectability to never: a user who makes themselves an admin raises no error |
| A null guard sits in an internal method, and the compiler's null checks already reject null from every caller | Do not test the null guard; keep the compiler's null checks on | Detectability moves to immediately and Reversibility to trivial: a caller that passes null fails the build |
| The validation runs once, in an import of a legacy customer CSV | Test the rules differently: run the import on a copy of the data and review every rejected row | Change frequency falls to once: a test in the suite would never run again |
| The validation checks the arguments of a script that only you run | Do not test the checks; run the script with one bad argument after each change | Blast radius falls to none: nobody but you sees a wrong rejection |
What breaks if you don't test
Validation fails in two directions, and neither raises an error. A max_length dropped in a refactor lets a 40,000-character name into the database, and a PDF invoice breaks on that row weeks later. A phone field that starts to demand +49 turns away every customer who types 030 1234567. The server answers 422, error alerts usually watch only 5xx responses, and the order is lost without a trace.
What you lose if you over-test
The common over-test checks the library instead of your rules: that Field(max_length=100) rejects 101 characters for every field, or that the error message matches word for word. A test of library behaviour repeats the declaration, and a pinned message breaks on a library upgrade while the rule still holds. A browser test per rule adds a slow check that fails for reasons unrelated to validation.
How to test
Write unit tests against the schema or validator function, without a server. The minimum set per rule is one accepted and one rejected input at the boundary, plus the request bodies each supported client version sends, captured from real traffic with personal data removed. Assert which field failed, not the message text. Add one HTTP-level test per endpoint that sends an invalid body and expects 422, so a route that stops running the validator fails CI. Pytest's parametrize turns the cases into one table, and the OWASP Input Validation Cheat Sheet explains why every browser check needs the same check on the server.
When the answer changes
- The rule guards money, such as a quantity, or decides which fields a caller may set.
- The check is a null guard in typed internal code that the compiler already covers.
- The validation runs once, in an import or a migration.
Real incident + Code example
The apartment number that became a 422
On a grocery delivery backend I worked on, we moved a FastAPI service from pydantic V1 to V2. The checkout Address model typed apartment as str, and our tests used hand-typed bodies, all with strings. Android app versions before 3.0 sent the apartment as a JSON number. V1 turned 12 into "12", and V2 does not by default, as the pydantic migration guide states, so those checkouts got a 422. Our alerts watched only 5xx responses, and 11 days passed before a weekly report showed Android orders down by 7 percent. We set coerce_numbers_to_str=True and added each app version's real body as a case:
import pytest
from pydantic import BaseModel, ConfigDict, Field, ValidationError
class Address(BaseModel):
model_config = ConfigDict(coerce_numbers_to_str=True)
street: str = Field(min_length=1, max_length=100)
apartment: str | None = None
postcode: str = Field(pattern=r"^\d{5}$")
VALID = {"street": "Main St 5", "postcode": "10115"}
@pytest.mark.parametrize("change", [
{}, {"apartment": "12"},
{"apartment": 12}, # body sent by Android versions before 3.0
])
def test_accepts(change):
Address.model_validate(VALID | change)
@pytest.mark.parametrize("field, value", [
("street", ""), ("street", "x" * 101), ("postcode", "1011"),
])
def test_rejects(field, value):
with pytest.raises(ValidationError) as err:
Address.model_validate(VALID | {field: value})
assert err.value.errors()[0]["loc"] == (field,)
With pydantic 2.13 and without the model_config line, the {"apartment": 12} case fails with a string_type error.
Related questions
FAQ
- Should I test argument validation code?
Yes, test argument validation in public methods and library APIs, with one test per guard that passes the bad argument and asserts the exception type. Skip the test for a null guard in internal code when the compiler already rejects null from every caller.
- Should I unit test validation code?
Yes, unit test each validation rule against the schema or validator function, with one accepted and one rejected input. These tests need no server. Add one HTTP-level test per endpoint to prove that the endpoint runs the validator.
- Should I test that the framework's validators work?
No, a validation library such as pydantic, Zod or Hibernate Validator already tests its own length and email checks. Test instead that your schema applies the library's checks to the right fields with the right limits, and that the bodies real clients send still pass.