Answer
No, doctests replace unit tests only for the examples a reader needs: keep two or three doctests that show the main path, run them in CI with `pytest --doctest-modules`, and write the edge cases and error branches as pytest tests.
Verdict on the code under testYes
Why
- Blast radiususers
- Change frequencyregularly
- Detectabilityeventually
- Reversibilitywith-effort
- Test costtrivial
No, doctests cover only the examples in a docstring, and the code still needs tests for its edge cases: rule R11 gives Test. The typical case is a pure Python function in a web backend, such as a parser for reminder offsets, with two or three calls in its docstring. Blast radius is users and Change frequency is regularly, about once a month. Detectability is eventually, because an input outside the examples returns a plausible wrong value. Reversibility is with-effort, since stored wrong values need a repair script, and Test cost is trivial: a pytest case for a pure function takes minutes, like a doctest.
| When | Decision | Why |
|---|---|---|
| The function rounds the tax on each invoice line | Test mandatory: pytest cases with `Decimal` amounts at each rounding boundary, while the docstring keeps one example | Blast radius rises to money and Reversibility to costly, because a wrong rounding ends in refunds |
| The tests cover the usage examples in a library's documentation, which change a few times a year | Test minimally: run the documented examples as doctests in CI | Change frequency falls to rarely, Detectability to same-day and Reversibility to trivial: a developer who copies a broken example reports the error, and a docs release fixes it |
| The function reads from the database or calls an HTTP API | Test it in pytest with a temporary database or a fake server as a fixture, and keep the call out of the docstring | Test cost rises to moderate, because a docstring cannot hold the setup, and Detectability stays eventually, which keeps Test |
| A one-time migration converts every stored duration string into minutes | Test it differently: run the migration on a copy of production data and query for rows it set to zero | Change frequency falls to once, so a test of the migration would never run again |
| The function lives in a script that only you run | Do not test: keep the docstring example as a note and look at the script's output | Blast radius falls to none, because only you bear a failure |
What breaks if you don't test
A docstring holds the common inputs a reader needs. The inputs that break a parser, such as an empty string or a number without a unit, stay out, so doctests alone leave them untested. Plain pytest collects doctests only from files that match test*.txt and runs docstring examples only with --doctest-modules, so a CI command without that option has never run them.
What you lose if you over-test
Edge cases moved into the docstring bury the explanation; the doctest documentation warns that obscure test cases make bad documentation. Doctest compares printed text character by character, so an example that prints a set or an object's default repr fails on correct output: a set has no fixed print order, and the repr embeds a memory address.
How to test
- Keep two or three doctests in each public docstring: the common call and one call that shows a rule.
- Add
--doctest-modulestoaddoptsin the pytest configuration, so CI runs the examples on every push. - Write empty input, malformed input and boundary values as pytest tests with
parametrize. Fixtures for a database or a network belong there too. - Where printed output varies, use the
ELLIPSISflag from the doctest documentation, or move the case into pytest.
Procedure and references
When the answer changes
- The function computes money, such as tax.
- The examples are the public documentation of a library that other teams install.
- The code runs once, as in a data migration.
Real incident + Code example
The reminder offset that became zero
On a team calendar app I worked on, users typed reminder offsets such as 1h30m, and parse_duration turned them into minutes. It had two doctests, run in CI, and no other tests. The first version used re.match, which matches zero or more characters at the start of a string, so "90" matched an empty string, returned 0, and the reminder fired when the meeting started. It took three weeks to trace the late reminders users reported to offsets typed without a unit, and a script recomputed 212 stored offsets. The fixed function keeps both doctests, and the pytest case below fails on the first version:
import re
import pytest
def parse_duration(text: str) -> int:
"""Return the minutes in a reminder offset.
>>> parse_duration("45m")
45
>>> parse_duration("1h30m")
90
"""
match = re.fullmatch(r"(?:(\d+)h)?(?:(\d+)m)?", text.strip())
if not match or not any(match.groups()):
raise ValueError(f"not a duration: {text!r}")
hours, minutes = (int(g or 0) for g in match.groups())
return hours * 60 + minutes
# tests/test_durations.py
@pytest.mark.parametrize("text", ["90", "", "1h30", "h", "-5m"])
def test_rejects_offsets_without_a_unit(text):
with pytest.raises(ValueError):
parse_duration(text)
Related questions
FAQ
- Does Python doctest remove the need for unit tests?
No, Python doctest checks only the examples in a docstring, and a docstring holds the calls that explain a function, not its edge cases. Write empty, malformed and boundary inputs as unit tests in pytest.
- When should I use doctests instead of unit tests?
Use a doctest in place of a unit test when the tested case is also an example a reader needs, such as the main call of a function. Malformed input, boundary values and code that needs a database or a fake server belong in unit tests.
- Should usage examples in documentation be tested?
Yes, run documentation examples as tests in CI, because an example that no longer runs misleads every developer who copies it. Python's doctest and Rust's rustdoc both run documentation examples as tests; Rust doc tests see only a crate's public items, so private functions need unit tests.
- Does pytest run doctests?
Pytest runs the doctests in docstrings only with the
--doctest-modulesoption; by default it collects doctests only from files that matchtest*.txt. Add the option toaddoptsso CI runs the examples on every push.