Verdict
Yes
Yes, measure branch coverage on every pull request in CI and read the changed lines that no test runs; do not set a percentage goal for the whole codebase.
Why
- Blast radiususers
- Change frequencyregularly
- Detectabilitysame-day
- Reversibilitytrivial
- Test costtrivial
Yes, measure coverage on every pull request, and read which changed lines no test runs. The typical case is a product team with tests in CI and no coverage report; the failure the report prevents is a new branch that ships untested. Blast radius is users, since that branch runs for customers, and Change frequency is regularly, since each monthly change can add one. Detectability is same-day, because a bug in such a branch usually breaks a page that a user reports, and Reversibility is trivial, because a redeploy removes it. Test cost is trivial, one flag in the test runner, so rule R11 gives Test.
| When | Decision | Why |
|---|---|---|
| The pull request changes code that calculates charges, refunds or invoice totals | Test mandatory: require full branch coverage on that folder and review every uncovered branch | Blast radius rises to money and Reversibility to costly, because a wrong charge ends in refunds |
| A standard or a contract requires coverage evidence, such as MC/DC under DO-178C for avionics | Test mandatory: measure the coverage the standard names and keep each report as evidence | Blast radius rises to safety-or-legal, and the framework puts required evidence first |
| The application is in maintenance and changes a few times a year | Test minimally: run the coverage report by hand after each change and read the changed lines | Change frequency falls to rarely, so fewer changes can add an untested branch |
| The only tests are browser end-to-end tests, and measuring them needs an instrumented build and reports merged across runs | Do not test: skip coverage and list the user flows the browser tests walk through | Test cost rises to heavy, while Detectability stays same-day |
| The project is a static marketing site, and its developer checks each template change in a preview | Do not test: skip coverage and look at the preview before each deploy | Detectability moves to immediately, because a broken template shows in the preview, and Reversibility stays trivial |
| The code is a side project that only you run | Do not test: skip coverage until someone else uses the code | Blast radius falls to none, because you alone bear a failure |
What breaks if you don't test
Without a report, nobody can list the branches the suite skips. A pull request adds an error branch, its author tests the main path, the build is green, and the branch first runs when a customer hits it. The team learns about the gap from a support ticket.
What you lose if you over-test
The cost comes from treating coverage as a goal. A global gate raised above today's level makes developers write tests that run lines and assert nothing, and it blocks merges over untested logging and startup wiring. A module at 85% can still have its only error branch at zero, so the percentage hides where the gaps are.
How to test
Measure in CI, per pull request:
- Turn on branch coverage:
pytest --cov --cov-branchwith pytest-cov,--coveragein Jest or Vitest, JaCoCo in Maven or Gradle. Branch coverage reports anifwhose false path never ran. - Post the uncovered lines of the diff in the pull request with diff-cover. Google found coverage actionable when shown for changesets in code review (Ivanković et al., 2019).
- The reviewer runs the five factors on each uncovered block and asks for a test where the framework gives one.
When the answer changes
- The changed code moves money, checks access, or falls under a standard that names a coverage figure.
- The only tests are slow browser tests.
- The code changes a few times a year, or only you run it.
Real incident + Code example
The import branch no test ran
On a project-management product I worked on, we had about 600 tests in CI and no coverage report. A pull request added a CSV import whose tests all used rows with a due date. Rows without one got the import day as their due date, so every undated task showed as overdue the next morning. Support got 40 tickets in two days, and we repaired the tasks with a script. Line coverage would have shown the function at 100%; branch coverage, which we turned on that week, reported the missing path:
# import_tasks.py
def task_from_row(row, today):
due = today # bug: a row without a date should get None
if row["due"]:
due = parse_date(row["due"])
return Task(title=row["title"], due=due)
# test_import_tasks.py: every line above runs, so line coverage is 100%
def test_row_with_due_date():
task = task_from_row({"title": "Ship", "due": "2026-10-01"}, date(2026, 9, 15))
assert task.due == date(2026, 10, 1)
# pytest --cov=app --cov-branch --cov-report=term-missing
# reports the missing branch 4->6: no test runs a row without a date
Related questions
FAQ
- Is code coverage worth it?
Yes, measuring code coverage is worth it, because it costs one flag in the test runner and lists the branches no test runs. Chasing a coverage percentage is not worth it, since a test that asserts nothing raises the number as much as one that checks behaviour.
- What code coverage goal should I set?
Set no global coverage goal above today's level; a CI floor at today's number only stops coverage from falling. Require that the uncovered lines of each pull request are read and then tested or accepted. Where a failure costs money, such as in a billing folder, require full branch coverage for that folder.
- Should I measure code coverage for GUI tests?
No, do not measure coverage of browser tests when that needs an instrumented build and reports merged across runs. Measure coverage of the unit and integration tests instead, and list the user flows the browser tests walk through.
- Should I exclude code from code coverage analysis?
Yes, exclude generated code and vendored libraries from coverage, so the report lists only code your team wrote and can test. Mark each exclusion in the code, such as
# pragma: no coverin coverage.py, so a reviewer sees it.