Should I test that?

Should I write acceptance tests?

Verdict

Yes

Yes, write an automated acceptance test for each acceptance criterion of a user story: run it through the API with seeded data instead of the browser, and keep it in CI after the story is closed.

Why

Yes, write one automated test per acceptance criterion, and keep it after the story is done. My typical case is a product team whose product owner checks each criterion by hand once, at the sprint review. Blast radius is users, and Change frequency is regularly, because later stories change the same code. Detectability is eventually, because a broken criterion rarely crashes: a list misses one row, or a reminder never goes out. Reversibility is with-effort, a hotfix plus a repair script, and Test cost is moderate, because an API-level test with seeded data takes me about an hour. Rule R11 gives Test.

When the decision changes
WhenDecisionWhy
The acceptance criterion sets a price, a discount or a refund amountTest mandatory: an acceptance test for every amount in the criterion and its boundary valuesBlast radius rises to money and Reversibility to costly, because a wrong charge needs a refund
The acceptance criterion says who may see or edit a record, such as 'a viewer cannot edit a shared document'Test mandatory: an acceptance test for each role, with records of two accountsBlast radius rises to safety-or-legal and Detectability to never, because a missing check fails without an error
The acceptance criterion describes a main flow that fails loudly, such as 'a new user signs up and lands on the dashboard'Test minimally: one test of the flowDetectability moves to same-day, because a user who cannot sign up reports it within a day
The acceptance criterion depends on a partner service with no sandbox, such as 'the courier confirms each pickup within an hour'Test it differently: alert when the share of orders without a courier confirmation risesTest cost rises to heavy, because the only realistic check runs against the partner in production
The acceptance criterion is simple, such as 'an overdue task stays in the assignee's Today list', and one function decides itTest: one unit test of that function, named after the acceptance criterionTest cost falls to trivial, but Detectability stays eventually, which keeps the decision at Test
The app is a side project that only you runDo not write acceptance tests; check each criterion by hand when you finish the featureBlast radius falls to none, because you bear every failure alone

What breaks if you don't test

A criterion checked by hand at the sprint review is checked once. The next story that touches the same query can break it while the unit tests stay green, because none of them checks the criterion as a whole. Customers see a report that looks normal but leaves out a few people, and the complaint arrives weeks later.

What you lose if you over-test

Acceptance tests that drive the browser take about eight seconds each in my suites, so 300 criteria add 40 minutes on one worker to every pull request. A Gherkin layer that no product owner reads needs a step definition for each sentence, and a reworded step breaks tests while the behaviour stays the same.

How to test

  1. Turn each acceptance criterion into one test named after it, before the story is closed.
  2. Run it through the API against a real database. The Practical Test Pyramid notes that an acceptance test does not have to go through the user interface.
  3. Seed one record for each outcome the criterion distinguishes, including records it must leave out.
  4. Keep the criterion's Given, When and Then as three blocks in the test, built with pytest fixtures or your framework's factories, and run the suite in CI.

When the answer changes

  • A criterion moves money or decides who can see a record.
  • A criterion depends on a service you cannot run in a test.
  • The criterion describes a flow whose failure users report within a day.

Real incident + Code example

The overdue report that forgot who never started

On a corporate learning product I worked on, a story had the criterion: after a course deadline, the overdue report lists every assigned learner who has not finished. The product owner checked it at the sprint review. Two months later a story for optional lessons rewrote the completion query with an inner join to the lesson progress table, so learners who had never opened the course dropped out of the report. The unit tests stayed green. An HR lead at a customer noticed 26 days later, when an employee who had never logged in was missing from her list. The fix was a left join and this test, written from the criterion:

# Criterion: "After the deadline, the overdue report lists every
# assigned learner who has not finished the course."
from freezegun import freeze_time

def test_overdue_report_lists_every_unfinished_learner(client, make_course, make_learner):
    # Given a course with a deadline and three assigned learners
    course = make_course(deadline="2026-03-01", lessons=4)
    make_learner("Ana", course, lessons_done=4)
    make_learner("Ben", course, lessons_done=2)
    make_learner("Chen", course, lessons_done=0)  # no progress rows at all

    # When a manager opens the report after the deadline
    with freeze_time("2026-03-02"):
        response = client.get(f"/api/courses/{course.id}/overdue/")

    # Then everyone who has not finished is listed; the inner join returned only Ben
    assert response.status_code == 200
    assert {row["name"] for row in response.json()} == {"Ben", "Chen"}

FAQ

Should you define acceptance tests for vital yet simple behaviour?

Yes, vital behaviour needs an acceptance test even when the code behind it is simple, because a later story can change that code and nobody checks the criterion again. A simple criterion also makes the test cheap: one function and one assertion.

Should acceptance criteria be mapped to unit tests?

Map an acceptance criterion to a unit test when one function decides it, and name the test after the criterion. When the criterion spans a request, a query and a response, map it to one API-level test, because unit tests with mocks do not run the query.

Do I need Cucumber to write acceptance tests?

No, acceptance tests can be ordinary tests in your test framework, with the criterion's Given, When and Then as comments. Cucumber pays off when product owners read or write the scenarios; otherwise each Gherkin step is one more step definition to keep in sync with the code.