Verdict
No
No, not every test should be automated: a manual check of a stable read-only screen whose failures users report costs more to automate than the failures it prevents, so look at that screen by hand when a release changes it, and automate a check once it covers money, access, saved data or a flow that changes monthly.
Why
- Blast radiususers
- Change frequencyrarely
- Detectabilitysame-day
- Reversibilitytrivial
- Test costmoderate
No, do not automate every test; automate a check when its factors say so. My typical case is a tester who checks before each release that a read-only screen, such as the activity history, loads. Blast radius is users, because customers see the broken screen, and Change frequency is rarely, because such screens change a few times a year. Detectability is same-day, because a customer who gets an error page writes to support, and Reversibility is trivial, because a screen that only shows data stores nothing wrong. Test cost is moderate: a browser test takes an hour and needs updates when the screen changes. Rule R13 gives Do not test; a screen that saves data or changes monthly gets Test minimally.
| When | Decision | Why |
|---|---|---|
| The check covers checkout, where a bug can charge a card twice | Test mandatory: automate the charge path in the payment provider's test mode | Blast radius rises to money and Reversibility to costly, because a double charge needs a refund |
| The check confirms that a customer cannot open another customer's records | Test mandatory: automate one allowed and one denied request | Blast radius rises to safety-or-legal and Detectability to never, because a leaked record looks normal |
| The check confirms that an edit form the team changes monthly saves every field | Test: automate saving a distinct value in each field and reading the record back | Change frequency rises to regularly, Detectability to eventually and Reversibility to with-effort, because a record with a dropped field looks complete |
| The check signs in and runs a search, and both flows change in most months | Test minimally: one browser test per flow on every pull request | Change frequency rises to regularly and Reversibility to with-effort, because a broken sign-in leaves support tickets |
| The check judges whether a sign-up flow reworked monthly makes sense to a first-time user | Test it differently: release the flow in stages, alert when step completion drops, and watch outsiders try it | Test cost rises to prohibitive, Change frequency to regularly and Detectability to eventually: no script judges understanding, and confused users leave silently |
| The check confirms a one-time migration that rewrites stored customer records | Test it differently: rehearse it on a copy of production data, compare the result with a query, and keep a backup | Change frequency falls to once and Reversibility rises to costly, because broken records are repaired by hand |
What breaks if you don't test
A break on a stable read-only screen reaches customers, a ticket arrives the same day, and a redeploy ends it. The damage comes when a check that belongs in CI stays manual: a tester who repeats a long list skims its end, and a form that drops one field on save passes a quick look, because the confirmation still appears.
What you lose if you over-test
Automating every manual case turns a list a tester read with judgment into browser tests that each need upkeep. Checks of stable screens seldom fail for a real reason, so their failures come from selectors and timing, and after a few false alarms the team reruns red builds unread.
What to do instead
- Sort the manual checks by the five factors, not by how easy each is to automate.
- Automate first the checks on money, access, saved data and flows that change monthly, for example as Playwright tests, and run them on every pull request.
- Look by hand at the stable screens a release changes, and drop repeated checks of screens it did not touch.
- Spend the saved hours on exploratory sessions on new features, as the Practical Test Pyramid recommends.
When the answer changes
- The screen saves data, charges a card, or decides who sees which records.
- The screen starts changing in most releases.
- A team rule or a standard requires automated tests for every requirement.
Cost estimate
One check: automated or looked at
The hours come from my projects with monthly releases; replace them with yours.
- Automating the check of a stable screen: 1 hour to write, plus three screen changes a year at 20 minutes of updates each, so 2 hours in year one and 1 hour a year after.
- Looking by hand: 3 minutes in each of the three releases a year that touch the screen, so 9 minutes a year.
- A break that reaches customers: about one every two years on such screens, at 2 hours of fixing and answering tickets, so 1 hour a year.
Automating the stable-screen check costs at least as much as the failures it prevents. The sign-in flow on the same projects changed monthly and broke about four times a year at 6 hours each: 24 hours of failures against 5 hours of test work (1 hour plus 12 updates of 20 minutes), so that test runs in CI.
Related questions
FAQ
- Should testers automate their work?
Testers should automate the checks they repeat on money, access, saved data and flows that change monthly, because a script then runs them on every pull request. They should keep exploratory sessions and judgment calls for themselves.
- Can automated testing replace manual testing?
No, automated tests cannot replace manual testing, because a script checks only the outcomes someone wrote down in advance. Exploratory sessions find failures nobody predicted, such as a confusing step.
- Which tests should be automated first?
Automate first the checks where a failure loses money or exposes records, then forms whose wrong saves look complete, then main flows that change every month. Checks of stable read-only screens come last or stay manual.