Should I add visual regression tests?

Add visual regression tests only for a shared component library or for output that leaves the browser, such as email templates. For the pages of a typical web app, browser assertions on the key actions catch the layout breaks that stop users at a lower cost.

Why do screenshot tests fail on CI but pass locally?

Screenshot tests fail on CI when the baselines came from another machine, because browser rendering varies with the operating system, fonts and hardware. Generate the baselines in the container image that CI runs.

Should I use visual regression tests for a component library?

Yes, a shared component library earns one screenshot per component variant, because a change shows on screens its author never opens. Render each variant with fixed props, so the pixels change only when the component does.

Can visual regression tests replace end-to-end tests?

No, a screenshot shows that a page looks the same, not that its form submits or its data is right. Keep a browser test for each key user path.

Is visual regression testing worth it?

Verdict

Visual regression testing is not worth it for the pages of a typical web app: check the elements users need with explicit browser assertions, and keep screenshot comparisons for a shared component library.

Why

Visual regression testing is not worth it for the pages of a typical web app. For screenshots of the main pages compared in CI, Blast radius is users, because customers see a broken layout, and Change frequency is regularly, since page CSS changes about once a month. Detectability is same-day, because a covered button draws a support ticket within a day, and Reversibility is trivial, because a CSS revert leaves nothing behind. Test cost is heavy: pixels change with fonts, operating system and live data, and every intended design change fails every baseline that shows it. Rule R13 gives Do not test; with Test cost moderate the decision becomes Test minimally.

When the decision changes
When	Decision	Why
The screenshots cover a shared component library that dozens of screens use	Test minimally: one screenshot per component variant, rendered with fixed props in one container image	Change frequency falls to rarely, Detectability moves to eventually and Test cost to moderate: a change shows on screens its author never opens, and fixed props give the same pixels on each run
The output is an email template that goes to every new customer	Test: screenshot each template rendered from fixed data and read every diff in review	Reversibility rises to impossible, Detectability moves to eventually and Test cost to moderate: a sent email cannot be taken back, and nobody on the team reads it
A layout break can cover the Pay button on the checkout page at phone width	Test mandatory: a browser test at phone width that clicks the Pay button	Blast radius rises to money and Reversibility to costly: customers who cannot pay buy elsewhere
The screens belong to a native mobile app	Test minimally: one screenshot test for each key screen on one device size	Reversibility rises to costly: a fix needs a new app store release
A layout break shows only in Safari, which nobody on the team uses, no end-to-end suite runs in WebKit, and Safari users leave without reporting it	Test it differently: count the sessions that finish the main action in each browser, and alert when one browser falls behind	Detectability moves to eventually while Test cost stays heavy: a production signal finds the break sooner than a screenshot suite in every browser

What breaks if you don't test

A CSS change on one page leaks into a page its author never opened. On a course platform, a new sticky progress bar covers the Next lesson link at phone width, and learners on phones cannot move on. One of them writes to support that day, and a revert fixes it within an hour.

What you lose if you over-test

A full-page screenshot suite fails on changes nobody meant to catch. The Playwright documentation warns that rendering varies with the host OS, browser version, hardware and headless mode, so baselines made on a laptop fail on a Linux CI runner. A date or an avatar fails its page until someone masks it. When a header redesign fails every page at once, the developer runs --update-snapshots, and a real break in the batch is approved with the rest.

What to do instead

Check the elements users need with browser assertions at the phone and desktop widths you support. Playwright runs actionability checks before a click, and one fails when another element covers the target, so a test that clicks the main action of each key page catches a covered button without baseline images. Have the reviewer open the changed pages in a preview deployment. For a shared component library, keep one screenshot per variant, generated in the container image CI uses, as the Playwright visual comparisons guide advises, with mask over data that changes.

Procedure and references

When the answer changes

Other screens or apps build on your component library.
The output leaves the browser: an email, a PDF, or a native app screen.
A layout break can hide a checkout button.

Real incident + Code example

The 140 screenshots we approved unread

On a course platform I worked on, Playwright compared about 140 full-page screenshots on every pull request: each main page at three widths. Every header redesign and font update failed most of them, and within a few months the team ran --update-snapshots without opening the diffs. A sticky progress bar that covered the Next lesson link, the last element on the page, went through in one of those updates, and a learner reported it the next morning. We deleted the page screenshots, kept 12 component screenshots, and added one test per key page:

import { test, expect } from '@playwright/test';

test.use({ viewport: { width: 375, height: 667 } });

test('Next lesson is clickable on a phone', async ({ page }) => {
  await page.goto('/courses/intro/lessons/1');
  // click() waits until no other element covers the link,
  // and fails with a TimeoutError if the progress bar still does
  await page.getByRole('link', { name: 'Next lesson' }).click();
  await expect(page).toHaveURL(/lessons\/2$/);
});

Sources

FAQ

Should I add visual regression tests?: Add visual regression tests only for a shared component library or for output that leaves the browser, such as email templates. For the pages of a typical web app, browser assertions on the key actions catch the layout breaks that stop users at a lower cost.
Why do screenshot tests fail on CI but pass locally?: Screenshot tests fail on CI when the baselines came from another machine, because browser rendering varies with the operating system, fonts and hardware. Generate the baselines in the container image that CI runs.
Should I use visual regression tests for a component library?: Yes, a shared component library earns one screenshot per component variant, because a change shows on screens its author never opens. Render each variant with fixed props, so the pixels change only when the component does.
Can visual regression tests replace end-to-end tests?: No, a screenshot shows that a page looks the same, not that its form submits or its data is right. Keep a browser test for each key user path.