Is it worth it to run E2E tests in production?

A short read-only subset of the end-to-end tests is worth running in production after each deploy; the full suite is not. The subset finds failures that only production has, while the full suite writes fake orders into real data.

Should I run smoke tests after deploying to production?

Yes, run a smoke test after every production deploy and roll the deploy back when it fails. A check that signs in finds a deploy that broke sign-in before customers report it.

How do I test in production without affecting real users?

Test in production with one account marked as synthetic, so analytics, email and billing skip it, and keep the checks read-only. For a path that must write, such as a purchase, undo the write in the same script.

Does testing in production replace staging?

No, testing in production does not replace staging or CI, because it runs after customers can already reach the build. Run the full suite before the deploy, and the production smoke test after it.

Should I test in production?

Verdict

Yes

Yes, run a short read-only smoke test against production after each deploy, with one test account that analytics, email and billing skip, and roll back when it fails; keep the full end-to-end suite in CI.

Why

Blast radiususers
Change frequencyregularly
Detectabilitysame-day
Reversibilitywith-effort
Test costmoderate

Yes, run a short smoke test against production after each deploy; rule R12 gives Test minimally. Blast radius is users, because a production failure reaches customers, and Reversibility is with-effort, because a rollback leaves failed requests and support tickets behind. Change frequency is regularly: what only production has, such as live secrets, CDN rules and partner keys, changes about once a month. Detectability is same-day, because a broken sign-in reaches the error tracker or support within hours. Test cost is moderate: a few read-only browser checks and one test account.

When the decision changes
When	Decision	Why
The deploy changes checkout, and only production holds the live payment keys	Test mandatory: cover every charge path in CI, and after each deploy buy a hidden one-euro product in production and refund it in the same script	Blast radius rises to money and Reversibility to costly, because orders lost while checkout is broken stay lost
The deploy changes which customer sees which records, such as an account export	Test mandatory: after each deploy, sign in to production as two test customers and assert that neither sees the other's records	Blast radius rises to safety-or-legal and Detectability to never, because a leaked record looks like normal output
A production-only setting can stop sign-up emails sent by a background job whose failures nobody watches	Test: every 15 minutes, sign up a test address in production and assert that the email reaches a test inbox	Detectability moves to eventually, because new users who get no email leave without reporting it
Every useful production check would write orders that the warehouse and billing systems act on	Do not test in production; rely on the CI suite, run the smoke test in staging before promotion, and open the main pages after each deploy	Test cost rises to heavy, because every downstream system needs a rule that skips test orders
The site is static pages, the host serves the exact build its preview URL showed, and a rollback takes one click	Do not test in production; open the preview before you promote it	Detectability moves to immediately and Reversibility to trivial, because the author sees the build before customers do

What breaks if you don't test

CI and staging run the build without what exists only in production: the CDN, the live payment and mail keys, the real DNS records. A deploy that breaks one of them passes every earlier stage, and a health check that asks only the app server reports success. The first request that hits the broken part comes from a customer, and you hear about it from a support ticket hours later.

What you lose if you over-test

The full end-to-end suite in production writes orders, accounts and messages into the data that reports and invoices read, and one test order that a downstream system does not skip ships a real parcel. Production tests fail on data you do not control, and after a few false pages the on-call developer ignores them. A green smoke run also misses features behind flags that the test account lacks.

How to test

After each deploy, run four Playwright checks with baseURL set to the production address: sign in, open the main page, run a search, load one script.
Mark the test account as synthetic, so analytics, email and billing skip it.
Roll the deploy back automatically when a check fails.
Run the same checks every five minutes as a monitoring probe, as the SRE book's testing chapter suggests, and page on-call after two failures in a row.

Procedure and references

When the answer changes

The deploy touches checkout or payouts.
A production failure is quiet: a job, an email or a partner call that raises no error.
Every useful production check writes data that other systems act on.

Real incident + Code example

The CDN rule that only production had

On a B2B scheduling app I worked on, staging served JavaScript from the app server, while production put CloudFront in front and sent /static/* to an S3 bucket. A build tool upgrade moved the bundles to /assets/, so production sent those requests to the app server, which answered with the HTML of its 404 page. The browser refused to run HTML as a script, so the Sign in button did nothing. CI and staging passed, and the deploy reported success. For 50 minutes customers could not sign in, until the third support ticket reached the on-call developer. Since then these checks run after each deploy, and a failure rolls the deploy back:

// smoke/production.spec.ts, run by the deploy job against the live site.
// The account has is_synthetic = true: analytics, email and billing skip it.
import { test, expect } from '@playwright/test';

test.use({ baseURL: 'https://app.example.com' });

test('a customer can sign in and see the calendar', async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel('Email').fill(process.env.SMOKE_EMAIL!);
  await page.getByLabel('Password').fill(process.env.SMOKE_PASSWORD!);
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page.getByRole('heading', { name: 'Calendar' })).toBeVisible();
});

test('the login page loads its scripts as JavaScript', async ({ request }) => {
  const html = await (await request.get('/login')).text();
  const script = html.match(/src="([^"]+\.js)"/)![1];
  const response = await request.get(script);
  expect(response.headers()['content-type']).toContain('javascript');
});

Sources

Should I test in staging?Yes
Should I load test in production?Code under test: Test it differently
Are end-to-end tests worth it?Yes
Should I write tests before the beta release?Yes
Should I test with production data?Test it differently

FAQ

Is it worth it to run E2E tests in production?: A short read-only subset of the end-to-end tests is worth running in production after each deploy; the full suite is not. The subset finds failures that only production has, while the full suite writes fake orders into real data.
Should I run smoke tests after deploying to production?: Yes, run a smoke test after every production deploy and roll the deploy back when it fails. A check that signs in finds a deploy that broke sign-in before customers report it.
How do I test in production without affecting real users?: Test in production with one account marked as synthetic, so analytics, email and billing skip it, and keep the checks read-only. For a path that must write, such as a purchase, undo the write in the same script.
Does testing in production replace staging?: No, testing in production does not replace staging or CI, because it runs after customers can already reach the build. Run the full suite before the deploy, and the production smoke test after it.