Should I test that?

Should I test in staging?

Verdict

Yes

Yes, run a short automated smoke test against staging after each deploy and promote the same build to production only when it passes; keep the full test suite in CI and do not repeat it in staging.

Why

Yes, run one automated smoke test in staging and let it gate the promotion to production; rule R12 gives Test minimally when CI already runs the full suite. Blast radius is users, because the same build goes on to customers, and Reversibility is with-effort, because a redeploy leaves failed requests and support tickets behind. Change frequency is regularly: what CI never sees, such as environment variables, secrets and service addresses, changes about once a month. Detectability is same-day, because a wrong setting turns a feature into errors that the error tracker reports within hours. Test cost is moderate: a few browser checks pointed at the staging URL.

When the decision changes
WhenDecisionWhy
The release changes checkout, and staging takes orders in the payment provider's test modeTest mandatory: before each promotion, place one order in staging with a test card and check the order and the chargeBlast radius rises to money and Reversibility to costly, because orders lost while checkout is broken stay lost
The release changes a nightly export to a partner, and a wrong setting makes the job send nothing without an errorTest: run the export once in staging against the partner's sandbox and assert that the file arrivedDetectability moves to eventually, because a job that sends nothing raises no alert
The release carries a migration that rewrites existing rowsTest it differently: rehearse the migration in staging on a masked copy of production, compare row counts with a query, and take a backup firstChange frequency falls to once and Reversibility rises to costly, because a code rollback does not restore rewritten rows
Staging needs its own copy of eight services and a weekly data refresh, so tests there fail on drift more often than on bugsDo not test in staging; rely on the CI suite and open the main pages in production after each deployTest cost rises to heavy, because keeping staging close to production takes days each month
The release changes an integration with a partner system that has no test instance, and a wrong call makes the partner drop records without an errorTest it differently: release to a small share of production traffic first, alert on the share of failed or empty partner responses, and keep the previous version readyTest cost rises to heavy and Detectability to eventually, because only the partner's production system shows the change, and dropped records raise no error

What breaks if you don't test

CI runs the code against a fresh test database, fakes and a test config, and it runs migrations on empty tables. When a deployed setting is wrong, the application still starts and a health check that skips the broken service returns 200, so the staging deploy reports success. Nobody opens staging before the promotion, and customers hit the broken feature first.

What you lose if you over-test

A full regression in staging repeats what CI already ran and adds hours between merge and production. Tests against a shared staging environment fail on data that another team changed, and after a few false alarms people promote red builds. A green staging run also says nothing about a setting that exists only in production's config, the gap the configuration answer covers.

How to test

  1. Deploy the build to staging, then run the smoke suite with Playwright's baseURL set to the staging address.
  2. Check one path per backing service: sign in, load a page from the database, run a search through the cache, save a record and read it back.
  3. Make the production job depend on the smoke job, inside a GitHub deployment environment.
  4. Keep staging on the same database engine and versions as production, as twelve-factor dev/prod parity asks, with invented accounts in place of customer data.

When the answer changes

  • The release touches checkout or anything else that moves money.
  • The release carries a migration that rewrites existing rows.
  • Keeping staging in step with production takes days each month.

Real incident + Code example

The cache variable nobody checked in staging

On a booking platform I worked on, every merge deployed to staging and, an hour later, to production, with no check in between. A release renamed the REDIS_URL setting to CACHE_URL in code, while the deploy config of both environments kept the old name. The client fell back to localhost, the health endpoint never touched the cache, and both deploys reported success. Search returned 500 errors for 40 minutes until support tickets reached the on-call developer. Staging had shown the same errors an hour earlier. Since then a smoke job stands between the two deploys:

jobs:
  deploy-staging:
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy.sh staging "$GITHUB_SHA"
  smoke-staging:
    needs: deploy-staging
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npx playwright install --with-deps chromium
      - run: npx playwright test smoke/
        env:
          BASE_URL: https://staging.example.com
  deploy-production:
    needs: smoke-staging
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy.sh production "$GITHUB_SHA"

The smoke folder holds a search test that fails when the cache is unreachable.

FAQ

Should QA test on staging and then again on production?

No, QA should not repeat the full pass on production when the same build already passed it on staging. The second pass can only find what differs between the environments: settings, secrets and data. Run a short read-only smoke test against production after the promotion instead.

Do I need a staging environment?

You need a staging environment when releases depend on settings, backing services or migrations that CI does not run, which holds for most web applications with a database. A team whose staging drifts from production and takes days a month to maintain gets more from its CI suite and a look at production after each deploy.

Should I run the full test suite against staging?

No, run the full test suite in CI and only a short smoke suite against staging.