Should I test that?

Infrastructure and CI

Whether to test infrastructure code, Dockerfiles, pipelines, deployment scripts, health checks and alerts.

Most asked

  1. Should I run tests in the Docker build?Yes
  2. Should I test the production build?Yes
  3. Should I test in staging?Yes
  4. Should I test against every language version?Code under test: Yes
  5. Should integration tests run in CI?Yes

Yes13

  • Should I run tests again after merging a pull request?Yes

    Yes, run the full test suite again on every push to main and deploy only a commit that passed there, because the pull request build tested your branch merged with an older main.

  • Should I run tests in the Docker build?Yes

    Yes, run the unit tests during the Docker build, in a test stage that installs the same runtime and packages as the image you ship; make CI build that stage, and run the integration tests against the finished image before you push it.

  • Should I test against every language version?

    Yes, test a package against every language version it declares as supported, with one CI matrix job per minor version from the oldest to the newest.

    Code under test: Yes

  • Should I test backups?Yes

    Yes, test backups with a scheduled job that restores the newest backup into an empty database every week and checks that the data is recent and complete; a backup job that exits 0 proves only that it wrote a file.

  • Should I test disaster recovery?Yes

    Yes, test disaster recovery with a scheduled drill that rebuilds production in a second region from code and the newest backup copy, checks the recovered data, and fails when the rebuild takes longer than the recovery time you promise.

  • Should I test Dockerfiles?Yes

    Write one smoke test in CI that starts the built image, waits for its health endpoint and loads every native library the application uses, and do not assert the Dockerfile's contents.

  • Should I test in staging?Yes

    Yes, run a short automated smoke test against staging after each deploy and promote the same build to production only when it passes; keep the full test suite in CI and do not repeat it in staging.

  • Should I test infrastructure as code?Yes

    Yes, test infrastructure as code with plan-level tests in CI that assert the settings protecting data and uptime, such as backup retention and deletion protection; do not assert every attribute or create real resources on every change.

  • Should I test shell scripts?Yes

    Yes, give each shell script that CI, cron or a teammate runs one bats-core test that runs the whole script in a temporary folder with fake versions of the commands that reach other systems, and run ShellCheck on every script in CI.

  • Should I test the production build?Yes

    Yes, run one smoke test in CI against the production build: serve the built output, open each top-level route in a headless browser and fail on any script error; keep the unit tests on the source.

  • Should I unit test CDK code?Yes

    Yes, unit test CDK code minimally: one test per stack that asserts its tables and buckets keep their logical IDs; leave wiring errors to the staging deploy, and do not assert every declared property.

  • Should integration tests run in CI?Yes

    Yes, run integration tests in CI on every pull request, against the production database engine in a service container, and block the merge when one fails; move only the tests that call a third-party sandbox to a scheduled run.

  • Should tests run on every branch?Yes

    Yes, run the unit and integration tests on every branch that opens a pull request, make that run a required check before merge, and run the tests again on main after each merge.

Test it differently2

  • Is chaos engineering worth it?

    No, a chaos engineering program is not worth it for a typical product team: put a timeout on every outbound call, alert on each dependency's latency and error rate, and give on-call a switch that turns off each optional dependency.

    Code under test: Test it differently

  • Should I load test in production?

    No, do not load test production as a routine check: watch its capacity instead, with an alert when database CPU, connection pool use or the 95th percentile latency at the daily peak rises after a release.

    Code under test: Test it differently