Should I test that?

All questions

173 testing questions, each with one decision derived from the framework.

Code

AI and LLM products

Do I need evals for my LLM app?Test it differently
Do I still need human review of LLM outputs?Code under test: Test it differently
Should AI write unit tests?Yes
Should I evaluate an embedding model on my own data?Yes
Should I evaluate RAG retrieval separately?Yes
Should I mock LLM calls in tests?Code under test: Yes
Should I retest my app after an LLM model upgrade?Code under test: Test it differently
Should I test a vibe-coded app?Yes
Should I test AI agents?Test it differently
Should I test AI-generated code?Yes
Should I test chunking for RAG?Yes
Should I test data science code?Yes
Should I test for hallucinations?Test it differently
Should I test LLM tool calls?Yes
Should I test MCP servers?Yes
Should I test prompts?Test it differently
Should I test structured outputs from an LLM?Yes
Should I use an LLM as a judge?Code under test: Test it differently
Should I use TDD with AI coding assistants?Code under test: Yes
Should I write tests before letting AI refactor code?Yes
Should I write unit tests for machine learning code?Yes
Should LLM evals run in CI?Code under test: Test it differently
Should tests use a real vector database?Yes

Product decisions

Test depth

External boundaries

Interface

Is automated UI testing worth it?Yes
Is visual regression testing worth it?No
Should I test accessibility with automated tools?Yes
Should I test component props?No
Should I test CSS?No
Should I test every screen size?Test it differently
Should I test in every browser?Test it differently
Should I test text content in the UI?No
Should I test views?Yes
Should I unit test UI components?Yes
Should I use data-testid attributes in tests?No

Data and state

Should I test data pipelines?Yes
Should I test data quality?Test it differently
Should I test for race conditions?Yes
Should I test SQL queries?Yes
Should I test stored procedures?Yes
Should I test with an in-memory database?Code under test: Yes
Should I test with production data?Test it differently
Should I use random data in tests?No

Regulated environments

Does GDPR require penetration testing?Code under test: Yes
Does HIPAA require penetration testing?Code under test: Yes
Does ISO 27001 require penetration testing?Code under test: Yes
Does SOC 2 require penetration testing?Code under test: Yes
Is 100% coverage required for safety-critical software?Code under test: Yes
Should I anonymize data before testing?Yes

Security

Infrastructure and CI

Is chaos engineering worth it?Code under test: Test it differently
Should I load test in production?Code under test: Test it differently
Should I run tests again after merging a pull request?Yes
Should I run tests in the Docker build?Yes
Should I test against every language version?Code under test: Yes
Should I test backups?Yes
Should I test disaster recovery?Yes
Should I test Dockerfiles?Yes
Should I test in staging?Yes
Should I test infrastructure as code?Yes
Should I test shell scripts?Yes
Should I test the production build?Yes
Should I unit test CDK code?Yes
Should integration tests run in CI?Yes
Should tests run on every branch?Yes

Mobile

Is testing on an emulator enough?Code under test: Yes
Should I test my app on a slow network?Yes
Should I test on low-end devices?Yes
Should I test React Native apps?Yes
Should I test view models?Yes

Practices

Do doctests replace unit tests?Code under test: Yes
Is it unprofessional not to write tests?Code under test: Yes
Is unit testing necessary in web development?Code under test: Yes
Is unit testing worth it?Yes
Should all tests be automated?No
Should beginners write unit tests?Yes
Should code review come before testing?Code under test: Yes
Should developers write their own tests?Code under test: Yes
Should I change code to make it testable?Yes
Should I commit failing tests?Code under test: Yes
Should I mock all dependencies in unit tests?Code under test: Yes
Should I test in production?Yes
Should I test what static analysis already checks?No
Should I use BDD?Code under test: Yes
Should I use mocks in unit tests?Code under test: Yes
Should I use TDD?Code under test: Yes
Should I write a failing test before fixing a bug?Yes
Should I write tests before code?Code under test: Yes
Should I write tests before refactoring?Yes
Should I write tests for code I can prove correct?Yes