Regression suite

The subset of tests rerun on every change to catch what used to work.

Mandatory for 17 of 46 product types. Usually run by a QA engineer.

What it catches

• reintroduced bugs

What it misses

• new bugs in new features

Which product types need it and why

Done conditions used on playbook pages

Each mandatory cell has its own done condition, written for that product type.

• clinical trial management software (Healthcare): Before each release, the validated scripts for edit checks, signatures and audit trail rerun and pass on the release candidate, and the result is filed in the validation package.
• data pipeline (Infrastructure and developer products): Reconciliation and edge-file cases rerun on every change, and any change in sample output totals blocks the merge.
• desktop application (Infrastructure and developer products): Install, update, uninstall and local-port cases rerun before each release, and any new listening port or deleted user file blocks it.
• e-signature service (B2B SaaS): The certificate, consent and tamper cases rerun on every release, and a missing certificate field or a passed tamper case blocks the release.
• electronic health record system (Healthcare): The regression set for ordering, results and medication workflows passes on each build before it reaches a live facility.
• email marketing platform (B2B SaaS): The unsubscribe, suppression and consent-capture cases rerun on every release, and a single failure blocks it.
• ERP system (B2B SaaS): The payroll, posting and period-close cases rerun on every release, and a single difference in totals blocks it.
• identity and access management service (B2B SaaS): On every release, a wrong password through the directory connector is refused, a session revoked in the admin console no longer opens the sample relying app, recovery still asks for the user's enrolled MFA factor, and any failure blocks the release.
• LLM-based assistant (Infrastructure and developer products): A fixed question set with approved answers is rerun after every prompt, model or retrieval change, and a named owner reviews every changed answer.
• machine learning model service (Infrastructure and developer products): Each new model version is scored on the same held-out set as the live one, and a worse result for any group blocks the release.
• mobile banking app (Fintech): Every app update replays the account switch, transaction list and statement download cases, and no screen shows a transaction belonging to another test customer.
• neobank (Fintech): Every release reruns the sign-up screening and ledger posting cases, and none changes outcome against the previous release.
• peer-to-peer payment app (Fintech): After each release, a payment marked private by either party never appears on the public or friends feed of a third test user.
• software as a medical device (Healthcare): The regression set traced to each software requirement passes on the release candidate and the results are filed with the release record.
• SDK or client library (Infrastructure and developer products): The public API surface diff and the crash cases rerun on every release, and a removed or changed public method without a major version bump blocks the release.
• social network (Consumer apps): After each release, privacy defaults for new teen and adult accounts match the approved settings list.
• video conferencing tool (B2B SaaS): The intruder, recording-access and uninstall cases rerun on every release, and any intruder join blocks the release.

Tools

Alphabetical. Not recommendations.