Methodology
46 of 73 product types are published, from 315 evidence records. This page states the rules that decide what appears on a playbook page.
The page skeleton
Every playbook page has the same 12 sections in the same order. An automated audit fails any page that is missing, reorders or merges a section.
- Direct answer
- Definition and scope
- Risk map
- Mandatory tests
- Recommended and optional
- Not needed and why
- Order of work
- Tools
- What to send
- FAQ
- Evidence
- Disclaimer
Evidence grades
- primary
- Regulator page, store guideline, standard, official post-mortem, court or fine record.
- secondary
- Engineering incident write-up, conference talk, audited industry report, or a by design analysis backed by a primary general requirement.
- practitioner
- Named practitioner post or answer. Two are needed to attach a risk.
Store guidelines and regulator pages are re-opened within 90 days of each build; standards within 12 months. Post-mortems and incident records do not expire and show their date. A link that fails to load twice is dropped and every risk that relied on it is re-checked.
How a risk is attached
- One evidence record of grade primary or secondary, or two of grade practitioner, that applies to the product type and the risk.
- By design analysis (from the structure of the product, not an incident) is allowed only for money loss on products that move money, data exposure on products that hold personal or health data, and store rejection on products shipped through a store. It is capped at secondary grade, labelled in the risk map, and still cites the general requirement (a store guideline, a standard, a regulator rule).
- A single dramatic incident with no second source is graded practitioner and labelled.
- Each risk cell carries a note written for that product type. An audit swaps notes between random pairs of pages; a note that still reads as correct on the other page is rewritten.
- harm
- Physical or psychological harm to a user.
- money
- Money leaves, fails to arrive or is taken by fraud.
- legal
- A fine, enforcement action or lawsuit.
- data
- Data is exposed or silently wrong.
- outage
- The product is down or does not recover.
- distribution
- A store or platform rejects or removes the product.
- trust
- Users or operators act on wrong output or lose their actions.
How a test becomes mandatory
- A test is mandatory only when it addresses at least one attached risk; the risk ids are shown next to it.
- Each published product type has 6 to 14 mandatory tests. Above 14, the lowest-severity tests move to recommended.
- Recommended tests carry the condition that would make them mandatory. Optional tests carry the reason they may help. Not-needed tests carry the reason and what would change it.
- Order: the release smoke check first where it is mandatory, then the test for the highest-severity risk, then the rest by severity (harm, money, legal, data, outage, distribution, trust), then recommended tests.
- Each mandatory test has a done condition: one sentence, present tense, that a non-engineer can check without reading code.
Publishing threshold
A product type is published only with at least 4 attached risks at grade primary or secondary and at least 6 mandatory tests. Types below the threshold are listed with their counts on the not-published page. Changing these numbers requires a dated entry on the changelog.
Tools
The tool list is a fixed set of names per test type, shown alphabetically with platform and licence. Tool names appear only in section 8 of a playbook page, on the tools page and on test type pages.
Refresh cadence
Refresh: quarterly. Each refresh re-opens every evidence link, updates verification dates, drops dead records, rebuilds risk maps, test plans, checklists and downloads, and adds a dated entry to the changelog. Terms are defined in the glossary.