Contract tests
Checks that a provider API still matches what each consumer expects.
Mandatory for 3 of 46 product types. Usually run by a developer.
What it catches
- breaking API changes between services or partners
What it misses
- business logic errors inside a service
Which product types need it and why
| Product type | Status | Reason or condition |
|---|---|---|
| accounting and invoicing SaaS | mandatory | Risks: Fine or enforcement by a regulator |
| buy now pay later service | mandatory | Risks: Money leaves or fails to arrive |
| SDK or client library | mandatory | Risks: Third-party dependency fails |
| clinical trial management software | recommended | Becomes mandatory when a study uses more than one external data provider. |
| crypto exchange | recommended | Becomes mandatory when trading API changes have broken market maker bots. |
| data pipeline | recommended | Becomes mandatory when a source team can change its schema without notice. |
| e-signature service | recommended | Becomes mandatory when customers' systems depend on envelope status callbacks. |
| ecommerce store | recommended | Becomes mandatory when a second payment or shipping provider is added. |
| electronic health record system | recommended | Becomes mandatory when a lab, pharmacy or second record system consumes the system's interfaces on its own release schedule. |
| email marketing platform | recommended | Becomes mandatory when partners sync contacts through the API. |
| ERP system | recommended | Becomes mandatory when a third feeder system is added. |
| EV charging app | recommended | Becomes mandatory when more than one roaming partner is connected. |
| identity and access management service | recommended | Becomes mandatory when a claim change breaks relying apps in a recorded incident. |
| insurance app | recommended | Becomes mandatory when more than one data vendor feeds the rating engine. |
| lending platform | recommended | Becomes mandatory when underwriting reads more than one external data provider. |
| LLM-based assistant | recommended | Becomes mandatory when more than one model provider or model version sits behind the assistant. |
| online marketplace | recommended | Becomes mandatory when the payment provider or shipping label provider changes its API version. |
| machine learning model service | recommended | Becomes mandatory when outside clients integrate with the endpoint. |
| mobile banking app | recommended | Becomes mandatory when the bank plans a core platform migration or supplier change. |
| pharmacy delivery app | recommended | Becomes mandatory when a second claims or courier provider is added. |
| ride-hailing app | recommended | Becomes mandatory when rider and driver apps release on different schedules from the backend. |
| subscription box service | recommended | Becomes mandatory when a second billing provider or payment method is added. |
| stock trading app | recommended | Becomes mandatory when more than one market data source feeds the same screen. |
| travel booking app | recommended | Becomes mandatory when a supplier changes its booking API and bookings fail or show wrong details. |
| messaging app | optional | May help when desktop and web clients release on a different schedule from the API. |
| dating app | not needed | Not needed: one team owns the app and its API. Becomes recommended when a partner app or web client consumes the same API. |
| desktop application | not needed | Not needed: no attached risk comes from API changes. Becomes recommended when the app depends on a versioned backend API used by other clients. |
| expense management tool | not needed | Not needed as a separate test: issuer data is checked by integration cases. Becomes recommended when the tool exports to several accounting systems. |
| fleet and logistics tracking | not needed | Not needed: devices and apps talk to one first-party API. Becomes recommended when third-party tracker brands are supported. |
| government services portal | not needed | Not needed: the portal is the only consumer of its backend API. Becomes recommended when several agencies consume the portal API. |
| grocery delivery app | not needed | Not needed: one team owns the shopper app, picker app and API. Becomes recommended when partner retailers connect through a public API. |
| smart home or IoT app | not needed | Not needed: the app talks to one first-party device API released with it. Becomes recommended when third parties build on the device API. |
| kids app | not needed | Not needed: one team owns the app and its backend. Becomes recommended when partners consume the API. |
| language learning app | not needed | Not needed: one team owns the apps and API. Becomes recommended when schools or partners use the API. |
| loyalty program app | not needed | Not needed: the integration test covers the point of sale feed. Becomes recommended when partners call a public points API. |
| mental health app | not needed | Not needed while one team ships the app and its API together. Becomes recommended when an employer or insurer integration consumes the API. |
| mobile game | not needed | Not needed: one team owns client and server. Becomes recommended when a separate team or partner ships against the game API. |
| neobank | not needed | Not needed: the partner bank feed is covered by integration and data-integrity tests. Becomes recommended when the neobank adds a second partner bank with a different feed format. |
| peer-to-peer payment app | not needed | Not needed: the app has one bank and one card processor, covered by integration tests. Becomes recommended when a second partner or public API ships. |
| patient portal | not needed | Not needed while the portal is the record vendor's own module released together with it. Becomes recommended when the portal reads the record through a separately released API. |
| software as a medical device | not needed | Not needed while the app, firmware and cloud API are released together as one cleared version. Becomes recommended when a third-party app or clinic system reads the device data API. |
| social network | not needed | Not needed: apps talk to one first-party API released with them. Becomes recommended when a public developer API ships. |
| telemedicine app | not needed | Not needed while the app and its backend are built and released by the same team. Becomes recommended when a pharmacy, lab or EHR partner consumes the visit API. |
| event ticketing platform | not needed | Not needed: one team owns the queue, seat map and checkout services. Becomes recommended when promoters or resale partners connect through a public API. |
| video conferencing tool | not needed | Not needed: no attached risk comes from partner integrations. Becomes recommended when a public meetings API is sold. |
| wearable companion app | not needed | Not needed: the app talks to one first-party sync API released with it. Becomes recommended when third parties consume the sync API. |
Done conditions used on playbook pages
Each mandatory cell has its own done condition, written for that product type.
- accounting and invoicing SaaS (Fintech): Each VAT return submission in the tax authority sandbox is accepted, and the fraud prevention header check reports no missing field.
- buy now pay later service (Fintech): A merchant plugin release that changes the refund or capture message fails a contract check before it reaches merchants.
- SDK or client library (Infrastructure and developer products): Before any server config change goes live, staging responses match the schema that every supported SDK version expects.
Tools
Alphabetical. Not recommendations.
contract tests
- PactConsumer-driven contract testing with a broker. apiopen-source
- PostmanAPI client with collections and scripted assertions. apifreemium
- SchemathesisProperty-based testing from OpenAPI and GraphQL schemas. apiopen-source