QA Interview Hub · Knowledge base · Quizzes

API

Contract testing in practice

Consumer-driven contracts: where they sit in the test pyramid

In a multi-service system the most expensive bugs live at the seams: a provider renamed a field and the consumer found out on staging or in production. The classic answers have weak spots: e2e tests are slow, flaky and need an environment with every dependency up; integration tests against a real provider break on foreign data and unavailable services; mocks in a consumer's unit tests drift from the real API over time, and everything stays green until production falls.

Contract testing closes this gap. The idea of consumer-driven contracts (CDC): in its own test the consumer describes the minimum it needs from the provider (a request and the shape of a response). The description is stored as a contract, and the provider checks in its own CI that the real code fulfils it. No test needs both services at once, so everything is fast and deterministic.

Place in the pyramid. Contract tests sit between unit and integration/e2e: they check the shape of communication between two services, not business logic. They do not replace:

  • the provider's functional tests: a contract says the status field is a string, not that an order is paid correctly;
  • a small number of end-to-end tests on critical paths: they check configuration, the network, authorization and real data;
  • tests where there are many unknown consumers (a public API): there an OpenAPI schema and breaking-change checks work better.

When CDC fits. Teams know each other and live in one company (microservices, BFF + frontend, mobile client + backend), the API changes often and e2e has become the release bottleneck. When it does not: a single monolith, one team on both sides, or unknown consumers.

The business result is easy to measure: fewer incidents caused by API changes, independent service releases without an "alignment meeting", and a short pipeline where e2e stays as a small smoke set.

Pact end to end: consumer test, Broker, can-i-deploy

The full Pact cycle has four steps.

  1. Consumer test. The client code (not the UI!) calls a mock provider that Pact starts locally. The test describes an interaction: a state (given), a request and the expected response. When the test passes, a pact file (JSON) is generated;
  2. Publishing. The consumer's CI publishes the pact to a Pact Broker (or PactFlow) with a version (usually the git SHA) and a branch: pact-broker publish ./pacts --consumer-app-version $GIT_SHA --branch $BRANCH;
  3. Provider verification. The provider runs a Verifier that fetches pacts from the Broker, replays each request against the really running provider (with stubbed dependencies) and compares the response. The result is published back with the provider version;
  4. Deploy decision. Before a release either side asks: pact-broker can-i-deploy --pacticipant web-app --version $GIT_SHA --to-environment production. The Broker answers from a matrix of "consumer version x provider version currently in that environment"; after deploying we record the fact: pact-broker record-deployment --environment production.

Provider states. An interaction says "when order 42 exists, GET /orders/42 returns…". During verification the provider must bring the system into that state: it implements stateHandlers that create test data or configure stubs. States must be independent and repeatable.

Versioning and branches. The modern model: pacticipant version = git SHA, the branch is passed separately (--branch), and "what is in production now" is tracked by environments (record-deployment, record-release for libraries and mobile apps). Old tags (--tag) are outdated for this purpose. Verification selects pacts through consumerVersionSelectors, usually mainBranch, matchingBranch and deployedOrReleased.

Pending pacts and WIP. A new pact from a consumer branch that the provider does not yet fulfil should not break the provider's build: with enablePending: true its verification does not block until it has passed at least once. includeWipPactsSince pulls such "work in progress" pacts in automatically.

Webhooks. To avoid waiting for the provider's next build, the Broker has a webhook on the contract_requiring_verification_published event: a new or changed contract triggers verification in the provider's CI, and the result goes back to the Broker and to the consumer's PR status.

Key detail: without can-i-deploy, Pact is just "extra tests". It is the matrix check before deploying that guarantees this combination of versions has been verified.

Pact JS/TS: an example, matchers and what not to put in a contract

An example of a consumer test in Pact JS (the @pact-foundation/pact package, the PactV3 API; for spec v4 there is PactV4 with the same idea):

import { PactV3, MatchersV3 } from '@pact-foundation/pact';
const { like, eachLike, regex } = MatchersV3;

const provider = new PactV3({ consumer: 'web-app', provider: 'orders-api' });

it('returns an order by id', async () => {
  provider
    .given('order 42 exists')
    .uponReceiving('a request for order 42')
    .withRequest({ method: 'GET', path: '/orders/42' })
    .willRespondWith({
      status: 200,
      headers: { 'Content-Type': 'application/json' },
      body: {
        id: like(42),
        status: regex('NEW|PAID|SHIPPED', 'NEW'),
        items: eachLike({ sku: like('A-1'), qty: like(1) }),
      },
    });

  await provider.executeTest(async (mockServer) => {
    const order = await new OrdersClient(mockServer.url).getOrder(42);
    expect(order.items.length).toBeGreaterThan(0);
  });
});

Matchers. An example value in a contract does not mean "must be exactly this". like(x) checks the type, eachLike(template) an array with at least one element of that shape, regex(pattern, example) a string matching a pattern; there are also integer, datetime, uuid. Without matchers Pact compares values literally and the provider fails on every data change.

A consumer test must run the real client code (the HTTP client, parsing), not check the mock itself. The goal: if the client expects a field, it is in the contract; if the client does not read it, it is not.

What NOT to put in a contract:

  • Business logic and exact values. The contract says "total is a number", not "total = 150 for this cart". Logic is checked by the provider's functional tests;
  • Fields the consumer does not use. Extra fields bind the provider for no benefit and hinder its refactoring;
  • Every error code "just in case". Only those the client really handles (404, 409, 422 with an error body);
  • UI details. A test through a browser instead of the client layer makes the contract slow and brittle.

A useful rule: "write a contract like an acceptance test for the consumer's need", not as a copy of the provider's response.

Bi-directional, schema-based and message contracts; adopting it in a team

Bi-directional contract testing (a PactFlow feature) removes the need for provider states and verifying each pact. The provider publishes its OpenAPI specification together with the results of its own tests ("the specification really matches the code"), the consumer publishes a pact with its expectations (it can be generated from mocking tools such as Mock Service Worker or Cypress), and PactFlow statically compares them for compatibility. Pros: a low barrier to entry, the provider need not maintain states. Cons: a weaker guarantee. The correctness of the spec depends on what confirms it, and the comparison does not execute a real request.

The provider-driven (schema-based) approach. When consumers are many or unknown, it is easier to lean on the provider's OpenAPI: a spec linter, breaking-change checks between versions (for example oasdiff) and client generation. This complements CDC rather than competing with it.

Schema-based fuzzing. Tools such as Schemathesis read OpenAPI and generate hundreds of requests (boundary values, missing fields, wrong types), checking that responses match the schema and the server does not return 500: st run openapi.yaml --url http://localhost:8080. It finds mismatches between the spec and reality, but does not check what a specific consumer needs.

Asynchronous interactions. For queues and events (Kafka, SQS, RabbitMQ) Pact has message pacts: the contract describes the shape of a message. The consumer checks that its handler accepts such a payload; in verification the producer proves its code generates a message of that shape. The transport takes no part in the check, so topic and broker configuration is verified separately.

Typical pitfalls:

  • An over-specified contract: exact values, extra fields, element order. The provider fails without a real break and the team stops trusting the tests;
  • Shared fixtures and dependent provider states: the "order exists" state relies on data from another test, and verification is unstable;
  • Verifying against the wrong version: the provider's main is checked although production runs another version. Cured with selectors and can-i-deploy --to-environment;
  • A contract without can-i-deploy: pacts exist, but nothing blocks a deploy;
  • Verifying against the provider's real dependencies: that is an integration test again. Dependencies must be stubbed.

How to adopt it. Start with one critical edge (for example frontend → orders-api), not the whole system. Agree on a Broker (self-hosted or PactFlow), pacticipant names and the version model (SHA + branches + environments). Wire verification into the provider's CI and can-i-deploy before deploy, and only then expand. The QA role: help write "thin" contracts, make sure states are independent and show the value through metrics (breaking changes caught, fewer e2e).