Contract testing with Pact (hands-on): the API change both teams' tests missed

The FoodNow app shows the customer their order total. It gets the total from the orders API, which another team builds. One day the API team tidies up how money is sent: "total_paise": 41000 becomes "total": {"amount_paise": 41000, "currency": "INR"}. They update their tests, and the tests pass. The app team changed nothing, so their tests pass too. The release goes out, and every order screen in the app breaks.

Both teams tested. What nobody tested was the agreement between them: the requests the app sends and the answers it needs back. That agreement is a contract, and contract testing checks it without running the app and the API together.

This post shows how contract testing works with Pact, the most widely used tool for it. Then it runs the story above for real, with FoodNow's app code and orders API. Every command and line of output below was run while writing this post.

Why not just test them together?

You can start the app and the real API in one environment and run tests across both. That's an end-to-end or integration test, and you should have a few. But as the only check on the agreement, they're a poor fit:

• They're slow and flaky. Every service, database and network link must be up at once.
• They tell you something broke, late. Often after both changes were merged.
• They don't scale. With ten apps and services, you need one environment where the right version of every one runs together.

And mocks don't solve it. In Part 5, a mock of the payment provider happily accepted calls the real provider would have refused. A mock only knows what you told it.

How Pact works

Contract testing splits the check in two. Each half runs in one team's normal test suite, with nothing else running:

1. The consumer test. The app team writes tests for their API client, against a mock server that Pact runs. Each test says "when the app sends this request, it needs this response". Pact checks that the app really sends that request, and answers with that response.
2. The pact file. When the tests pass, Pact writes every request and response to a JSON file, the contract.
3. The provider verification. The API team runs Pact's verifier against their real API. It replays every request in the pact file and checks that the real responses match.

Two words you'll see: the consumer is the side that calls (the app), and the provider is the side that answers (the orders API). A provider state is the situation a request needs, such as "order 7 exists". The API team writes a small function for each state, to set it up before the request is replayed.

This is called consumer-driven contract testing: the contract holds only what the consumer actually uses. If the API sends 30 fields and the app reads 4, only those 4 are in the contract, and the API team can change the other 26 freely.

Step 1: Set up

You need Python 3.10 or newer:

mkdir foodnow-contract && cd foodnow-contract
python -m venv .venv && source .venv/bin/activate
pip install pact-python==4.2.0 pytest==9.2.0 fastapi==0.162.4 uvicorn==0.54.0 requests==2.34.2 httpx==0.28.1
mkdir -p app_client orders_api tests/consumer tests/provider
touch app_client/__init__.py orders_api/__init__.py tests/__init__.py tests/consumer/__init__.py tests/provider/__init__.py
printf '[pytest]\npythonpath = .\n' > pytest.ini
export PACT_DO_NOT_TRACK=true

pact-python 4 runs Pact's core, written in Rust, inside Python, so there is nothing else to install. By default, Pact sends anonymous usage statistics (the Pact version and operating system); PACT_DO_NOT_TRACK=true turns that off.

In a real project the two halves live in two repositories, owned by two teams. Here they share one folder so you can run everything.

Step 2: The app's API client

This is the part of the FoodNow app that talks to the orders API. It reads an order, and it places a new one with a tip. Save it as app_client/client.py:

"""The FoodNow app's code for talking to the orders API."""
from dataclasses import dataclass

import requests


@dataclass
class Order:
    id: int
    status: str
    total_paise: int
    tip_paise: int


class OrdersClient:
    def __init__(self, base_url: str):
        self.base_url = base_url

    def get_order(self, order_id: int) -> Order | None:
        r = requests.get(f"{self.base_url}/orders/{order_id}", timeout=5)
        if r.status_code == 404:
            return None
        r.raise_for_status()
        d = r.json()
        return Order(d["id"], d["status"], d["total_paise"], d["tip_paise"])

    def place_order(self, items: list[dict], tip_paise: int) -> Order:
        r = requests.post(
            f"{self.base_url}/orders",
            json={"items": items, "tip_paise": tip_paise},
            timeout=5,
        )
        r.raise_for_status()
        d = r.json()
        return Order(d["id"], d["status"], d["total_paise"], d["tip_paise"])

Money is in paise (1 rupee = 100 paise) as a whole number, so there are no rounding errors.

Step 3: The consumer test writes the contract

Save this as tests/consumer/test_orders_contract.py. Each test describes one interaction, starts Pact's mock server, and calls the real client code against it:

from pathlib import Path

import pytest
from pact import Pact, match

from app_client.client import OrdersClient

PACTS = Path(__file__).parents[2] / "pacts"


@pytest.fixture
def pact():
    pact = Pact("foodnow-app", "orders-api").with_specification("V4")
    yield pact
    pact.write_file(PACTS)  # adds this test's interaction to the file


def test_get_order(pact):
    (
        pact.upon_receiving("a request for an order")
        .given("order 7 exists")
        .with_request("GET", "/orders/7")
        .will_respond_with(200)
        .with_body(
            {
                "id": match.int(7),
                "status": match.str("preparing"),
                "total_paise": match.int(40000),
                "tip_paise": match.int(4000),
            }
        )
    )
    with pact.serve() as srv:
        order = OrdersClient(str(srv.url)).get_order(7)
    assert order.total_paise == 40000
    assert order.tip_paise == 4000


def test_missing_order(pact):
    (
        pact.upon_receiving("a request for an order that doesn't exist")
        .given("order 999 does not exist")
        .with_request("GET", "/orders/999")
        .will_respond_with(404)
    )
    with pact.serve() as srv:
        assert OrdersClient(str(srv.url)).get_order(999) is None


def test_place_order(pact):
    (
        pact.upon_receiving("a new order with a tip")
        .with_request("POST", "/orders")
        .with_body({"items": [{"name": "paneer wrap", "qty": 2}], "tip_paise": 5000})
        .will_respond_with(201)
        .with_body(
            {
                "id": match.int(1),
                "status": match.str("placed"),
                "total_paise": 41000,
                "tip_paise": 5000,
            }
        )
    )
    with pact.serve() as srv:
        order = OrdersClient(str(srv.url)).place_order(
            [{"name": "paneer wrap", "qty": 2}], tip_paise=5000
        )
    assert order.total_paise == 41000

Two kinds of expected value are used on purpose:

• Matchers, like match.int(7), mean "any whole number; 7 is just an example". Use them where the exact value doesn't matter. The order id depends on what's in the API's database, so the contract shouldn't fix it.
• Exact values, like "total_paise": 41000, mean "exactly this". Two paneer wraps at ₹180 plus a ₹50 tip is ₹410, whatever is in the database. Here the value is the point. Step 7 shows why that choice matters.

The fixture is created fresh for each test, because a pact can't be changed after its mock server has started. Add tests/consumer/conftest.py to start every run with an empty pact folder:

import shutil
from pathlib import Path


def pytest_sessionstart(session):
    shutil.rmtree(Path(__file__).parents[2] / "pacts", ignore_errors=True)

Run it:

$ pytest tests/consumer -q
...                                                                      [100%]
3 passed in 0.07s

The contract is now in pacts/foodnow-app-orders-api.json. This is the part for reading an order:

{
  "description": "a request for an order",
  "providerStates": [{ "name": "order 7 exists" }],
  "request": { "method": "GET", "path": "/orders/7" },
  "response": {
    "status": 200,
    "body": {
      "content": { "id": 7, "status": "preparing", "tip_paise": 4000, "total_paise": 40000 }
    },
    "matchingRules": {
      "body": {
        "$.id":          { "combine": "AND", "matchers": [{ "match": "integer" }] },
        "$.status":      { "combine": "AND", "matchers": [{ "match": "type" }] },
        "$.tip_paise":   { "combine": "AND", "matchers": [{ "match": "integer" }] },
        "$.total_paise": { "combine": "AND", "matchers": [{ "match": "integer" }] }
      }
    }
  }
}

(We removed the headers and some fields Pact adds, to keep it short.) This file is what the app team hands to the API team.

Step 4: The provider verifies the contract

Here's version 1 of the orders API, in orders_api/app.py. It's a small FastAPI app that keeps orders in memory:

"""FoodNow orders API, version 1."""
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

MENU = {"paneer wrap": 18000, "masala chai": 4000}
ORDERS: dict[int, dict] = {}


class Item(BaseModel):
    name: str
    qty: int


class NewOrder(BaseModel):
    items: list[Item]
    tip_paise: int = 0


@app.get("/orders/{order_id}")
def get_order(order_id: int):
    if order_id not in ORDERS:
        raise HTTPException(404, "order not found")
    return ORDERS[order_id]


@app.post("/orders", status_code=201)
def place_order(body: NewOrder):
    food = sum(MENU[i.name] * i.qty for i in body.items)
    order_id = max(ORDERS, default=0) + 1
    return save_order(order_id, food, body.tip_paise)


def save_order(order_id: int, food_paise: int, tip_paise: int, status: str = "placed"):
    ORDERS[order_id] = {
        "id": order_id,
        "status": status,
        "total_paise": food_paise + tip_paise,
        "tip_paise": tip_paise,
    }
    return ORDERS[order_id]

The verification test starts the real API on a local port, tells Pact how to set up each provider state, and runs the verifier. Save it as tests/provider/test_verify_contract.py:

import threading
import time
from pathlib import Path

import pytest
import uvicorn
from pact import Verifier

from orders_api import app as api

PACTS = Path(__file__).parents[2] / "pacts"


@pytest.fixture(scope="module")
def server():
    config = uvicorn.Config(api.app, port=8123, log_level="warning")
    srv = uvicorn.Server(config)
    threading.Thread(target=srv.run, daemon=True).start()
    while not srv.started:
        time.sleep(0.05)
    yield "http://localhost:8123"
    srv.should_exit = True


def order_7_exists(action, parameters=None):
    if action == "setup":
        api.ORDERS[7] = {"id": 7, "status": "preparing",
                         "total_paise": 40000, "tip_paise": 4000}


def order_999_missing(action, parameters=None):
    api.ORDERS.pop(999, None)


def reset(action, parameters=None):
    if action == "setup":
        api.ORDERS.clear()


def test_orders_api_keeps_the_contract(server):
    verifier = (
        Verifier("orders-api")
        .add_transport(url=server)
        .set_coloured_output(enabled=False)
        .add_source(PACTS)
        .state_handler(
            {
                "": reset,  # for interactions with no given state
                "order 7 exists": order_7_exists,
                "order 999 does not exist": order_999_missing,
            },
            teardown=True,
        )
    )
    verifier.verify()

Run it with -s to see the verifier's report:

$ pytest tests/provider/test_verify_contract.py -q -s
Verifying a pact between foodnow-app and orders-api

  a new order with a tip (0s loading, 15ms verification)
    returns a response which
      has status code 201 (OK)
      includes headers
        "Content-Type" with value "application/json" (OK)
      has a matching body (OK)

  a request for an order (0s loading, 8ms verification)
     Given order 7 exists
    returns a response which
      has status code 200 (OK)
      includes headers
        "Content-Type" with value "application/json" (OK)
      has a matching body (OK)

  a request for an order that doesn't exist (0s loading, 6ms verification)
     Given order 999 does not exist
    returns a response which
      has status code 404 (OK)
      has a matching body (OK)

.
1 passed in 0.79s

The app and the API agree. Look at order_7_exists, though: it writes an order straight into the API's storage. Keep that in mind.

Step 5: The API team changes how money is sent

Now the API team ships version 2. Money becomes an amount plus a currency, and the request field tip_paise is renamed to tip. These are the lines that change:

class NewOrder(BaseModel):
    items: list[Item]
    tip: int = 0  # paise


def money(paise: int) -> dict:
    return {"amount_paise": paise, "currency": "INR"}

# in place_order:
    return save_order(order_id, food, body.tip)

# in save_order:
    ORDERS[order_id] = {
        "id": order_id,
        "status": status,
        "total": money(food_paise + tip_paise),
        "tip": money(tip_paise),
    }

They also update their own tests, in tests/provider/test_api.py:

"""The orders team's own tests, updated for version 2."""
from fastapi.testclient import TestClient

from orders_api import app as api

client = TestClient(api.app)


def test_place_order_adds_the_tip():
    r = client.post("/orders", json={"items": [{"name": "paneer wrap", "qty": 2}], "tip": 5000})
    assert r.status_code == 201
    assert r.json()["total"] == {"amount_paise": 41000, "currency": "INR"}


def test_missing_order_is_404():
    assert client.get("/orders/999").status_code == 404

$ pytest tests/provider/test_api.py -q
..                                                                       [100%]
2 passed, 1 warning in 0.25s

Green. (The warning is a deprecation notice from Starlette's test client, not from this code.) The app team's tests are green too, because they haven't changed anything. Now the contract:

$ pytest tests/provider/test_verify_contract.py -q
...
Failures:

1) Verifying a pact between foodnow-app and orders-api - a new order with a tip
    1.1) has a matching body
           $ -> Actual map is missing the following keys: tip_paise, total_paise
    {
  "id": 1,
  "status": "placed",
    -  "tip_paise": 5000,
  "total_paise": 41000
    +  "tip": {
    "amount_paise": 0,
    "currency": "INR"
  },
  "total": {
    "amount_paise": 36000,
    "currency": "INR"
  }
    }

There were 1 pact failures
FAILED tests/provider/test_verify_contract.py::test_orders_api_keeps_the_contract
1 failed in 0.86s

Pact caught the change before release, and the diff shows a second problem nobody planned for. Read the numbers: the tip is 0 and the total is ₹360, not ₹410. The app still sends tip_paise. Version 2 only knows tip, and Pydantic ignores fields it doesn't know, so the tip is silently dropped. No error, no 422. The customer is charged less, and the delivery rider loses the tip.

But only one interaction failed. Reading an order passed, although version 2 sends total too. Why?

Step 6: The state handler that hid a failure

Back to order_7_exists from Step 4:

api.ORDERS[7] = {"id": 7, "status": "preparing",
                 "total_paise": 40000, "tip_paise": 4000}

It writes the order straight into storage, in the old shape. The GET endpoint returns whatever is stored, so the verifier got back exactly the version 1 response the test had written in. The test checked the state handler, not the API.

The fix is to create test data through the provider's own code, the same way real data is created. Both versions of the API have a save_order function, so:

def order_7_exists(action, parameters=None):
    if action == "setup":
        api.save_order(7, food_paise=36000, tip_paise=4000, status="preparing")

With the fixed state handler, version 1 still passes. Version 2 now fails both interactions:

2) Verifying a pact between foodnow-app and orders-api Given order 7 exists - a request for an order
    2.1) has a matching body
           $ -> Actual map is missing the following keys: tip_paise, total_paise
    {
  "id": 7,
  "status": "preparing",
    -  "tip_paise": 4000,
  "total_paise": 40000
    +  "tip": {
    "amount_paise": 4000,
    "currency": "INR"
  },
  "total": {
    "amount_paise": 40000,
    "currency": "INR"
  }
    }

There were 2 pact failures

This is the most common way contract tests lie. If a state handler builds the response data by hand, the contract only proves that the handler agrees with the contract.

Step 7: Change the API without breaking the app

The API team can still move to the new money format. They just can't remove the old fields while app versions that read them are still on customers' phones. The usual pattern is expand, then contract: add the new fields next to the old ones, wait until no app version uses the old ones, then remove them.

Their first try puts the old response fields back:

    ORDERS[order_id] = {
        "id": order_id,
        "status": status,
        "total": money(food_paise + tip_paise),
        "tip": money(tip_paise),
        # kept until every app version has moved to "total" and "tip"
        "total_paise": food_paise + tip_paise,
        "tip_paise": tip_paise,
    }

1) Verifying a pact between foodnow-app and orders-api - a new order with a tip
    1.1) has a matching body
           $.tip_paise -> Expected 0 (Integer) to be equal to 5000 (Integer)
           $.total_paise -> Expected 36000 (Integer) to be equal to 41000 (Integer)

There were 1 pact failures

The fields are back, but the tip is still being dropped: the request side was missed. This failure exists only because the consumer test used exact values for the total and tip in Step 3. To check, we changed those two to match.int(41000) and match.int(5000), regenerated the pact and ran the verification again:

1 passed in 0.79s

With type matchers, Pact passed an API that drops every tip, because 0 and 36,000 are whole numbers too. Matchers check the shape. They don't check that the numbers are right. Use exact values where the value comes straight from the request, and add your own functional tests for the rest. Contract tests are not a replacement for those.

We put the exact values back. The second fix accepts the old request field too:

class NewOrder(BaseModel):
    items: list[Item]
    tip: int = 0  # paise
    tip_paise: int | None = None  # old name, still sent by older app versions

# in place_order:
    tip = body.tip if body.tip_paise is None else body.tip_paise
    return save_order(order_id, food, tip)

The whole suite, consumer and provider:

$ pytest tests -v
platform linux -- Python 3.11.17, pytest-9.2.0, pluggy-1.7.0
rootdir: ~/foodnow-contract
configfile: pytest.ini
plugins: pact-python-4.2.0, anyio-4.16.0
collected 6 items

tests/consumer/test_orders_contract.py::test_get_order PASSED            [ 16%]
tests/consumer/test_orders_contract.py::test_missing_order PASSED        [ 33%]
tests/consumer/test_orders_contract.py::test_place_order PASSED          [ 50%]
tests/provider/test_api.py::test_place_order_adds_the_tip PASSED         [ 66%]
tests/provider/test_api.py::test_missing_order_is_404 PASSED             [ 83%]
tests/provider/test_verify_contract.py::test_orders_api_keeps_the_contract PASSED [100%]

========================= 6 passed, 1 warning in 0.87s =========================

Old app versions keep working, and new versions can start reading total and tip. When every app version still in use has moved to them, no contract mentions total_paise any more, and the API team can remove it.

Step 8: The contract checks the app too

The contract works in both directions. Suppose the app team is the one who renames tip_paise to tip, in place_order in the client. Their contract test fails straight away, in their own suite:

E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://localhost:37013/orders
...
E               pact.error.MismatchesError: Mismatches:
E                 (1) Request mismatch: POST /orders
E                   (1) Body mismatch: Expected a Map with keys [items, tip_paise] but received one with keys [items, tip]
1 failed, 2 passed in 0.11s

Pact's mock server answers a request that doesn't match the contract with a 500, then names the mismatch. So the app can't quietly start sending something the contract doesn't describe. If the change is meant, the app team updates the contract, and the API team's verification shows whether the API can handle it.

Sharing the contract between teams

In this post the pact file sits in a folder both halves can read. Real teams need a way to pass it from the app's CI to the API's CI. The usual answer is a Pact Broker, an open-source server that stores pact files and verification results by version. (We didn't run a broker for this post.) With a broker:

• The app's CI publishes the pact after its tests pass.
• The API's CI fetches the pacts for all the app versions in production and verifies against them.
• Before deploying either side, a command called can-i-deploy asks the broker whether this version has been verified against the versions it will talk to. If not, the deploy stops.

PactFlow is a hosted broker with extra features, run by SmartBear.

Where contract tests fit

Test	What it checks	What it misses
Unit tests with mocks	Each side's own logic	Whether the mock matches the real other side
Contract tests	Requests and responses the two sides exchange	Whether the values are right (unless exact), what each side does inside
API functional tests	The API's behaviour: totals, rules, errors	What the app needs from it
End-to-end tests	A full journey with everything running	Little, but they're slow, flaky and late

Contract tests replace most of the end-to-end tests that only exist to check the "wiring" between services. Keep a few end-to-end tests for the journeys that matter most.

Common mistakes

• State handlers that build the response by hand. Step 6 passed a broken API this way. Create test data through the provider's own code or API.
• Matchers everywhere. They check shape, not values. Step 7 shows a contract with only type matchers passing an API that drops every tip.
• Exact values everywhere. The opposite problem: the contract breaks whenever test data changes, people stop trusting it, and they turn it off. Use exact values only where the value follows from the request.
• Putting the whole API in the contract. Include only what the consumer actually uses. A field the app never reads shouldn't stop the API team from changing it.
• Removing old fields as soon as the new ones ship. Older app versions stay on phones for months. Expand first, contract later.
• Treating contract tests as functional tests. They check the agreement, not the business rules.

What's next

In Part 8 of Testing in Practice we look at behaviour-driven development (BDD): writing tests as Given/When/Then scenarios that product owners can read, and running them with pytest-bdd.

ProvaLab doesn't replace Pact, but both halves here are ordinary pytest tests. ProvaLab's pytest plugin sends every result to the ProvaLab test management system, so a failed contract verification shows up in the API team's test run, with the test's history across runs, and you can link the test to the requirement it protects. See it in the two-minute product tour.