Consumer driven contract testing with Pact

Contract testing ensures that there is a mutual understanding or an agreement in between the consumer and provider.

image courtesy realpython.com

What is contract testing

Simply contract testing is testing integration points of an application. Which means testing APIs, to make sure payloads, headers, requests, URLs etc. are matched into a predefine agreement.

Having contract testing ensures that both consumer and provider are not get surprised out of unexpected payload changes. Hence avoids payload mismatches.

Pact

Pact is an open source contract testing tool. Pactflow is the paid version of it with some added features. In this article we would focus more on free version.

In the context of PACT, contracts can be either consumer driven (Driven by Frontend) or Provider driven (Driven by backend). Consumer does not need to be a FE necessarily. It could be a another backend service as well. Even may be a Kafka like API.

How it works

If you are trying out consumer driven contract testing, Frontend decides how the api contract should look like. In real world this should be a combined decision taken by both backend and frontend engineers together.

In order to use pact in FE,

1. You need to install pact on client side.

npm i @pact-foundation/pact
npm i @pact-foundation/pact-node

2. Then you need to generate schemas via jest and mock services using pact instead of jest.mock. Please check how I did it in following code snippet.

import { Pact, Matchers } from '@pact-foundation/pact';

import { getCountryConfig } from './index';

const { like, regex } = Matchers;

const mockProvider = new Pact({
    consumer: 'Frontend',
    provider: 'ConfigProvider',
    dir: 'pacts/',
});

const country = 'DE';

describe('common services', () => {
    beforeAll(() => mockProvider.setup());
    afterEach(() => mockProvider.verify());
    afterAll(() => mockProvider.finalize());

    it('should use call to GET configs', async () => {
        const expectedResponse = {
            VE: {
                market: 'DE',
                currentWeek: '26',
                productionStartDay: 'FRIDAY',
                categories: ['test1', 'test2'],
            },
        };
        await mockProvider.addInteraction({
            state: 'Configuration DE',
            uponReceiving: 'a request to get Configuration DE',
            withRequest: {
                method: 'GET',
                path: '/config/DE',
            },
            willRespondWith: {
                status: 200,
                headers: {
                    'Content-Type': regex({
                        generate: 'application/json; charset=utf-8',
                        matcher: 'application/json;?.*',
                    }),
                },
                body: like(expectedResponse),
            },
        });

        const configs = await getCountryConfig(country, true);

        expect(configs).toMatchObject(expectedResponse);
    });
});

3. Run tests. Once you run the above test you would noticed a pact contract is being generated in the given directory. In my case contract was generated into a folder named “pact” which looks like bellow.

{
  "consumer": {
    "name": "Frontend"
  },
  "provider": {
    "name": "ConfigProvider"
  },
  "interactions": [
    {
      "description": "a request to get Configuration DE",
      "providerState": "Configuration DACH",
      "request": {
        "method": "GET",
        "path": "/config/DE"
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json; charset=utf-8"
        },
        "body": {
          "VE": {
            "market": "DE",
            "currentWeek": "26",
            "productionStartDay": "FRIDAY",
            "clearDownDay": "FRIDAY",
            "categories": [
              "test1",
              "test2"
            ]
          }
        },
        "matchingRules": {
          "$.headers.Content-Type": {
            "match": "regex",
            "regex": "application\\/json;?.*"
          },
          "$.body": {
            "match": "type"
          }
        }
      }
    }
  ],
  "metadata": {
    "pactSpecification": {
      "version": "2.0.0"
    }
  }
}

4. So now you have the generated contracts. One thing you could do with this contracts is that you could use the generated pact file to run a pact http stub server. Could be useful when mocking data in E2E testing.

Now we are going to push this contract in to a intermediatory layer called pact brocker. Pact brocker is a intemediatry service who is responsible for,

1. Storing pact contracts
2. Provides auto generated documentation
3. Provides network diagrams
4. HAL browser embedded
5. HAL documentation.

network diagrams

auto generated documentation

In order to publish contract to pact broker I used following node script.

const opts = {
    pactFilesOrDirs: [FolderPath],
    pactBroker: pactBrokerUrl,
    tags: ['stg', 'test'],
    consumerVersion: gitHash,
};
fs.existsSync(FolderPath) &&
    pact
        .publishPacts(opts)
        .then(() => {
            console.log('Pact contract publishing complete!');
            console.log('');
            console.log(`Head over to ${pactBrokerUrl} and login with`);
            console.log('to see your published contracts.');
        })
        .catch(e => {
            console.log('Pact contract publishing failed: ', e);
        });

5. Once published from consumer side, backend service needs to re-run the tests against the new contract. If backend introduced new changes to the api, it would required to rerun tests against existing contracts.

6. We used following code snippet in our GitHub action pipelines in order to check if all contract are green and services are good to be deployed. can-i-deploy is a plugin provided by pact module which can be used to validate the status of contract tests.

- name: Verify valid contract
              working-directory: ./
              shell: bash
              run: ./node_modules/.bin/pact-broker can-i-deploy --pacticipant 'Frontend' --latest --broker-base-url https://broker.testdomain.io/

API responses get change all the time while product is evolving over the time. pact ensures that changes to the APIs are validated by Frontend before they are pushed into live environments.

Observations

1. Having a quality gate to validate the API contracts ensures smooth api workflow between client and backend.

2. Can match types/type check of the contract properties

3. Automated documentation. (Open Api needs to update manually)

4. In case of some services it is highly unlikely that contracts would change over time.

5. Adds lot of boilerplate code in mocking jest apis via PACT

6. Be causes not to address provider internal logics with contract testing. Useful link which distinct contract testing vs functional testing

7. May have duplicate work with cypress e2e testing

8. Pact broker is free and open-source where as pactflow is limited/paid.

Let me know your thoughts on pact and contract testing in general. Would you use it in your project ? what sort of issues do you foresee in using this too ? Until we meet again, Cheers.

Reference

https://docs.pact.io/getting_started/how_pact_works