Hi folks, I'm quite interested in exploring and using a tool like Pact/Pactflow, and have done a bit of reading of things to try and find my way. I've got a situation where my team controls both the provider and the consumer of an application, and I'm looking to find a way to make sure that deviations in one are properly accounted for in the other. I'm sure I'm preaching to the choir when I say that obviously a unit test in Jest with a mocked adapter and a unit test in the API of a given view don't cover the actual contract between them (consumer could change to sending a parameter the provider is unaware of, provider could change the response that the consumer was expecting, etc) With that in mind, my ideal solution for things would be: • Provider contract is generated via Swagger (I happen to be working with Django so using https://drf-spectacular.readthedocs.io/en/latest/ to document all of the endpoints) ◦ I've got this working and have uploaded this to Pactflow using the CLI, everything good here! • Consumer contract is generated by tapping into the mock adapter in Jest ◦ Every mocked request should be stored and then merged into a contract that represents every API call that the Jest suite performed ◦ This is where my question begins It seems like from my understanding of reading the docs that the consumer side pattern I'm describing has been deemed an 'anti-pattern' by the pact team. I'm curious to discuss why as it seems to me as an ideal path. In this situation a passing Jest test would have performed all of the API requests that the application/component requires from the provider. That seems to me to be exactly what I'd want to have, as that would describe the total contract that the consumer needs to function. If I understand what is recommended by https://github.com/pact-foundation/jest-pact it seems to be saying that I should have a Jest test that manually defines an endpoint, and then explicitly calls a function in an API wrapper (

client.getHealth()

in their example there) to hit that defined endpoint. However, in my application we do not have an API class that holds the logic for calls. Instead we are putting that directly in the component:

const FooComponent = () => {
  // useAPI is effectively a hook wrapping axios.get in this case
  const { isLoading, response, error } = useAPI("/bar/foo/");
  ....

  if (isLoading) {
    ...
  } else if (error) {
    ...
  }
  return <p>{response.data}</p>
}

With a component structure like that how would you recommend generating a contract for the

/bar/foo/

endpoint? I don't want to write a standalone pact test as shown in the jest-pact documentation, as that would not call any production code:

pactWith(...), provider => {
   ...
   provider.addInteraction({
      ...
      withRequest: {
         method: "GET",
         path: "/bar/foo/"
   })

   axios.get("/bar/foo/")
   // assert that the request returned 200
}

And obviously because the above doesn't call the actual FooComponent at all it doesn't represent the contract and won't fail if the FooComponent changes. It seems to me a better solution would be something like this:

const mock = new MockAdapter(axios); // <https://www.npmjs.com/package/axios-mock-adapter>
describe("Test the FooComponent", () => {
    it("handles loading", async () => {
         mock.onGet("/bar/foo").reply(200, {foo: "Is good"});
         render(<FooComponent />)
         // wait for loading message to go away
         // assert "Is good" appears
    })

    it("handles error", async () => {
         mock.onGet("/bar/foo").reply(500, {detail: "Oops"});
         render(<FooComponent />)
         // wait for loading message to go away
         // assert "Oops" appears in document
    })
});

This is a functional test that tests the component and describes the API:

/bar/foo/

shall exist and when GET requested shall return a 200 response with a

foo

key that holds a string. If it can't it will return an error with a detail key. No other methods are handled on this endpoint. It would be nice if running a test like that could build a contract off of the mocked requests defined in the test. Is something like that possible? Thanks for your advice!

Hello! The reason that it’s an anti-pattern, is that with pure “Pact” testing (where Pact is used to verify a contract), generating the contract from real requests will potentially create a lot of duplications (e.g. multiple similar requests with slightly different data needs but nothing structurally different, that don’t give you any more reasoning power about the needs of the consumer) Also, because you aren’t setting up the scenarios using “states”, it will be difficult for the provider to know how to prepare itself for verificatations. If you use PactFlow’s bi-directional contract testing approach (which I think you’re alluding to with a “provider contract” generated as a swagger file (OAS), then this problem is not a problem - it’s in fact how it is designed to work.

Hi Matt! Yeah, I definitely see how there would be 'duplicate' requests, even as you can see in my example from there would be two requests to

/foo/bar/

- but that seems like something that could be resolved through an additive deduplication before the contract is generated. Just as a swagger file describes the provider's options (

/foo/bar

returns a 200 with a JSON object with a 'foo' key, or returns a 500 with a detail key) presumably it should be possible to reduce the two mocked calls in the Jest test to describe the multiple requirements of the single API endpoint consumed? I don't fully understand the comment about states there either, the swagger document doesn't provide explicit states in it so how would it match up anyways? I also don't follow what you're saying about how using the swagger file resolves the problem? As far as the anti-pattern, I've not seen that specific verbage mentioned anywhere, but I've seen multiple sites talking about how using pact for functional testing is a no-no: • https://docs.pact.io/consumer#use-pact-for-contract-testing-not-functional-testing-of-the-provider As one example

For my answers, not that BDCT = bi-directional contract testing = compare a pact file against an OAS (see https://docs.pactflow.io/docs/bi-directional-contract-testing/#trade-offs)

Yeah, I definitely see how there would be ‘duplicate’ requests, even as you can see in my example from there would be two requests to

/foo/bar/

- but that seems like something that could be resolved through an additive deduplication before the contract is generated

that might be straightforward for basic examples, but quickly gets hard - e.g. there might be different shaped payloads, or payloads with different keys that are enums etc., that are worth testing. It would be hard to de-duplicate these without being lossy. But yes, it’s plausible.

I don’t fully understand the comment about states there either, the swagger document doesn’t provide explicit states in it so how would it match up anyways?

I also don’t follow what you’re saying about how using the swagger file resolves the problem?

Correct. But we don’t need states when comparing consumer requests against the spec, so the problem about “too many” goes away, as it doesn’t create an additional burden on the provider API team. It’s only a problem if the validation happens in a Pact style (request-response) model, because the actual API call is made to the provider that needs to be able to respond accordingly - this is where the maintenance and alignment problem creeps in. See also this cautionary tale 👉 https://pactflow.io/blog/a-disastrous-tale-of-ui-testing-with-pact/

I’ve seen things talking about using cypress to generate a consumer’s contract...how is that any different from generating it via jest tests? both are functional things - cypress is driving a browser to do actions, jest is simulating a browser

You’re right - it’s not, but it’s (and the MSW, Wiremock and similarly others) not designed for use with Pact. It’s designed for use with the BDCT capability.

From the cypress side it seems like you’re assuming that it’s using network stubs: https://pactflow.io/blog/use-cypress-in-contract-testing/ - however for our product we are using cypress to test the whole stack (End to End testing). Therefore we’re not mocking out the cypress requests at all, and instead are performing the cypress testing on a full live server.

If you’re doing that, then do you need contracts? You’re testing the real thing anyway? All you’re really doing there is recording what you know (this is a simplistic view of the situation, but I want to make sure you understand the value). The point of Pact is to avoid e2e tests where you can, replacing them with faster, lightweight and reliable tests that can be used in mulitple contexts - including in cypress tests.

Regarding the cypress side, yeah, the reason I'm interested in Pact is I would like to test for broken contracts before it gets to the Cypress, which is why I was interested originally in trying to have the already existing functional Jest tests create the Pact contract. Because, you're absolutely right - Cypress is running against a live system it is validating contracts (and everything else an e2e framework would do). I was hoping that by having the Jest tests generate a contract, and having the swagger file be a contract that I could use the two of those through the CI tooling to block any PRs that break the contract as defined in the PR being tested. We aren't running Cypress on PRs (takes too long, etc), so this would allow developers to catch breaking contracts before feature merge. Sounds like a good idea to me! Unfortunately, yeah, it seems that because we're not using an API service class and our Cypress suite is doing true e2e testing, I'm not seeing a way to get a consumer pact generated in a sensible way. Defnitely still willing to be convinced otherwise, as I very much see the value that is provided by Pact here, I just don't see how my team can use it without restructuring every client side API call to a service.

I was hoping that by having the Jest tests generate a contract, and having the swagger file be a contract

So yes, you can do that, just not with Pact (the BDCT feature above would support it). You could by all means try if you think you can workaround those problems/limitations discussed - they aren’t insurmountable, but they are definitely something you’d need to handle