Hi folks 😄, Do you have a strategy that make Pact Contract Testing for a provider easy to maintain when a provider has multiple consumers? For a consumer, pact is as easy as yet another mock server. However, for a provider that has multiple different consumers. I find it difficult to write a test that can handle different variation of pact files from different consumers, because a provider has to actually start their test server and mock behavior under the hood to make it work each pact file. For example, A provider have a GET /products/:id endpoint Consumer A's Pact file specify that GET /products/420 will returns 404 and GET /products/69 will returns a product. Now the provider implement a mock that perform such behavior. Consumer B's Pact send their pact files with GET /products/420 will return a product for the same endpoint. Now a maintainer of the provider has to read this new pact file manually to understand why it fail and update their mocked behavior to match these behaviors or reach back to the provider to have them change the product ID, because the product ID is conflicting with another consumer. Thank you 🙏

I would have expected that the provider states should handle both cases fine. That is, in the first case, there should be some hook (probably two actually) that sets up the provider in a state that

420

is unknown and

69

is know; and then in the second case, there should be another that sets up the provider in a state where

420

is unknown. It's generally a good idea to coordinate between consumers and providers which states are handled for consistency. In this case, you would have two parametrised states: 1. given

a product exists

with an

id

parameter (and maybe additional parameters) 2. given

a product does not exist

with an

id

parameter.

I found another tool. https://github.com/pactflow/swagger-mock-validator This is not ideal, but probably could fill early adoption. It can compare a pact file against a swagger schema. So, no need to set up a mocked behavior.

We sometimes use provider states, but we found it's easier for provider to specify a set of known defaults for all consumers to use in their tests, similar to stripe test cards: https://docs.stripe.com/testing#cards this way the provider needs to support only a limited set of mock values and it expects all consumers to use those values in their tests. It might be a higher initial investment to build a higher fidelity mock/standalone mode for the provider, but long term might we feel it'll be easier to maintain. It doesn't stop from using provider states to make the tests look more descriptive, but provider won't be consuming them. but I wonder what pact experts actually think about this approach 😄

I think that's fundamentally the same intent, but with shared magic values instead of descriptions. I also don't think it saves anything you'd need to document or talk about 🙂 If you're using Pact anyway, I'd probably stick to using states, to keep that kind of documentation and mapping in a minimum number of places.

I agree that it's pretty similar, but there are a couple of differences.With magic values you import them from via code, but I guess you can also import known states from the provider repository too, so that's pretty similar still. If you're using parametrized states though, provider implementation becomes a bit more difficult, depending on how complex their mocks are.

I found another tool. https://github.com/pactflow/swagger-mock-validator

This is not ideal, but probably could fill early adoption.

It can compare a pact file against a swagger schema. So, no need to set up a mocked behavior.

so this tool is part of a wider “schema validation” approach to contract testing. It’s incorporated into our (PactFlow’s) bi-directional contract testing feature. It’s worth noting the trade-offs of this approach, although it is definitely aimed at reducing the barrier to entry (in particular when you already have good existing test coverage - see the primary use cases) Please also note that we are re-writing this tool for performance, code readability etc. but it’s stable enough to use. You would still need to integrate it into your workflow, unless you use PactFlow in which case this is all run in our cloud environment.

Yarr, with state params, you've got more options for how to use/implement things, which gives you more rope/foot-guns, for sure.

We sometimes use provider states, but we found it’s easier for provider to specify a set of known defaults for all consumers to use in their tests, similar to stripe test cards: https://docs.stripe.com/testing#cards

as for your comment Stan, I often advocate for a similar approach to customers with lots of consumers who have identified it as problematic for them. It is one of the areas of Pact that has some usability/scale challenges, because it does require higher collaboration.

Not sure about consumer-namespaced states, I don’t think I’ve heard of that before . . .

Actually the creator of provider states (Beth) advocated for this, but I don’t think pretty much anybody has followed that advice 😛 . I’ll see if I can find it

Beth advocated for this, but I don’t think pretty much anybody has followed that advice

I feel like I've heard that a few times joyfacepalm

hahaha