I have been researching contract testing, specific...
# general
s
I have been researching contract testing, specifically Pact and although I have watched the videos and read many articles, there are a couple of things I am not 100% clear on. Hoping someone can clarify the following. Pact simple steps: 1. The consumer writes a test with the expected response from the provider. It’s not actually calling the provider, but instead is using a mock service provided by the Pact library. Does this service know what the actual response from the provider is? Or in this test are we just mocking the desired response ourselves? What are we actually testing here if the response is dictated by the client? 2. We run the test and a pact file is generated. We push this to the pact broker during the CICD run. Does the broker do anything when it receives this file? If it does do something, if the pact file is unchanged or changed does this affect the action? 3. The provider needs to pull down the pact file(s) from the broker during their CICD builds, and verify the contract by writing verification tests, again using the Pact library. A mock provider test provider/server is loaded to run the verification against. Does the Pact library do this? Does it create the mock provider service to run the tests against? Or does it just simulate a consumer call? 4. Once the verification is complete the results are published back to the broker. Is the broker essentially a UI and a data store and doesn’t actually do any testing itself?
👋 1
b
Here goes. Others will amend / correct where my answers are incomplete / incorrect :) 1. The mock provider responses are generated by Pact from the expectations you wrote down in the contract. You can use this to test whether your client can process the responses correctly in case the provider behaves as expected. 2. Not by default, other than storing / versioning the contract, but you could for example set up a webhook that triggers a provider build to verify the newest expectations / newest contract. 3. On the provider side, Pact replays the interactions from the contract against a real provider instance, not against a mock version. That’s the only way you can be sure the actual provider meets expectations written down by the consumer. 4. In consumer-driven contract testing, the broker is the single source of truth for (versions of) contracts and corresponding verification results. In bidirectional contract testing, PactFlow (because this isn’t a feature available in the OSS Pact Broker), the broker also performs the compatibility check between consumer and provider contract.
1
s
Thanks for taking the time to reply. So for 1. Assuming our client code never changes, the client test will always pass, even if the provider code changes. The client will not know about any provider changes, so the client test will always pass. (Unless of course we change our internal client business logic which breaks the test but that would be caught in a unit test, not a pact test). Is the purpose of a client pact test really just to generate a pact file and is not really testing anything? However the provider verify tests will fail if they push a change which will break the contract....
m
1. As Bas said. Think about it as unit test of your API client
So for 1. Assuming our client code never changes, the client test will always pass, even if the provider code changes. The client will not know about any provider changes, so the client test will always pass. (Unless of course we change our internal client business logic which breaks the test but that would be caught in a unit test, not a pact test). Is the purpose of a client pact test really just to generate a pact file and is not really testing anything?
However the provider verify tests will fail if they push a change which will break the contract.... (edited)
Yes, and hopefully preventing a production issue! This is because of the information you provided in step 1. You can also use this information to remove unused fields from the provider.
You might also like to watch JB’s video (the last one) - or indeed the whole thing 😉 - in our recent Pactober conference:

https://www.youtube.com/watch?v=s1LOP4Ak5SA

He talks about the benefits of collaboration contract tests outside of just catching bugs - such as applying better pressure to your software design.
s
Thanks Matt I will check out that video. Originally I was using this example and running it locally. If we look at this test it doesn't actually test anything in the sense that the mock server returns what the code tells it to. so I wasn't sure if there should be some additional business logic that should be tested or if the intent of the consumer spec is simply to create a pact file and ignore the internal business logic of the consumer system: https://github.com/pact-foundation/pact-5-minute-getting-started-guide/blob/main/consumer/consumer.spec.js I will watch the video and dig more into consumer side "tests" which I guess are really generated to be executed on the provider side.
m
Well, that’s not entirely true
The client library unmarshals the result into a domain object (
Order
). That’s important, because it means it knowns how to handle the result. If it used an arbitrary API client then it would be useless
🤔 1