Darren Oc
04/20/2023, 9:33 PMrohan shah
04/20/2023, 9:44 PMBas Dijkstra
04/20/2023, 9:49 PMDarren Oc
04/20/2023, 9:59 PMBas Dijkstra
04/21/2023, 4:46 AMPriyaranjan Mudliar
04/21/2023, 6:26 AMMatt (pactflow.io / pact-js / pact-go)
We might get even closer when there would be some kind of mutation-testing-for-contract-tests kind of thing but that doesn’t exist as far as I knowOh, I like the sound of that!
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
Darren Oc
04/21/2023, 12:00 PMYousaf Nabi (pactflow.io)
Is there a way to ensure that when a consumer's behaviour is modified, the pact gets updated?
We are introducing contract testing, but I'm worried about the case where a developer changes the consumer's real interactions with the provider but forgets to update the contract.Automation can't replace good engineering practises, nor good review practises. It is possible to suffer from consumer drift, whereby the consumer drift drifts in some way from the underlying code, usually this isn't the case as they are an output of your unit test, but it all too easy to encode in extra fields into the contract which aren't relevant for a particular consumer test. By avoiding abstraction layers, and testing closest to the collaborating code, you can make it easier to detect these discrepancies. As Matt said, these contracts can then be reused in other areas of your testing, knowing that they are a good source of truth, although not infallible. Still better than no mocking (you can test in insolation) and better than full mocking (you share some idea of your expectations with others) Depending on your ecosystem and team make-up, teams can share factories/fixtures https://docs.pact.io/consumer#in-dynamic-languages-ensure-the-models-you-use-in-other-tests-could-actually-be-created-from-the-responses-you-expect we used similar approaches in typescript, through shared types and a test data service
Yousaf Nabi (pactflow.io)
but I'm worried about the case where a developer changes the consumer's real interactions with the provider but forgets to update the contract.The contract is generated as part of a unit test run, therefore if the underlying behaviour of the consumer changes, and the outward behaviour of the SUT changes, there would need to be changes in the developers tests for it to pass, or they would need to update the data in the pact expectations, in order to setup their mock provider in the correct state. if they can change the underlying behaviour of their code, without changing the expectations in their test, they probably aren't testing their actual code.
Yousaf Nabi (pactflow.io)
We might get even closer when there would be some kind of mutation-testing-for-contract-tests kind of thing but that doesn’t exist as far as I knowI like the idea of being able to use a pact mock provider, on the consumer side, to act as my http mock and test cases that I wouldn't necessarily want to share with a provider, maybe a flag, write to pact. Or here is my happy request, send some garbage and see how my collaborater code behaves On the provider side, as a pact verification provides a good amount of test coverage of the SUT, it could be nice to create a contract to functionally test your provider (with a use case that could represent how your consumers could use the api) utilising pact to act as the mock consumer. That could serve as documentation for potential consumers, and there could be analysis done to see divergence from real consumers, and the providers own view of a consumer, which may highlight new emergent use cases that the providing team didn't expect (and now want to cater for)
Darren Oc
04/24/2023, 3:53 PMFoo
currently looks like:
1. Functional tests for `Foo`: verify that when my application runs, it takes an input and makes the expected API calls to two downstream services (one we control called Bar
, and another that is a public cloud API)
2. Consumer test for `Foo`: invokes the BarClient
of Foo
directly to generate the contract using provided example requests
3. Provider test for `Bar`: verifies the contract from step 2
2 isn't really a test in this setup, it's more of a scaffold for generating the contract so that it can be used for 3. I think what you're saying is that 1 and 2 should be combined right? I.e. Pact annotations should be added to the existing functional tests, rather than writing new tests purely for the sake of generating the Pact as I've done.