General question here after feedback from my dev t...
# general
j
General question here after feedback from my dev team. We have a proof of concept implemented with Pact right now in our CI pipeline across 2 teams. One complaint I am hearing is that since Pact does not test data changes, only changes to the endpoint structure, we aren't really catching anything important. We usually do not encounter changes to our endpoint structure. What are peoples' thoughts on this critique?
šŸ‘‹ 1
y
What do you mean by data change? do you have an example scenario that isn't passing? If you remove a field from a provider, that a consumer relies on, or change the fields type, it would fail verification. depending on your type of matchers setup on the response. if the data change, isn't due to structure, but due to actual identifiers (you require a book with a specific id to exist) you would use provider states for the provider to setup the required state prior to running the test in a decoupled fashion. Otherwise the alternative in an integrated environment is setting up external state which could be affected outside of your test (which is bad)
It isn't a replacement for other types of testing, but rather compliments it and provides quicker feedback and enhanced visibility, some teams may decide they want to use alternative methods, they may have really good channels of communications, endpoints and payload structures don't change often, or systems aren't that critical. Systems that don't change now, will invariably change in the future, either intentionally or accidentally. Having safety blankets helps šŸ™‚ people in teams move on, people come in with new ideas.
j
Thanks for responding, @Yousaf Nabi (pactflow.io)! Things we see change are changes in business logic, like...someone changes what types of data an endpoint can accept...but didn't realize that impacts another team. For example, changes like having data in a certain state, like 'finalized,' that causes issues downstream.
we use cypress to test our individual services, but need to mock out the downstream services in the CI pipeline. We still then have this issue where business logic changes are causing uncaught bugs.
y
where are you testing with Pact? through Cypress, or at a unit test level directly with Pact? You can restrict values by enums, with a regex matcher. if the data needs to be different for a particular state
finalized
vs
pending
then you can setup tests for that, but only if they are because of specific client logic, otherwise you can just have one test. You don't want to be looking to functionally cover all cases for the provider.
j
We are testing Pact at the unit test level directly with Pact. Is testing client logic like this recommended for Pact?
Do folks commonly do this sort of thing with Pact?
y
j
@Lucas Dicesaro is the dev from my team. I invited him to the thread šŸ™‚
l
Thanks Jane, hey Yousaf! I'll read the thread.
I have a question, Yousaf. Pact will validate only the communication between the services. (I mean endpoints). So if we want to emulate an E2E we need work together with the unit tests. Pact for communication and unit tests for functionality, right?
Technical questions: • We are using the
webmock
gem (see Comparisons with other tools) in our unit tests (ruby specs). If we want a client (consumer side) to be validated with Pact we should make a copy of that client to point it to the Pact mock service. What is the best practice here? Should we make a copy, or should we replace
webmock
for that client? • When we define specs on the Consumer side, and we want to generate a pact file, we need to define the request body/response body/headers and finally, execute the code. That's fine. The thing is that we need to code trivial expectations (according to the documentation) too. I understand the expectation on functional tests. But in these cases, there is no a complex logic in the middle, we are testing the Pact tool itself. What is the purpose to create expectations of something we defined a few lines above? I await your responses. Thanks
j
@Yousaf Nabi (pactflow.io) does this seem like something you have knowledge about?
l
Should we post it in a new thread maybe? šŸ¤”
j
i think so šŸ™‚