Hello! I've got a general question regarding provi...
# general
v
Hello! I've got a general question regarding provider verification step. What happens when you run it? Is a test request gets generated from the pulled pact and goes through the providers handlers? We have argues in the team regarding whether it is the same when you run provider verification tests and when you generate provider's spec, push it to pactflow and let it to compare the consumer's and provider's pacts.
b
Sounds like there might be some confusion between consumer-driven and bi-directional?
Fwiw, I have only done consumer-driven. In that flow, the consumer tests generate a contract, it goes to the broker, which triggers provider verification. There's no provider pact.
v
right...some people of my team think it is more easy to just generate a pact for provider and let broker to compare pacts. But I am not quite sure this will be the same as if we run provider verification. So this is what I am asking - if during provider verification we do more than just pulled consumer pact and provider specification comparison
b
Yes, there's a significant difference. The consumer-driven flow gives more reasoning power. There's a nice article about it here: https://www.google.com/amp/s/pactflow.io/blog/schemas-are-not-contracts/amp/
Basically, schemas can only tell you that the consumer and provider might not be incompatible.
v
thank you! will read this!
b
Np, I hope you can win your people over 🙏
v
me too :)))
t
it is more easy to just generate a pact for provider and let broker to compare pacts.
Definitely not.
The point of pact is to be sure that two services can communicate with each other. With a spec-driven approach, the best you can get is “they’re not definitely incompatible”
Sometimes people who have graphql or grpc specs as part of their development flow say they don’t need contract testing. I think this is a bit like saying “all our endpoints use json, so we don’t need contract testing”. Just because (for example) grpc ensures that the payload is receivable by consumers with older versions of the spec doesn’t mean that it will have the intended meaning.
As far as I know, there is no “provider pact” written by any pact framework at the moment. Pactflow’s BDC approach is based on a swagger spec, which isn’t a Pact.
v
right
just spec
t
When a pact file is written, you are guaranteeing that you can send that request, and promising that you will understand that response (for each req/resp pair in the pact)
this is important, because it’s not a specification that describes the design. It’s a list of examples that definitely work.
v
and how exactly verification on provider side is made? is a request generated from the pact and "send" through the provider's handler?
t
Yes, that’s right. You can think of it as replaying the requests, and seeing if the responses match what the consumer will understand (that’s not quite what happens, but the subtleties don’t actually matter in practice).
The pact test which writes the file says: “I definitely send this request, and I have checked that I can understand this response” When you do provider verification you are saying: “I am checking that I can understand this request, and I definitely send that response”
v
got this!
Thank you Timothy! you gave me needed confidence
t
You’re welcome! Let us know if there’s anything else we can help with.
You’re welcome to invite the team in here and we can answer their questions directly, too.
💯 1
v
I will! thanks!
@Artem Zhuchenko fyi
a
Hi Team! thank you so much for your detailed explanation! happy to be talking to someone who actually make this work. So am i right that you’re saying that BCDC should only be used as a compromise when let’s say the “provider team” doesn’t want to put too much effort? 😉 Thanks for sharing the post - however i was under the impression that the idea of it is to prompt people to use the commercial pactflow solution which is supposed to help with most of listed concerns (which we already do!) - in conclusion it suggests to learn how you can make schemas work with contract testing Speaking of the specifics: my original idea was to implement it in a way described in pactflow tutorial or workshop (provider schema created -> provider schema validated against the provider implementation in a “blackbox” style -> validation results and schema uploaded to broker -> broker validates consumer pact against provider schema). With such an approach it doesn’t seem to me that the concerns listed in the post are real, lemme try to go through them: 1 - not sure we faced any issues with that; if we use .yml schema the broker seems to understand it in a way we want. or am i misunderstanding what that means? 2 - totally valid! but is this fully solved with CDCT? for instance could we test with 5xx error from provider when it’s not caused by the inputs from consumer? 3 - looks like supposed to be solved by “provider vefification step” from the workshop 4 - same (“provider vefification step” ) 5 - same (“provider vefification step” ) 7 and 8 apparently are supposed to handled by pactflow 9 - not sure is this could be a problem with OAS yaml schema ? In general, i understand what you mean when you say that the interactions generated by consumer and “replayed” against prodiver are more “real”, i have same “feeling”; but the tradeoff seems to be fair considering much easier implementation especially with multiple consumers Is there something i am missing? would really appreciate any pointers 😉 thank you guys and have a great day!
😉 1
b
Bi-directional is also a helpful stepping stone when adding testing to a large, established ecosystem. If you're starting from scratch, there's pretty much no compelling reason to avoid consumer-driven 😇
t
I’m a bit unclear on what points you’re replying to with the numbers
a
ok seems everyone here is leaning to consumer driven approach then we shall try it, thanks again guys for your help! @Timothy Jones oh sorry i was replying to the problems listed and numbered in the post Boris shared above in the thread 😅
party parrot 2