Hi, I am a newbie with contract testing and have a...
# general
k
Hi, I am a newbie with contract testing and have a question to ask. A book I read said that "Contract testing can be seen as a programmatic solution to a communication problem", and presents it as a way to detect accidental or uncoordinated changes in the consumer api and provider api. A picture of the model describing the relationship and process of contract testing is attached. ( I blocked out some text bec I do not wish to copy the entire section verbatim ) That model seems to imply that the contract file is the key element that holds the process in place. In fact, it would not matter who/what created the contract file. This is very different from the advice in https://docs.pact.io/faq#can-i-generate-my-pact-file-from-something-like-swagger where the pact file is not considered key but the process of generating it in the consumer side and using it on the provider side is considered more kosher. It would seem that both scenarios are valid depending on what the engineer values since the operational aspects of Pact does not know which usage is being intended anyway.
👋 2
m
Which book is that, btw?
Are you referring to this line in the shared link?
The reason the pact file exists is to ensure the tests in both projects are kept in sync - it is not an end in itself.
The contract itself isn’t the end - the safety you get from the process is the ends game. Of course, the artifact is critically important to this process.
In fact, it would not matter who/what created the contract file.
In a way, yes, but an OAS (if this is what you’re interested in persuing) is not a contract in the way Pact would define it. See the link below (schemasarenotcontracts) for a detailed exploration of that topic
k
The book is "Testing Web APIs" by Mark Winteringham, 2022.
🙌 1
I am both referring to the line about the pact file not being an end to itself and the opinion that writing the Pact file directly or converting from Swagger is not a good idea.
The diagram in the book seems to emphasis the key solution is that the same Pact file is used by 2 sides, to ensure correctness/agreement.
b
As you mentioned, they're both valid, so it depends how much reasoning power (and therefore safety) you desire. Tests that execute your code to prove it does the right thing are more powerful than schemas that only validate that the output isn't invalid.
💯 1
m
Well put Andras
🙏 1
I am both referring to the line about the pact file not being an end to itself and the opinion that writing the Pact file directly or converting from Swagger is not a good idea.
If you can guarantee (to some level of accuracy) that both consumers are providers do indeed match the contract (however it is generated), then you are certainly in a good space. The problem is, OAS and other hand cranked contracts, are usually too abstract to be meaningful in reasoning about how it is used (consumer part) and if the provider supports it (provider part). Of course, Pact and consumer-driven contracts give you many other benefits - eg knowing which parts of the interface the consumer actually uses, ability to evolve over time, specification by example, network diagrams of relationships etc.
☝️ 1
b
I think there's an OAS approach that introspects the code to generate the schema—that's also better reasoning power than, say, a YAML file that can drift from the code, or annotated comments (in JVM land) that can also drift from the code.
1
m
Yes, this is true. Albeit (theoretically) you could use some scaffolding tool to produce an API with routes and models, and then use the OAS generation tool to generate a valid OAS, but have not actually written the implementation. You can imagine other variations on this also
💯 1
k
I just learned that PactFlow supports bidirectional contract testing where the provider pact is generated by hand or code but not from the consumer side. It introduces the idea of consumer pact and provider pact files. See https://pactflow.io/bi-directional-contract-testing/ That seems to support the idea that the pact file is the central element. Of course, I think best to solicit ideas and comments for those who have more experience here. p*actflow.io* Pactflow Contract Testing Platform Bi-directional Contract Testing enables new modes of contract capture and compliance, a broader set of use cases, and a simplified developer experience exclusively for PactFlow customers.
👍 1
b
When we mention OAS, this bi-directional contract testing is what we're talking about. It's less powerful & reliable than a consumer-driven contract with provider verification.
Also, it's only available in Pactflow. That might not be a problem for you, but it's an important point for consideration (in costing, OSS contribution, etc).
k
U can tell I'm still new with contract testing and Pact if I have to ask what does OAS stand for? ChatGPT tried to help but gave very funny answers.
Found it - OAS is Open Api Specification.
b
That's right 👍 OAS is kind of the newer name for Swagger, which started life as a code-and-documentation-driven schema-and-documentation generator, and has since grown into a bigger suite of automated collaboration and testing tools.