Getting started with bi-directional contract testing....

Let us know how you go!

What does this error message mean?

The self verification results by the provider signified a failure, this would normally be the error code from your providers test run. We are looking for evidence as part of the test run, that the provider can implement the specification. The onus is on the provider testing team in BDCT, where the Pact Verifier would do the heavy lifting for us here, in CDCT. As such the provider self verification are free text so we currently can provide the user granular detail, but as we see common tools being used, we may be able to build parsers to extract suitable information a particular tools output. ( junit format might be something nice and standardised ) Ultimately it would be nice to get more insight so we could say providers tests covered x% of the provider spec, so you know what you can safely consume

It's not clear to me whether this message is saying that the provider code has the wrong version of something pact related, or whether it means the contract was violated. It might be better to say something like "Your ${contractType} contract is violated by the provider"

The message isn't very clear when you are verifying openapi spec. It's not really the codebase that's involved

in this instance you are self-verifying a provider codebase against the provider's specification (currently only OAS is supported) to ensure that implementation of the provider matches that documented in the specification. the provider contract we are speaking of consists of the spec + verification results, so it possibly be more correct to say, "Your provider specification is not compatible with your provider codebase" and the user to be guided to view the results and determine what action to take. I would agree that the messaging could be improved and more directive as to the next steps, the current docs around the provider publication are here https://docs.pactflow.io/docs/bi-directional-contract-testing/provider#step-4-publish-your-provider-contract-and-verification-results https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/oas#interpreting-verification-result-failures https://docs.pactflow.io/docs/user-interface/bi-directional#provider-contract-tab Thanks for the feedback Tim/Lewis

Thanks!!

There is a some detail on the cross compatibility failures between the consumer + provider for BDCT but not enough on the provider contract (spec plus evidence) failures. I think a genuine quick win improvement is getting the UI screenshots up there on the docs, along with highlights to describe the valid/invalid views and next steps (along with provided next steps in the error message too) - danke

Specifically the word codebase is confusing, I think. To me that reads like “the problem is the version of something that your code is based on, not your code”

Do you have a suggestion then Tim?

I'd prefer something like "Your ${contractType} contract is violated by the provider"