An important part of contract testing is being able to run your service with mocked out dependencies. We have realized that there’s a great opportunity in contract testing to make “fresh mocks”. Essentially mocks that closely align with their production counterparts, using contract testing to enforce this alignment. We’d like to have mocks that take in requests and look up the appropriate responses by consulting pact interaction files. Logically, we’d need a way of parsing pact interaction files to help us here. We also realize that this desired behavior is very similar to the pact mock server, but we saw that the pact mock server cannot handle multiple interactions. So our questions are: 1. Do you have any suggestions on how to integrate pact with mocks such that we can reliably call our mocks and get similar behavior to production counterparts? 2. Is it possible to expose the Pact interaction file parser (or get a code pointer to it) a. If not, do you know how often the pact parsing logic changes? (We may consider writing our own fork in Golang to avoid CGO issues) 3. Is it a large change to allow the pact mock server to handle multiple interactions at a time? We looked at the code a bit and am concerned about breaking the plugin api. The team would love to jump on a call to brainstorm with you if someone is willing. Let us know! 🙏 cc @Matt (pactflow.io / pact-js / pact-go)

Yes, this idea makes sense. Probably, the stub server is a better use case here: https://github.com/pact-foundation/pact-stub-server. It takes a pact file and provides a stub service.

I think I need to learn about plugins and plugin interactions a bit. cc @Cameron Trando @Stanislav Vodetskyi from my team. (I didn’t realize they were in this slack 😂 ! 👋) Maybe ya’ll already know about this?

we do use grpc plugin. Afaik plugins just drop custom data under

metadata.plugins

field - like grpc plugin puts proto filedescriptor set plus the proto itself there. How much work would it be to have stub server support plugins?

Thanks for the response @Matt (pactflow.io / pact-js / pact-go)! So if every plugin has its own interaction format, would you recommend that we try to augment the stub server to handle plugins (not sure how big of an undertaking that is) - or try to write a custom parser ourselves that can handle the limited plugins we need (which is just protobuf/grpc to my knowledge) ? Our company is having some big issues where everyone has their own mocks and only a few of them are correct so people only trust running in prod. Moreover, even if we migrate people to contract testing, if provider verification is performed on a service with unrealistic mocks, then the pacts can’t be trusted either (correct me if I’m wrong here). So we’re really looking for solutions on how to use Pact to provide accurate mock behaviors, any suggestions would be greatly appreciated 🙏

which types of provides are you mocking? currently the stub server is built for http interactions. there was a request for a stub server to be built for, or the existing extended to cover grpc. there is already tooling to generate grpc mocks and you could use pact to verify these if required. plugins can have their own interaction format and most likely will do to deal with specific of their particular use case.

Moreover, even if we migrate people to contract testing, if provider verification is performed on a service with unrealistic mocks, then the pacts can’t be trusted either (correct me if I’m wrong here).

why would this be the case? the pacts are generated from real services in a unit test and verified against a real service which is running locally. if you are verifying against a service that is mocked what are you checking? that sounds like a recipe for drift, unless i’m missing something here

which types of provides are you mocking? currently the stub server is built for http interactions. there was a request for a stub server to be built for, or the existing extended to cover grpc.

We’re mocking interactions between grpc clients.

there is already tooling to generate grpc

mocks and you could use pact to verify these if required.

Can you elaborate more on this? Do you mean using something like gomock?

why would this be the case? the pacts are generated from real services in a unit test and verified against a real service which is running locally.

In order to run the pacts against our real service, we have to get our real service running locally, which requires mocking out its dependencies. So if the mocks necessary to have our service run locally aren’t high fidelity, then it’s difficult to trust that the pact was verified on the provider side

if you are verifying against a service that is mocked what are you checking? that sounds like a recipe for drift, unless i’m missing something here

It’s not that the service is mocked, it’s that the service dependencies are mocked in order to get it to run locally

there is already tooling to generate grpc mocks

Here are a few that I’ve bookmarked in the past, or listed via https://github.com/grpc-ecosystem/awesome-grpc#tools • https://github.com/testinggospels/camouflage • https://github.com/tokopedia/gripmock • https://github.com/Fadelis/grpcmock • old - https://github.com/carvalhorr/protoc-gen-mock • https://github.com/bradleyjkemp/grpc-tools • https://learning.postman.com/docs/sending-requests/grpc/using-grpc-mock/ So your provider code, depends on other grpc services, which are being mocked out? and you are lacking confidence in these mocks?

Yes exactly! And I’m looking over the mocks you sent - it seems a lot of them are configured by files in a request-response type matching way. We were looking at file based mocks as well, but we wanted to see if Pact could directly verify these request/responses - i.e. we wouldn’t have to convert a file format from gripmock into a pact interaction file to verify that the mock is accurate. Rather we would be able to start a grpc mock using pact interaction files alone and then verify them directly. Do you know if that’s possible at the moment?

negative I believe, you may need to convert the pact interaction into the format required by one of the mocking tools.

Ah I see - yeah that’s why we were looking at Pact parsing mechanisms. We would have to handle the various pattern matching keywords in the pact interaction files and convert them to some other file format that a grpc mocking library could handle (or vice versa).

I believe, you may need to convert the pact interaction into the format required by one of the mocking tools.

do you know if anyone has tried this before?

I’ve seen Pact -> Postman (https://github.com/ITV/pmpact) albeit for http. and PactFlow provide guidance for any mock format for HTTP -> Pact Interaction which is the other way round https://docs.pactflow.io/docs/bi-directional-contract-testing/contracts/pact#converting-mocks-into-a-pact-compatible-format this pattern used in a feature of wiremock-net and a few other adapters (cypress, msw)

Ah it looks like matchers are ignored in the second link - maybe we can do something like that where we just ignore matching (although it might limit the ability of pact) But I’ll take a look at the first link to see if we can try and implement our own parser to convert between file formats!

we ignore matching rules when converting mocks -> pact files for Bi-directional, as the comparison is performed against an OpenAPI file, which doesn’t utilise matchers (it checks the examples are a subset of the openapi schema) there have been lots of requests to include matchers so people can use the adapters for regular Pact CDCT, with a Pact provider verification, with a few attempting it but giving up/moving on to other things - probably easier just to use Pact on the consumer side natively, rather than indirectly.

Thanks for the links and the guidance so far! Reading through the thread and the links. To kinda take a step back here, pact consumer tests already run a mock server under the hood, it's just configured for a single interaction. Is the stub server based off the same codebase? I'm just wondering on how much work to be to either grab the built-in mock server from consumer tests and expand it to multiple interactions; or add plugin support to stub server.

Pact isn’t a replacement for functional tests, there should be sufficient functional testing of your provider to ensure it can fulfil its purpose.

Yeah, 100%. I believe you folks discussed this above, might've missed some of the posts. Basically we were thinking along the lines of - if in my functional tests I mock out a service with which I already have pact contract tests, can I reuse these pacts to either (a) generate mocks from the pacts, which means they satisfy the contract and we have confidence; or (b) use existing pacts to verify the mocks generated by a 3rd party tool. Just to have confidence that the mocks behave in the same way the real services do, to a reasonable extent.

To kinda take a step back here, pact consumer tests already run a mock server under the hood, it’s just configured for a single interaction. Is the stub server based off the same codebase? I’m just wondering on how much work to be to either grab the built-in mock server from consumer tests and expand it to multiple interactions; or add plugin support to stub server.

Yes, you can see the stub server uses some of the key libraries of the core.