Hi, curious question. We have created a consumer test to test how we handle error 500 by the provider. However, a http 500 is not defined in the openapi spec (see pic). Thinking about it, I guess that makes sense? a 500 is usually an unexpected exception. So defining a http 500 response in your openapi implies that you expect the unexpected? We are testing this using bi-directional (hence the #CLS16AVEE channel), but I think the problem persists in consumer-driven too. Given we give the provider a contract which expects a 500 response, how would the provider even simulate an exception for that test? Are there any thoughts on covering http 500 cases?

blobwave > Are there any thoughts on covering http 500 cases? Does your consumer code do anything different if it gets a 5xx error, over a 4xx error. If not, then I don’t think its worth covering that granular of a case on the consumer side and encoding that into a pact file.

"Does your consumer code do anything different if it gets a 5xx error, over a 4xx error." In this case no, we are just looking to add a few more testcases to our test suite to see how it grows. But what if it did? 😄 Based on your response, I think in this case we would have to open discussions with the provider team. I am sure they would like to know how internal server errors get exposed to consumers too.

In this case no, we are just looking to add a few more testcases to our test suite to see how it grows

Yeah, Pact provides a powerful mock server, that would allow you to test additional cases, which you may not want to send over to provider, and it would remove the need to use another mocking tool, for these corner cases. There has been some discussing in the past about having pending pacts, or allow pacts not to be serialised for particular tests. In theory it would be doable but would involve some work in each client library as its a composition of calls from the FFI, or we could compose them in the FFI, and then just have a way of exposing that to the client libs.

But what if it did? 😄 Based on your response, I think in this case we would have to open discussions with the provider team. I am sure they would like to know how internal server errors get exposed to consumers too.

Yep, I think I would certainly be having this conversation with the provider team, and maybe a pairing session where you can do some exploratory poking of each others systems/apis. If often does highlight things you may not have considered on both sides.

However, a http 500 is not defined in the openapi spec (see pic)

You should still define this in your OAS as an expected-unexpected response type. The point of an OAS is to try to be explicit about these things, to the degree possible with OAS