hi, we're doing bi-directional testing and I have ...
# pactflow
m
hi, we're doing bi-directional testing and I have a query. We have a service doing some backwards compatibility for a legacy service, where an incorrect field value type would still return a 200, just with an error message. The swagger file the field is defined with a specific value type (ie date-time instead of plain string). The test currently involves sending a request body where one field has a string instead of the expected date-time value, and expects a 200 with an error response (one-of responses defined for the 200 code). Pact fails it in itself due to the incorrect field value (expected date-time, got string). Am I correct in assuming that this is done to encourage appropriate service behaviours (i.e. this should really return a 400), so if any value is incorrect in the request body, it'll automatically assume that a 200 should not be returned?
m
Yes I think you might be right. We do have the ability for negative test scenarios, but I believe they rely on the HTTP 4xx status code range to function. That is, you can knowingly give a bad request, in order to test a 4xx.
I’ve definitely worked in places where the architects think they know better and think it appropriate to send a “protocol” level status code (“yep, we got the message - 200 OK!“) and an “application” level status code (e.g. a JSON body with properties indicating the problem). I would like to have an argument with these people, but also I’m already running out of hair as it is
😆 1
sarcasm aside, given that the verification doesn’t actually rely on you sending a bad code, could your consumer test just send a valid value knowing that there’s a polymorphic response?
m
oh yeah I have gone through this info with folks already, imo the OAS file is too strict to indicate the date-time field in this case because the service isn't flat out rejecting it with a 400 😄 I gave them options to either adjust swagger for now, or do the regular value with a error response as you mentioned above, with the idea that they should return a 400 once the backwards compatibility with the legacy app is no longer needed
🙌 1
luckily ( I hope ) no one would argue on the appropriateness of a 400 response here on the dev side
thanks Matt ❤️
m
No worries!
m
actually, just to play the devil's advocate here a bit, given real-life situations sometimes suck and above is just what we're dealing with, would it be possible to have something like a metadata tag in the contract to define a 200 expected response as an error response?
again, not to say that I agree with this approach and would first of all urge the devs to adjust this first where possible, but in those cases that this is not possible, it would be good to have a valid test run with an invalid request + 200 response error json