https://pact.io logo
Join Slack
Powered by
Hello all I am struggling to think of an example ...
# general
n
Hello all I am struggling to think of an example of an “unhappy path” for an asynchronous interaction. Let me clarify: On a synchronous interaction I would consider an unhappy path to be something like when the consumer sends a request to the provider and gets a 400 response back. The consumer test then asserts that the consumer handled the 400 response correctly (e.g. correct exception is thrown). For an asynchronous interaction I would consider an unhappy path to be something like when the provider sends a message that is syntactically incorrect (e.g. with missing mandatory fields) or with wrong data, causing the consumer to reject it. The problem is I can’t force the provider to send a message that is syntactically wrong (as the server code enforces the syntax), that leaves me with trying to send a message with wrong data. In this case should I build a scenario for each wrong data combination that can be sent (this is functional testing), or should I just pick one or two examples of wrong data and build tests for it?
y
I can’t force the provider to send a message that is syntactically wrong (as the server code enforces the syntax), that leaves me with trying to send a message with wrong data.
Are you trying to cover a case on your consumer side, where it handles the errors returned from your provider, in the case of bad data? If you can't force your provider to send a message that in syntactically wrong because the code enforces it, would you expect that scenario to happen in production, and therefore worthy of testing? What is the particular use case here, is your client posting a message to a queue and expecting to do something at some point later with the provider response. Where is the providers response being posted too? There is always a little overlap and I wouldn't be concerned but you wouldn't want to cover all the edge cases. If the shape of the error message is always the same, then you can just test that one case. If in specific error cases, you do some different work in your client, based on a specific response, then I would want a contract to cover that.
My experience with async has been posting messages to SQS and various lambdas listening to them and performing actions (such as generating a file and saving it in a bucket, based on a ref or payload provided in the message). The only responses the client would get, would be that from SQS as to whether they had published the message correctly or not.
I would have lots of provider tests (not Pact tests) that ensure that my provider returns nice error messages, when things go bork. We would consider the difference between retryable errors and non-retryable errors. a valid client payload (to the queue) but garbage to the provider (non existent id for example) wouldn't be retried and chucked straight on a DLQ. If the db connection was down we would allow the SQS retry mechanism to kick in, and dump in on the DLQ after x redrives
n
Are you trying to cover a case on your consumer side, where it handles the errors returned from your provider, in the case of bad data?
Correct. In this particular case the provider communicates with the consumer via kafka. Essentially when an upstream event occurs the provider writes an event to a kafka topic, which the consumer reads, does additional processing and then responds to another consumer further downstream. This interaction is started by the provider. I take it from your answer that I should handle the majority of unhappy paths (messages with wrong data) on the component tests, and pick a few relevant cases for Pact -- even that makes the tests overlap a bit. That makes sense to me.
y
Have you ever seen this site? It is really good to quickly make diagrams to represent your domain, I find it quite useful as visual references are easier to relate to than text. https://www.websequencediagrams.com I’ll have a think on your particular problem, first thing that stands out is there are three parties here, one is acting as a pass through with transformation. Not sure if it’s different in Async land but I know that causes pain in provider verification with regular consumer driven contract testing and we have suggested a slightly different pattern. I’ll find the docs. https://docs.pact.io/getting_started/what_is_pact_good_for#why-pact-may-not-be-the-best-tool-for-testing-pass-through-apis-like-bffs This may or may not be applicable to your use case so take it with a pinch of salt
🙏 1
👀 1
n
In my case it is not a pass through API the individual micro services do implement a fair bit of logic which controls whether they interact with the other services dowstream and the content of that interaction.
b
A lot of async things end up being pairs of sync things: instead of getting an immediate response for final success (or failure), you get first an immediate response of acceptance, and then later a separate call back for final success (or failure). If it can truly be modelled in that way, then it's multiple interactions, simple 🙂
If not, then a diagram would help a lot. It's hard to visualise which bits you want to cover in just words.
Generally, things like malformed messages (e.g. bad syntax) aren't part of the contract, they're at a more fundamental layer (e.g. web framework rejects malformed requests, even if you control the error response shape).
But things that can be invalidated with domain logic (i.e. contract things), can't often be invalidated with only shape or syntax checks, without refactoring your transaction models. That's exactly what contract tests aim to solve, across network boundaries.
n
The whole system I am testing contains many microservices, typically any two pairs of services interact in the following manner: • service A sends an HTTP request to service B • service B responds with a 200 if the request is valid • service B does additional processing and may send a request to a service C • service B writes an event to a Kafka topic when it gets the response from service C • service A reads the event from the topic
Indeed there are multiple interactions involved @Boris
My question was more generic, I wasn't targeting a particular interaction.
👍 1
b
sounds like you want • A->B http • B->C http • B->A message
and you might have multiple states and/or interactions for each, including unhappy paths if they're warranted 🙂
n
my question was precisely whether or not I should cover the unhappy paths in Pact.
😎 1
b
it's valid to have cyclical relationships, but you have to be more careful with them, because they can be hard to change
how much you want to cover unhappy paths is usually determined by how much confidence they'll give you
There is always a little overlap and I wouldn't be concerned but you wouldn't want to cover all the edge cases. If the shape of the error message is always the same, then you can just test that one case.
If in specific error cases, you do some different work in your client, based on a specific response, then I would want a contract to cover that.
That pretty much covers the general advice 👌
But, for example, if you have a complex multi-step transaction model, and there are significantly different errors possible in different transitions, then you probably do want coverage of them.
That coverage might be better placed in unit tests, especially if the error shapes don't change.
Generally, I have centralised error handling at the gate where messages are marshalled through the data layer into domain-meaningful models, so I tend to only need one sad path in contracts, for each category of error. The branching for how the errors are handled can be done internally, outside of the contract.
(I think I got there in the end, sorry for the other tangents 😅)
n
That makes a lot of sense
🙏 1
Thank you very much.
b
np ^_^ conveniently, I think the async bit might be a red herring
n
On the synchronous interactions the provider responses have pretty much the same format, the only thing that varies is the message, and in some rare cases the status code.
But on the async interactions there are is a larger number of variations that cause the consumer to reject the message
So I was debating whether I should handle the unhappy paths on the component tests alone.
But your response shed some light
I have centralised error handling at the gate where messages are marshalled through the data layer into domain-meaningful models, so I tend to only need one sad path in contracts, for each category of error. The branching for how the errors are handled can be done internally, outside of the contract.
Essentially I only need a Pact test to exercise the bit of the client code that rejects the message and I can test the various error cases on the component tests / other unit tests.
👍 1
👌 1
b
in that case, the state for the message consumer might be like "a transaction is in progress" or "a transaction is in <blah> state", but either way, the response is probably a pithy "ok", and the actual error handling happens inside the app, as you suggest 👍
Open in Slack
PreviousNext
Powered by