Hi folks! I'm updating our provider verification ...
# pact-js
j
Hi folks! I'm updating our provider verification tests, and have come across some interesting things around the
afterEach
hook. 1. The documentation for the lifecycle seems to be incomplete, as it doesn't mention when the
State Handler Teardown
step runs. After some trial and error, I believe the following is correct:
Copy code
BeforeEach -> State Handler Setup -> Request Filter (request phase) -> Execute Provider Test -> Request Filter (response phase) -> AfterEach -> State Handler Teardown
Is this right? Happy to get a PR open to update the docs if this is correct. 2. Are there any constraints around the
afterEach
hook, such as maximum execution time? In the
afterEach
hook for our tests, I'm resetting some of the data in our test database, as per one of the use cases described for it in the docs. However, it seems that my
afterEach
hook does not always finish executing. I've littered it with logging debug statements and
try/catch
statements to find out exactly where it gets to, and there are no errors being thrown - the logging statements sometimes just stop. What might I be missing? Any help is appreciated! Thanks in advance šŸ™‚
t
One moment, I’ll check the code
1. Yes, that’s correct. This is probably a bit unintuitive given the name, and might be worth changing so that it runs in the other order. (also I think there might be a bug with the way it works, it doesn’t look right to me, but it’s late here and I’m no longer a maintainer so I don’t know if things have changed - I know not many people use
afterEach
, so ….possibly we haven’t noticed).
For that, I’ll do a closer read tomorrow and either confirm it’s correct here, or raise a PR that fixes it.
Reading your question 2 now
. However, it seems that my afterEach hook does not always finish executing. I’ve littered it with logging debug statements and try/catch statements to find out exactly where it gets to, and there are no errors being thrown - the logging statements sometimes just stop. What might I be missing?
Right. Yes, there’s a bug.
There are no intentional constraints, other than the usual timeouts that your tests have.
j
Thanks for the speedy (and late night) responses! 1 - sounds great, like I said, happy to help improve docs for others if I can. 2 - interesting - ok, glad to know I hadn't missed something obvious. I did dig into the code a little myself because my gut feeling was "is this just a missing
await
?", but didn't see an obvious cause. Again, happy to contribute. Do you have an idea of what the cause is already, or do you need anything else from me to help reproduce?
t
Ok, so some non-obvious context: 1) the way that pact-js works is to defer all the mock requests / contract checking to the Rust core (which is wrapped in the js dependency
pact-core
). 2) The rust core expects to call setup and teardown hooks on an endpoint running on your provider, but this is clumsy for most users in JS land 2) But, pact-js doesn’t ā€œknowā€ when the setup and teardown hooks happen. 4) To get around this (and to allow a DSL where you can provide native JS setup and teardown hooks), pact-js runs a proxy server (built with express) that proxies the app that you’re verifying. It catches the request to the setup hook, swallows it, and calls the user provided hook.
This line looks suspicious to me. I am pretty sure (but not certain) that it is an error in express to: 1) Call
next
more than once 2) Call
next
and return a promise (or have an async function, which is like returning a promise)
Also, the way that reads is it does it on every call that is not a state setup call.
j
Right, that makes sense - I was curious about the motivation for the Express-based proxy, but that context really helps.
šŸ‘ 1
t
The reason that was done is because not every test has a state, so you can’t rely on the teardown call (because there might not be one)
Really,
beforeEach
and
afterEach
(or similarly named states) should be special states in pact that always happen
Oh yeah, also: 5) The ā€œstateSetupā€ endpoint is also used for teardown. Whether it’s setup or teardown is a parameter in the body of the request that the core makes. So, what I think the intent of this code was to say ā€œhey, if this is a request that isn’t a state setup, do the normal request, then before you return, additionally do this teardownā€
But I don’t think that’s what it does.
Practically, what you could do if you control the consumer is to add a teardown hook for a named state that you make sure is on all interactions in the contract
(thus avoiding this bug)
if you feel like digging in to it, you’re very welcome to, of course
j
I'm wondering if this placement is actually the issue. The call to
next()
to trigger subsequent middleware is happening before the call to the
afterEach
hook. Good shout on the "do teardown" state. I'll give that a go as an immediate workaround to unblock me, and then if I get chance, I will have a play about with the middleware execution order.
t
Yep.
I think it is. I also think it’s wrong to have an
async
function that also has a
next
My guess is the intent was to call
next
first, so that the request would do its thing, then do the teardown
basically we just need to make sure that the
afterEach
is only called when the proxy returns.
another way to fix it (in pact) might be moving this code entirely to the place where the proxy request is made
j
Right, ok, because this middleware isn't actually being invoked after the main request processing, so it's trying to workaround that fact.
t
I think that might be a library.
yep
lemme see if I can find the place where the proxy call is
šŸ‘ 1
Yep, that’s here. I think any fix involves a close read of the
express
or the
http-proxy
docs (or both)
j
Cool, ok. I'll try your suggested workaround for now, and then explore the underlying issue more if I get a chance. What's the normal process for tracking this kind of thing? Raise a bug issue in the repo?
t
A bug would be great. I’m no longer a maintainer, just an interested community member
šŸ‘ 1
*bug report
j
And I appreciate your help all the more for it šŸ˜„ thanks again for the support!
t
You’re welcome!
I think it would be ideal if pact had framework-wide before each / after each calls - I’ll raise a feature request and link it here so you can upvote it
šŸ‘ 1
I’m also working on a new contract testing tool that will have this feature (and solves a few other pain points with pact). My plan is for it to be as pact compatible as possible - the pact / pactflow brokers will be part of the workflow, just like with pact - giving access to all the can-i-deploy, etc. One feature it does have is the ability to run each verification individually, so you can just use Jest’s
afterEach
and it will work. However, it’s not quite ready for use yet.
I’m going to start working on the documentation tomorrow - at the moment it’s just a loose list of notes to myself - but if you’re interested you can follow it here: https://github.com/TimothyJones/case
j
Sounds interesting, thanks for sharing šŸ™‚
t
In summary - 1) PRs for pact definitely appreciated - although maybe hold off on the documentation one, as it might not be true once the issue with the afterEach is fixed. 2) Sorry about the bug - it looks like I was the last person to touch that code, so I probably missed it at least once 3) I’m glad to hear that the workaround of a special state that everything has will work for you šŸ™Œ
šŸ‘ 1
Oh, actually - If you’re not using multiple states per interaction, you can also just cheekily put
clearEverything()
at the top of each setup state handler. That might be faster, but would fail if you’re using more than one state for the same interaction.
That’s what we used to do before teardown and multiple states were features
j
Alas, we're using multiple. But it's not too much hassle to chuck the extra
given(...)
in the consumer tests.
Just tried (3), and seems to work as expected šŸŽ‰
t
Awesome!
j
I'll hold off on docs changes for now, but I'll get the bug raised for (2) later on today šŸ™‚
t
šŸ‘ 1