Hi all - we have some unit tests using Pact via FF...
# general
a
Hi all - we have some unit tests using Pact via FFI for contract verification and I'm trying to improve their performance. The Application is in C++ and we run the tests via GTest. We use an updated fork of Pact's C++ client which uses FFI under the hood. Each test does the following - 1. Creates a provider instance 2. Instantiates a custom plugin 3. Performs verification Rust is not my area of expertise, so apologies if this isn't correct but it looks to me like each call to pactffi_using_plugin will block the client thread for at least 0.5s, so each of our unit tests is taking at least that long to complete.
Copy code
let result = RUNTIME.block_on(async {
          let result = load_plugin(&dependency).await;

          // Add a small delay to let asynchronous tasks to complete
          sleep(Duration::from_millis(500)).await;

          result
        });
I've tried using a fixture to make the plugin and provider instance persistent across multiple tests, but haven't had much luck (it looks like there's some state not being cleaned up between unit test runs - or maybe ths issue is with our plugin?) - the first test passes but subsequent ones fail due to contract verification failures. I can probably provide a minimal example but I thought I'd ask first in case you had any ideas. Many thanks!
y
Is this a provider verification task we are talking about here, or on the consumer side?
pactffi_using_plugin
needs to start the plugin, wait for it to start up and report back its running port before it can move on.
pactffi_using_plugin
is called on the consumer side, for the provider side, it is read from the pact itself and I don’t think it will call that function. Looking at the history, it was added in to fix a race condition https://github.com/pact-foundation/pact-reference/commit/213d1459c578662532e64ec7a1b1ce9af15cb676 and then again the code was subsequently updated https://github.com/pact-foundation/pact-reference/commit/d41e2440a8d5011e51a90eeb44dcaa7dbe448b0d
The issue with subsequent tests failing seems separate. If this is the consumer side, it may be possible that the pact handle is being reused and the mock server is not being cleaned up
pactffi_cleanup_mock_server
or if that method isn’t being called, it isn’t shutting down the plugin. Repro would be really useful.
a
Hi - thanks for the rapid response. It's consumer side. I'll put together an example using one of the example plugins.
The problem is that
TEST_2
fails, because in the second test,
pact_ffi_mock_server_matched
still checks for the interactions that were specified in
TEST_1
(even though a new mock server has been created on a different port). You can see this if you write out the pact file from each test. I need some way of clearing all the interactions from the
PactHandle
at the end of each test, so that I can reuse the
PactHandle
(and hence its plugins).
thank you 1
I'll code up a proper working example in case the pseudocode isn't clear, now that I have something minimal.
y
Ahh I believe you should be creating a new pact handle each time, this is how you can support parallel running tests. Pact will handle merging pacts, and file locks on the resultant pacts
your plugin should be capable of being invoked multiple times (multiple sub processes, providing a random port each time, ideally checking the port is free before allocating), with each pact handle invoking it. each pact handle returns an opaque pointer which is auto-incremented
So I think you can’t escape that delay without code modification and seeing how it fares (as it was put in for race conditions - which I hadn’t seen personally)
a
Thanks - I'll maybe just try removing that delay. If it works I'll add a PR to make it a parameter.
y
that sounds like a decent path forward chap, best of luck!
👍 1
a
I've submitted a PR - a quick test with a delay of
0
worked fine for me (and reduced my test suite runtime from 3 minutes to 18s).
wow 1
🚀 1