I’m a bit confused about the way consumer tests wo...
# general
f
I’m a bit confused about the way consumer tests work when it comes to multiple interactions. For context, I’m working on a C++ wrapper that uses a pact plugin for a custom transport, in case that matters. When I create a pact and define the interactions in each test case, each time my
runTest
method is called which calls
pactffi_create_mock_server_for_transport
I see the
StartMockServer
method being called on the plugin with the JSON pact only containing the interactions defined up to that point. Now the part I don’t understand is what the mock server is supposed to do here, is it supposed to listen for all of the defined requests? If so, how are you supposed to distinguish between interactions with the same request but with different provider states? Or is the idea that the mock server only responds to the last defined interaction?
r
By design, each test starts it's own mock server. Provider states are only used on the provider side.
f
Ok, but so my concern is that within a single test file, there can be different test cases each with a different provider state, where the same request will give a different response. Right now if I try to write a test file like that, the plugin gets called to start the mock server for each test case, but the pact data will contain multiple interactions, with different responses for the same request.
So what request is the mock server supposed to check for, and what should it return?
r
Each test should define the expected interactions for that test. Then the mock server will respond with those interactions. The mock servers are very short running.
Each mock server on lives for while the test is running. For the Rust example, the mock server will go out of scope at the end of the test block, and the destructor will run to shut it down.
f
Ok, I’ll need to verify my code (because I think I found a bug) but in case you know off the top of your head how pactffi behaves with plugin servers, should it work like this? 1. Set the interactions with
pactffi_with_body
calls 2. Call
pactffi_create_mock_server_for_transport
3. Plugin’s StartMockServer rpc receives the interactions set in step 1 4. Pactffi clears the interactions set in step 1 once the mock server stops My wrapper code does some caching of the interaction data so the
pactffi_with_body
calls happen as part of the
runTest
method of the wrapper, and I just found it’s not clearing this cache
r
Each test would create a new PactHandle, so there is no step 4, more like a step 0 which creates a new handle
f
Oh okay, I was under the impression you needed a single PactHandle to build the pact file with the multiple interactions, I’m seeing each test overwrite the previous pact file
r
It should merge it on disk after each test
There is probably a flag to make it overwrite the file
f
Yes, that’s what I’m thinking now, I’m going to see if I can test it making sure that flag is set to false
facepalm thankyou
That was indeed the issue
I should probably read up on the overwrite flag to better understand the behavior
Thanks a lot for the help!
r
No worries!
f
One thing that just occurred to me though is one of the reasons I ran into this is the performance of the tests. If each test needs a new pacthandle, then that means the plugin needs to be started up and stopped for each test, which we’re finding to be quite slow.
r
It should try re-use the same running plugin, the plugin driver does that
f
hmm ok, I’ll take a deeper look at the plugin lifecycle then, maybe I’m reaching the wrong conclusion based on how I implemented some stuff like logging
I see in the pactffi log file the plugin is being cleaned up and restarted. It’s being triggered by the PactHandle being cleaned up by my wrapper code and doing the following calls:
Copy code
pactffi_cleanup_plugins(pact_handle);
pactffi_free_pact_handle(pact_handle);
According to the function documentation, the plugin is ref counted so it’s reaching 0 after every test… Should I be holding on to all the pact handles in the test until all the test cases are done?
r
Hmm, need to check this. Is there a way for you to call those functions in a test teardown function? Or maybe run the tests in parrallel?
Ideally it should only shut the plugin down once all tests are done
f
Yes, with a bit of extra work I can hold on to those handles and clean them up before exit. Ideally I’d like to prevent this as it’ll be an extra burden in writing consumer tests, but it’s manageable.
r
The only way I could see it working is add a flag to the
pactffi_free_pact_handle
to not shut down the plugins, but then we loose which handles are using which plugins so you would need to have a way to shut down all plugins at the end of the test run.
The gRPC plugin has a timer built in so it shuts down automatically after 15 minutes of being idle, to avoid hanging processes in case the tests don't clean up properly.