https://pact.io logo
Join Slack
Powered by
Hi I noticed on my API the pact verifier is making...
# general
h
Hi I noticed on my API the pact verifier is making two sets of requests to run one test:
Copy code
2023-04-18 10:44:07  "POST /_pact/provider_states HTTP/1.1" 200 4 "-" "Faraday v1.10.3"
2023-04-18 10:44:07  [INFO] base.py(270): POST <http://elasticsearch:9200/battery.notification*/_search> [status:200 request:0.024s]
2023-04-18 10:44:07  "GET /battery-notifications?limit=100&offset=0&id=36baa499-ce79-4c03-96e3-93a90fb7f1d9 HTTP/1.1" 200 309 "-" "-"
2023-04-18 10:44:28  "POST /_pact/provider_states HTTP/1.1" 200 47 "-" "Faraday v1.10.3"
2023-04-18 10:44:30  [INFO] base.py(270): POST <http://elasticsearch:9200/battery.notification*/_search> [status:200 request:0.047s]
2023-04-18 10:44:30  "GET /battery-notifications?limit=100&offset=0&id=36baa499-ce79-4c03-96e3-93a90fb7f1d9 HTTP/1.1" 200 309 "-" "-"
My provider states endpoint is set to return after a 20 second delay to give my app time to be ready but it seems that the verifier is using the result from the first request before the app was ready, and the test is failing because there is no data available yet. test output for the above log:
Copy code
Verifying a pact between fleet and battery-api
  Given Fleet has a list of notifications
    a request for notifications
      with GET /battery-notifications?limit=100&offset=0&id=36baa499-ce79-4c03-96e3-93a90fb7f1d9
        returns a response which
          has status code 200
          has a matching body
          includes headers
            "Content-Type" which equals "application/json"

1 interaction, 0 failures
WARN: Ignoring unsupported combine AND for path $['vehicle_id'][0]
WARN: Ignoring unsupported combine AND for path $
It shows 1 interaction but two are always happening behind the scenes. I can see all my test data is duplicated twice in the local instance of the app. It shows no failure locally because my app is now full of this data but in CI where it is a fresh instance, it always fails with:
Copy code
Matching keys and values are not shown
791        {
792       -  "notifications": [
793       -    {
794       -      "id": "e2490de5-5bd3-43d5-b7c4-526e33f71304",
...
806       -    },
807       +  "notifications": [,
808       +
Can anybody advise me please?
m
Can you please share the pact file that is being replayed here? Also, please set the logs to debug/verbose and share (redacting any private info)
✅ 1
Please share the full log output, I appreciate you’re trying to be succinct/helpful, but the long form logs will be most helpful I think here
✅ 1
h
scratch_1.json
scratch_1.json
OK the verbose log in CI actually shows
Copy code
Failures:
...
  3) Verifying a pact between fleet and battery-api Given Fleet has a list of notifications a request for notifications with GET /battery-notifications?limit=100&offset=0&vehicle_id=36baa499-ce79-4c03-96e3-93a90fb7f1d9 returns a response which includes headers "Content-Type" which equals "application/json"
     Failure/Error: set_up_provider_states interaction.provider_states, options[:consumer]
     Pact::ProviderVerifier::SetUpProviderStateError:
       Error setting up provider state 'Fleet has a list of notifications' for consumer 'fleet' at <http://localhost:8000/_pact/provider_states>. response status=500 response body={"message": "Internal Server Error"}
     # ./bin/pact:15:in `<top (required)>'
1 interaction, 1 failure
That's interesting, I don't get an internal server error in the local Verbose log It's not a Pact issue then it's something to do with my environment.
Though I would still be curious why it seems to make duplicate requests when running locally
local verbose log.txt
local verbose log.txt
the verbose log shows two runs but no clear reason why and in the ordinary log that all gets rolled up into one result
It seems to be running some other version of this pact
m
ah!
docker run --net=host -e VERBOSE=‘true’ -e PACT_DO_NOT_TRACK=‘true’ pactfoundation/pact-cli:latest verify ‘https://pact.dev.somedomain/pacts/provider/battery-api/consumer/fleet/version/a97efee-QE-74-batterycontracts+a97efee.SNAPSHOT.runner-5ybevqbc-project-456-concurrent-3’ --pact-broker-base-url=https://pact.dev.somedomain/ --provider-base-url=http://localhost:8000 --provider-states-setup-url=http://localhost:8000/_pact/provider_states --provider=battery-api
You’re providing the broker information to dynamically discover pacts, and you’re passing in the specific URL
So it’s discovering two pacts, one dynamically:
The pact at https://pact.dev.somedomain/pacts/provider/battery-api/consumer/fleet/pact-version/912bb33eea2af5694ce93c3c7b5517c4be3e7ad9 is being verified because the pact content belongs to the consumer version matching the following criterion:
* latest version of fleet from the main branch ‘develop’ (1b87217-featuremoving-status-embedded+1b87217.SNAPSHOT.runner-5ybevqbc-project-456-concurrent-2)
and
INFO: Reading pact at https://pact.dev.somedomain/pacts/provider/battery-api/consumer/fleet/version/a97efee-QE-74-batterycontracts+a97efee.SNAPSHOT.runner-5ybevqbc-project-456-concurrent-3
The one you explicitly asked for
If you want to dynamically discover pacts, don’t pass the Pact URL explicitly (and you should pass in additional selectors). If you only want to verify a specific URL, don’t provide the pact broker details
h
Ah, thank you Matt that's really helpful. I think in my implementation it makes sense to validate only the explicitly named pact because this is supposed to run in the context of a webhook from the broker to GitLab done at the time the pact changes. So in my mind, a new Pact is generated on the consumer pipeline, and that is the one I want validated, I don't really care about any other version. Does that make sense or am I doing it wrong? How do I stop it retrieving the pacts automatically? I still want the verification results to be published so I would need to provide the broker details for that wouldn't I?
Maybe it would be easiest to remove the explicitly named Pact if the automatically retrieved one will always be the latest, which it should be. Though I don't necessarily want the latest, I want the one that relates to a specific upstream branch of the consumer build. So I think I am right to pass in the Pact uRL
The command in CI is currently
Copy code
docker run --net=host -e VERBOSE='true' -e PACT_BROKER_BASE_URL=<https://pact.dev.somedomain/> -e PACT_DO_NOT_TRACK=true pactfoundation/pact-cli:latest verify "$PACT_URL" --provider-base-url=<http://localhost:8000> --provider-states-setup-url=<http://localhost:8000/_pact/provider_states> --provider-version=$CI_COMMIT_SHA --publish
m
How do I stop it retrieving the pacts automatically? I still want the verification results to be published so I would need to provide the broker details for that wouldn’t I?
the verification results will be published as long as you set the flag for it. Because the URL is a PactBroker URL, it will have the HAL relations in the response, and the verifier will use that URL to publish results.
To stop it using the dynamic lookup, just drop off the 
--pact-broker-base-url
arg
Maybe it would be easiest to remove the explicitly named Pact if the automatically retrieved one will always be the latest, which it should be. Though I don’t necessarily want the latest, I want the one that relates to a specific upstream branch of the consumer build. So I think I am right to pass in the Pact uRL (edited)
that’s right. You’re doing the right thing, but you just don’t want to dynamically retrieve pacts for the webhook (and the reverse for regular builds)
2 Views
Open in Slack
PreviousNext
Powered by