I wasn't able yet to replicate this issue to actua...
# general
a
I wasn't able yet to replicate this issue to actually report it with repro steps, but just for my sanity I'd already like to confirm that something like this probably shouldn't happen (first image). The verification result is marked as failed even though all the interactions (just one here) have succeeded, which seems quite odd to me. However, in the same run where that verification result was published, there was one other unrelated contract that did fail (on purpose). I'm guessing the failure from this contract propagated somehow to this other contract, but I'm not yet sure how since I was unable to reproduce this with some pseudo-pacticipants. A follow-up question here actually is that when you run the
pact_verifier_cli
(Rust) using
--url
, should it also select additional contracts using the default consumer version selection rules? In the second picture the first contract was the one passed with
--url
but I didn't expect the verifications to run with what I'm guessing is the rule
latest: true
(or whatever is the default?) in addition to it.
I noticed this because
can-i-deploy
started reporting failures for the contract in the first image, even though I was quite sure all of the interactions should succeed.
I'll try to continue reproducing this situation with some pseudo-contracts tomorrow if I have time from team ceremonies, but I'm curious if this situation should at all be possible so that you have a failed verification with all interactions being successful 🤔
For reference, the full command for the verification was
Copy code
docker run -e PACT_BROKER_TOKEN pactfoundation/pact-ref-verifier:0.9.11
--provider-name An...-Backend 
--provider-version 3746...
--hostname ...
--port 443
--transport https
--state-change-url ...
--enable-pending
--publish
--broker-url <https://xxx.pactflow.io>
--url <https://xxx.pactflow.io/pacts/provider/An...-Backend/consumer/An...-UI/version/ba5d>...
--request-timeout 60000
m
There should be notices, explaining why certain pacts were collected and verified
if the pacts are different, I’d expect the verifications for each pact to be separately submitted. This could be a bug
@uglyog any thoughts?
We’ll ideally need a way to repro this to fix it, if it is of course
a
There should be notices, explaining why certain pacts were collected and verified
At least to my eyes if you run without specifying any
--consumer-version-selectors
manually, by default it doesn't report why it chose the ones it did. When giving
--url
the one specified by it always gets run first, but by default I assume it selects the rest with
latest: true
for all related consumers without any other mentions. If you do give the
--consumer-version-selectors
argument, then indeed it reports nicely why it chose the specific ones. (I already changed in my pipeline to run it with
--consumer-version-selectors "[ { \"mainBranch\": true }, { \"deployed\": true } ]"
to be safe).
I’d expect the verifications for each pact to be separately submitted
Yeah, I was expecting that there might be some weird condition where this might get violated based on what I saw.
We’ll ideally need a way to repro this to fix it, if it is of course
I attempted to do a quick repro with some pseudo-pacticipants but I was unable to get it to work at least so far. I'll try to continue later if I have time.
👍 1
m
Could you please try with verbose? That should should show us what options went across
a
The original situation is not available anymore since there have been additional contracts published after it. So I'll have to try to get the repro version up first.
m
Ok cool, thank you
r
This is going to be hard to replicate. Ideally, you shouldn't use the
--url
option if you are using the
--broker-url
one. You should select all the pacts via the consumer version selectors.