Hi, team! Just realized that pact_verifier_cli exi...
# pact-broker
s
Hi, team! Just realized that pact_verifier_cli exits with status 0 even if the pact verification fails. This makes it difficult to configure the CI/CD pipeline. Could someone advise if there is a way around this without resorting grep output?
y
Hey, This would best be raised in pact-rust as it is built on from the pact-reference (rust-core) repo. It should exit with 0 checking on previous issues https://github.com/pact-foundation/pact-reference/issues/47 would you be able to create a reproducible issue on there please
s
I deployed a new broker from scratch, no pact in it yet. Then I published the first contract in it and made the very first verification attempt (it failed). And it still ended with a status code of 0. So I don’t think it has anything to do with whether it’s a new issue or not.
@uglyog Could you please tell me is this expected behavior?
y
Please don't tag maintainers directly, as stated you can raise an issue in the respective repo please
s
I’m sorry, Ok
y
It sounds like a regression based on the previous issue which was raised (sounds similar to yours) and was fixed as a bug, so raising an issue, with your exact scenario would help the maintainers and contributors offer support 👍
Hope that helps buddy
The code should be failing with an exit code of 2, if there are mis-matches, 0 if successful, and 1 if anything else https://github.com/pact-foundation/pact-reference/blob/f63b7b66cc5d275f6331eeccf9f05f2a4de6359d/rust/pact_verifier_cli/src/main.rs#L395-L410
that code was recently changed to introduce a new report format (junit) https://github.com/pact-foundation/pact-reference/commit/c9333f94cfe5db67a75a227d250418cbba461753#diff-e4e228771bcfba0411[…]7eba75b960173e750e02fcc2 released as Pact Verifier CLI 0.10.3 is this issue also present with an older version?
s
Let me check
y
actually sorry, that piece of code I showed earlier will exit if it fails to write the errors to a file (json or junit) this portion https://github.com/pact-foundation/pact-reference/blob/f63b7b66cc5d275f6331eeccf9f05f2a4de6359d/rust/pact_verifier_cli/src/main.rs#LL391-L393C8 this maps an error
Copy code
.map_err(|err| {
      error!("Verification failed with error: {}", err);
      2
    })
not sure if that is used on 410, or it just exists with exit code 1 on a failure
I've just tried a few examples • no pact found • no provider running • failed verification with local pacts and all failing with exit code 1
Copy code
❯ pact_verifier_cli --version
pact verifier version   : v0.9.19
pact specification      : v4.0
models version          : v1.0.2
will update to latest verifier
s
Unfortunately, the two previous versions are subject to this issue.
Didi you tried v0.10.x ?
v0.10.1, v0.10.2, v0.10.3 — the same for me
y
I've just said I will upgrade now and try, can you show us the exact commands you are using and your failing output, and successful exit code
my examination so far shows the correct exit code being emitted
y
ty!
happy hearts
it's because you have
enable-pending
set
Copy code
This pact is still in pending state for any version of ProductService from branch 'feature/flask-smorest' as a successful verification result has not yet been published

Pending Failures:

There were 6 non-fatal pact failures on pending pacts or interactions (see <http://docs.pact.io/pending|docs.pact.io/pending> for more information)
s
Yeah
y
you've not verified that consumer pact, on the feature branch and therefore the verifications wont fail your verification task
s
Just got it
y
that is by design
s
all the magic in
--enable-pending
y
if your provider was incompatible with the deployed consumer it would fail with an exit code of 0
running through the pact ci/cd workshop introduces these concepts 👍
s
Thank you so much for helping me understand
🙌 2
Again, I apologize for my conduct
no problem 1
y
It's ok chap! Appreciate you wanting to get involved and contribute, but it's just difficult when people get tagged (some people are in different timezones as well). That was a really good bug report on GitHub so thank you for that. The art of asking a good question, is a skill that takes a lifetime to hone. TY for your Python efforts too (we would like to increase maintainer / contributor capacity there so your eyes/PR's are appreciated, even if we don't get to them straight away!)
🔥 1
💯 1
u
Glad you sorted it out. For a moment there I thought I was going to get shouted at for breaking something.
😆 1
m
Again, I apologize for my conduct
not at all!