Hi Team, Our team at tripadvisor landed into unique issue: 1. Feature branch published pact(not new, existing pacts) 2. It triggered webhook and ran provider verification build. 3. Due to some intermittent issue(that I am checking), build failed. 4. Verification result was copied to all other consumer feature branches and canIDeploy check for all consumer feature branches started failing. 5. There was panic situation all around as all builds are failing and nobody changed pact. How it was resolved: I published new feature branch which triggered webhook and got consumer test verified. What i want to ask is : 1. What should be best way to resolve such situation? 2. When is webhook triggered, since its not new pact 3. Was the above behaviour expected or Does pact pipeline setup needs improvement?

First up, here are some tips for debugging webhooks: https://docs.pact.io/pact_broker/webhooks/debugging_webhooks. If using Pactflow, we have screens to see the webhook executions also

The verification result for a feature published pact shouldn't be applied to all consumers, so I think something is not set up right

The verification result for a feature published pact shouldn’t be applied to all consumers, so I think something is not set up right

even if contract is same?

The verification result is tied to the provider version. Because multiple provider versions can verify a contract

its gradle task where i setup these variables:

project.ext.pacticipantVersion = "${project.rootProject.ext.COMMIT_HASH}"
project.ext.pacticipant = "${project.parent.name}"
project.ext.toTag = "production"

so essentially it is, feature branch checks it has production tag for provider(This tag is added by webhook build which runs provider contract test on develop version of provider)

so If I understood, you’re adding the tag

production

on the develop version of the provider during verification?

yes, as for our case develop branch go to production

so then you’re getting the expected failure. Now all consumers can’t deploy to

production

because you just told it that the provider doesn’t satisfy the contract

Here are a number of useful hands-on labs that teach all of the key concepts: https://docs.pactflow.io/docs/workshops and https://docs.pact.io/implementation_guides/workshops

so then you’re getting the expected failure. Now all consumers can’t deploy to

production

because you just told it that the provider doesn’t satisfy the contract

yes, I understand that. Since it happened due to intermittent issue(not actual error), What should be best way to resolve it? rerun?

yes, i’d just re-run the provider build

unless there is a very short window from when code lands in

develop

and is deployed to

production

yes its 1-1.5 hour for us. We are working on ways to improve it

So that’s definitely a reason to structure your tags differently to avoid this situation.

yes, i’d just re-run the provider build

With above you mean webhook build that can add production tag?

webhooks don’t add tags, builds add tags

yup i mean webhook triggered build(we have different setup for both, these build just run provider contract test)

nice, that’s what they should do 🙂

I would re-run whatever build is failing that’s causing the issues. Ideally, find out why it failed and make it more resilient also

yes, I kind of did that.

but your tagging is iffy, i’d recommend not tagging dev builds

production

because that will prevent consumers deployinsg to prod if your dev builds of provider fail, even though production is not affected

For our provider pipeline, whatever is merged to develop goes to production(There is separate pipeline that runs after merge to develop which takes ~1 hour). We want things to fail if develop build is failing. Provider ci-cd pipeline verifies against release-version of consumer. Develop is kind of master for us

Currently we dont have way to tag after depolying to production,

What do you do if your deploy fails? Then your tags will be inaccurate, and you may have problems

not for now