Hey there everyone! I have a question on the flow of working with pending pacts. We just ran into a problem, where the pact was verified on a feature branch and the results were published to the broker as part of the pipeline. This causes the provider tests on most other feature branches to fail. The documentation (and from I can tell the code as well), a pact is verified if either a) The contract has been verified on this branch before. b) The contract has been verified on any branch, that existed before this branch. Does this mean, the other feature branches would need to wait for the branch that originally verified the contract to be merged? Or shouldn't a feature branch upload verification results in the first place? For context: • the consumer team hasn't fully finished up setting their pact configuration and so far has only published pacts for their tagged versions (where the providerBranch is also set to the Tag). Their main branch in the Pact Broker is not yet set. • we are using WIP pacts What are we doing wrong?

Sorry Tim, have been a bit AWOL here / heads down Just so I’m clear, the story is something like this 1. Branch A is created 2. Branch B is created 3. Consumer X adds a new pact with new expectations 4. Branch C is created, verifies (3) 5. Branch A starts failing 6. Branch B starts failing Pending pacts and WIP pacts are enabled, so branches A and B are also collected and verified by Branch A and B?

Not quite. 1. Branch A is main. 2. Consumer X adds new pact with new expectations 3. Branch B is created from A. verifies, but is not yet merged. Verification Results are published however (maybe that's the mistake?) 4. Branch C is created. Contracts fail. 5. Branch A continuous to be okay. I think this is by design, pending pact documentation says

A pact's content is pending for all branches of a provider until the first successful verification has been published.

I just don't get why the pact is no longer verified, if the verification success was on a feature branch.

Got it. I wonder if Branch A-1 continues to pass, or all of a sudden starts to fail also.

I just don’t get why the pact is no longer verified, if the verification success was on a feature branch.

this is new info. What’s not verifying it?

sorry, i meant: I just don't get why the pact is no longer PENDING, if the verification success was published on a feature branch.

OK that makes more sense. I suspect it is by design, although admittedly this is detail that I tend to forget (just re-reading the details now).

A pact’s content is pending for all branches of a provider until the first successful verification has been published. From then on, it is no longer pending for the provider branch that published the successful verification, and will not be pending for any other new provider branches created thereafter. It will stay in pending for branches that already existed at the time of the successful verification.

So I think this confirms it, and presumably the feature branch prior to A (

A-1

) will also treat the pact as pending

false

. The reason for the “…any other new provider branches” logic is probably because it’s hard for the broker to check if new branches actually contain the code that implements the pending pact, and crudely says “it was created after this branch” (implying it may have that code in it).

What I would expect (at least with our branching) is that pacts are pending as long as they haven't been verified on main. But I understand this might be different in other branching models (although I would think most setups to expect that). The only workaround that comes to mind for me would be not publishing verification results on feature branches. Do you have any other ideas? I'm also interested in why it seems no other projects seem to have head this problem? Maybe there are little cases where the branch is not merged when the contract tests passed?

It’s a good question. I think one way to potentially try and solve this, is make pending more user configurable - rather than Pact Broker enforcing a workflow, allow the users to decide (if they want to override the behaviour).

In terms of short term options, I suppose not publishing verification results could work, but then you’d lose out on other important info (like, is the branch safe to merge/deploy)

Are you sure? I would still publish but only on main (or whatever is the default branch). Then the consumer branch would be safe to publish (or deploy) when the provider has merged (which is correct). What I think would change is: • the consumer would know a little later, that there is a version of the provider that is able to verify this pact • a feature branch that once verified the contract would be allowed to break it again without failing the test, because the broker does not know of the succesful verification

sorry buddy, had a few days of scattered leave, had seen this in general but hadn’t been able to give it the time. https://docs.pact.io/pact_broker/advanced_topics/pending_pacts#how-it-is-calculated

A pact’s content is pending for all branches of a provider until the first successful verification has been published. From then on, it is no longer pending for the provider branch that published the successful verification, and will not be pending for any other new provider branches created thereafter. It will stay in pending for branches that already existed at the time of the successful verification.

a pact would be pending for the providers main branch, even once it has been verified by the consumers feature branch. new branches would not see it as pending, existing branches would see it as pending. you should have

enablePending

set to true, in the providers job, and you could restrict the include WIP pacts, to only run on the providers main branch. that way provider features branches only test against your consumer version selectors ( deployed/released true, mainBranch true is the default), and don’t pick up WIP pacts.

Does this mean, the other feature branches would need to wait for the branch that originally verified the contract to be merged? Or shouldn’t a feature branch upload verification results in the first place?

I don’t believe, the earlier branches would need to wait for the verification. Will the enablePending flag, it would not cause the CI job to fail, (as these are considered to not have been pending). They will only have to honour the deployed/released and consumer main. consumer main failure shouldn’t block provider, if provider has never verified it before (ie consumer missed going through a feature branch to verify changed pact first), enablePending, avoids the provider main branch from stopping being deployed, for a version of a contract is has never verified.

We have been able to reproduce this issue with a local broker setup and it doesn’t seem to be relevant, wether the consumer has the mainBranch set or not. What I would expect is that pacts are pending as long as they haven’t been verified on main. Why isn’t this the case?

Nice work on a local setup, might be worth creating a public reproducer mapping out the scenarios and expected behaviour, so that we can huddle around the example and test out hypothesis.

• the consumer team hasn’t fully finished up setting their pact configuration and so far has only published pacts for their tagged versions (where the providerBranch is also set to the Tag). Their main branch in the Pact Broker is not yet set.

You can’t use pending pacts with your current setup, as you are using tags, not branches

• Ensure consumer has configured the mainBranch property

• Ensure consumer sets branch and version properties when publishing pacts

• Ensure provider sets branch and version properties when verifying pacts

• enable the pending pacts feature by setting

enablePending

(or equivalent for your language) to

true

in the provider verification configuration

Hey @Yousaf Nabi (pactflow.io), thanks for your extended response, though I have some questions on it.

a pact would be pending for the providers main branch, even once it has been verified by the consumers feature branch. new branches would not see it as pending, existing branches would see it as pending.

Yes, that's the problem. Any new branch will not see the contract as pending, but also not contain the changes to verify it successfully, because the implementing branch has not yet been merged.

you should have

enablePending

set to true, in the providers job, and you could restrict the include WIP pacts, to only run on the providers main branch.

enablePending

is configured. WIP currently are enabled for all branches. However, if I disabled it, the feature branch trying to implement the new contract also wouldn't include it in its verification attempt, right? This means that I would need to specify it in the consumer version selectors to verify it on that branch, which seems to eliminate most of the benefits of pending pacts and WIP pacts.

I don’t believe, the earlier branches would need to wait for the verification.

Yes, but we have problems with the branches created after the branch that initially verifies the contract, but that is not merged.

Nice work on a local setup, might be worth creating a public reproducer mapping out the scenarios and expected behaviour, so that we can huddle around the example and test out hypothesis.

This was just a bit of docker compose and fiddling with the Example App and Example API project.

You can’t use pending pacts with your current setup, as you are using tags, not branches

When I referred to as Tags here were GitLab Tags, used for versioning the service.

• Ensure consumer has configured the mainBranch property

• Ensure consumer sets branch and version properties when publishing pacts

To be honest, I don't quite get how these influence the problem. Yes, the actual consumer seems to have messed up setting these correctly, but I was able to reproduce the mentioned issue with a setup that did have consumer branch, version, and mainBranch set correctly, but I could check that again. Also, from what I can tell the pending pacts implementation in the broker doesn't seem to care for any consumer property whatsoever when calculating the pending state of the contract.