Hello, just starting to mess about with contract testing, and I have an issue where a consumer has published a contract tagged with a feature branch name. When I try to verify said contract from the provider (no webhook flow set up yet), I can't seem get any verification of said feature branch to take place. I've tried using

--include-wip-pacts-since

with a date well into the past but nothing. Am I totally misunderstanding how this is supposed to work?

👋 Could you please provide • which client language and version ◦ which tags/branches/command are you using to publish • which provider language and version ◦ which filters are you using to pull down • which pact file specification version

I'll do my best to provide the information you asked for, I've put in some placeholders. Client language: PHP, version: 7.1.1 Branch being published:

dev-3808

Command being used to publish contract:

docker run --rm pactfoundation/pact-cli:latest publish ./pact --consumer-app-version 11.16.116-80952e64 --branch dev-3808 --broker-base-url <broker base URL here>

(the above runs fine, shows up in the UI with the correct values) Provider language: Go, although I'm using

pact-provider-verifier

, version 1.36.1, at the moment Verification command:

pact-provider-verifier --broker-base-url=<broker base url here> --provider-base-url=<http://localhost:8000> --provider=<provider name here> --enable-pending --include-wip-pacts-since=2022-05-01 --provider-app-version=1.1.6 --provider-version-branch=master --consumer-version-selector='{"tag":"master","latest":true}'

Pact file specification version: 2 (the above only verifies

master

for its consumer while I would've hoped it'd verify the

dev-3808

branch mentioned earlier)

I think you want to use branch in your

consumerVersionSelector

for co-ordinated development

--consumer-version-selector='{ "branch": "<branch>", "fallbackBranch": "master" }'

I don’t think you can use

tag

or

latest

, as you aren’t tagging in your consumer, you are using

branches

which is excellent 👼

`tag`: the tag name(s) of the consumer versions to get the pacts for. This field is still supported but it is recommended to use the

branch

in preference now.

`latest`: true. Used in conjuction with the

tag

property. If a

tag

is specified, and

latest

is

true

, then the latest pact for each of the consumers with that tag will be returned. If a

tag

is specified and the latest flag is not set to

true

, all the pacts with the specified tag will be returned. (This might seem a bit weird, but it’s done this way to match the syntax used for the matrix query params. See https://docs.pact.io/selectors).

I may be totally wrong tho, so happy to be corrected 🙂

--consumer-version-selector='{ "branch": "<branch>", "fallbackBranch": "master" }'

I thought the point of WIP pacts was to not have to manually specify what consumer feature branches to verify. Am I incorrect in thinking that?

I’m not 100% clear on pending pact vs work in progress pacts, especially with the move from tags -> branches at the moment, so I will do some reading, get some clarity and get back to you. just for my own clarity, have you got a verified pact between your consumer and provider on the master branches, and these are deployed to an environment? you now are introducing a change on a consumer branch, which you are trying to verify against the providers master build. Which version of the broker are you using? Pactflow or OSS 2.60.0+ Thanks for the questions though, you are in the right place and welcome to the contract testing journey

just for clarity, have you got a verified pact between your consumer and provider on the master branches, and these are deployed to an environment?

Yes

you now are introducing a change on a consumer branch, which you are trying to verify against the providers master build.

Correct

Which version of the broker are you using? Pactflow or OSS 2.60.0+

Pactflow

So I’ve just tested this out, Using this on the consumer side (publishing with branch) • https://github.com/pactflow/example-consumer/blob/3fb6acb42274a29e0f17d3913cf567aa37a73b1d/Makefile#L40 • Updating to use

--branch

◦

@"${PACT_CLI}" publish ${PWD}/pacts --consumer-app-version ${GIT_COMMIT} --branch ${GIT_BRANCH}

leads to not finding anything on the provider side, when specifying

tag

not

branch

. • https://github.com/pactflow/example-provider/blob/master/src/product/product.pact.test.js#L34 ◦

consumerVersionSelectors: [{ tag: 'master', latest: true }, { deployed: true } ]

If both the consumer and provider use

tag

everything matches up, or if they both use

branch

they match up. If I then publish a failing consumer pact on a branch, it won’t be validated until I enable the WIP Pacts feature

enablePending: true,
      includeWipPactsSince: "2022-05-01",

here

So, after enabling

includeWipPactsSince

, that feature branch contract gets verified for you? How odd, I can't see anything you've suggested that is different to what I've done, and yet, it does not work for me 😞

I can see you are publishing with

--branch

but verifying with

tag

I tried it with

branch

in both places just now, too, there is no difference for me.

DEBUG: The pact at <pact url here> is being verified because the pact content belongs to the consumer version matching the following criterion:

* latest version from branch 'master' (<latest version in master here>)

Is the only output regarding branches I get for that consumer.

🤔 hmm yeah not entirely sure there buddy. I’ve just taken the two canonical examples https://github.com/pactflow/example-consumer https://github.com/pactflow/example-provider (this is JS but we do have a Go version https://docs.pactflow.io/docs/examples/golang/provider/) This is the output after switching on the WIP feature, otherwise it wouldn’t find the branch. If you could create a sample GH repo, we could take a look at some point, or if you want to start with those two repos and have a go, you can just fork them, set your env vars and run

make fake_ci

to easily run it from your machine and cross compare.

{
  "_embedded": {
    "pacts": [
      {
        "shortDescription": "latest from branch master",
        "verificationProperties": {
          "pending": false,
          "notices": [
            {
              "when": "before_verification",
              "text": "The pact at <https://saf.pactflow.io/pacts/provider/pactflow-example-provider/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489> is being verified because the pact content belongs to the consumer version matching the following criterion:\n    * latest version from branch 'master' (ac94418+1653314421)"
            },
            {
              "when": "before_verification",
              "text": "This pact has previously been successfully verified by a version of pactflow-example-provider with tag 'master'. If this verification fails, it will fail the build. Read more at <https://docs.pact.io/go/pending>"
            }
          ]
        },
        "_links": {
          "self": {
            "href": "<https://saf.pactflow.io/pacts/provider/pactflow-example-provider/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489/metadata/c1tdW2JdPW1hc3RlciZzW11bbF09dHJ1ZSZzW11bY3ZdPTM5JnA9ZmFsc2U>",
            "name": "Pact between pactflow-example-consumer (ac94418+1653314421) and pactflow-example-provider"
          }
        }
      },
      {
        "shortDescription": "latest from branch wip_pact",
        "verificationProperties": {
          "pending": true,
          "wip": true,
          "notices": [
            {
              "when": "before_verification",
              "text": "The pact at <https://saf.pactflow.io/pacts/provider/pactflow-example-provider/consumer/pactflow-example-consumer/pact-version/b4d89ebc01885acfd726f527182ca47f8abc712c/metadata/d2lwPXRydWU> is being verified because it is a 'work in progress' pact (ie. it is the pact for the latest version of pactflow-example-consumer from branch 'wip_pact' and is still in pending state). Read more at <https://docs.pact.io/go/wip>"
            },
            {
              "when": "before_verification",
              "text": "This pact is in pending state for this version of pactflow-example-provider because a successful verification result for a version of pactflow-example-provider with tag 'master' has not yet been published. If this verification fails, it will not cause the overall build to fail. Read more at <https://docs.pact.io/go/pending>"
            },
            {
              "when": "after_verification:success_true_published_false",
              "text": "This pact is still in pending state for any version of pactflow-example-provider with tag 'master' as the successful verification results with this tag have not yet been published."
            },
            {
              "when": "after_verification:success_false_published_false",
              "text": "This pact is still in pending state for any version of pactflow-example-provider with tag 'master' as a successful verification result with this tag has not yet been published"
            },
            {
              "when": "after_verification:success_true_published_true",
              "text": "This pact is no longer in pending state for any version of pactflow-example-provider with tag 'master', as a successful verification result with this tag has been published. If a verification for a version of pactflow-example-provider with this tag fails in the future, it will fail the build. Read more at <https://docs.pact.io/go/pending>"
            },
            {
              "when": "after_verification:success_false_published_true",
              "text": "This pact is still in pending state for any version of pactflow-example-provider with tag 'master' as a successful verification result with this tag has not yet been published"
            }
          ]
        },
        "_links": {
          "self": {
            "href": "<https://saf.pactflow.io/pacts/provider/pactflow-example-provider/consumer/pactflow-example-consumer/pact-version/b4d89ebc01885acfd726f527182ca47f8abc712c/metadata/dz10cnVl>",
            "name": "Pact between pactflow-example-consumer (520ecb6+1653314859) and pactflow-example-provider"
          }
        }
      }
    ]
  },
  "_links": {
    "self": {
      "href": "<https://saf.pactflow.io/pacts/provider/pactflow-example-provider/for-verification>",
      "title": "Pacts to be verified"
    }
  }
}

What is the difference between WIP and pending? https://docs.pact.io/pact_broker/advanced_topics/wip_pacts#what-is-the-difference-betweeen-wip-and-pending

This ☝️ is excellent Beth 👏

Hiya, sorry for taking so long getting back to you. Somehow my issues disappeared the day after I initially wrote here, and frustratingly, I have no idea why. Started seeing expected behavior with WIP pacts being considered and all that.