Hey hey, I am trying to figure out an issue we're seeing on the provider verification side when it is invoked by a webhook. We have 2 providers in one repository and when the pact url is specified and we run that verifier flow, it's executing against more than the single provider the pact specifies within the consumer contract. I would assume that if a pact url is specified but the provider does not match my providers (provider-a) verification that it will ignore any pacts desinted for provider-b. Is there another step required to filter the provider verification tests to only run against the provider the pact is for?

what parameters are you passing when doing a verification by pact url? you should be omitting any consumer version selectors, enabling pending and wip pacts

const failIfNoPactsFound = false

  const coreOptions: PactMessageProviderOptions = {
    messageProviders: options.messageProviders,
    publishVerificationResult: options.pipelineExecution,
    provider: options.provider,
    pactBrokerUsername: options.authentication.username,
    pactBrokerPassword: options.authentication.password,
    providerVersion: options.version,
    providerVersionBranch: options.branch,
    logLevel: options.logLevel || 'error',
    failIfNoPactsFound,
  }

  const buildOptions = {
    pactBrokerUrl: config.pactBrokerUrl,
    enablePending: true,
    consumerVersionSelectors: [
      {
        mainBranch: true,
      },
      { deployedOrReleased: true },
      { matchingBranch: true },
    ],
    includeWipPactsSince: options.workInProgressPactsSince,
  }

  const verifierOptions = options.pactUrl
    ? { ...coreOptions, pactUrls: [options.pactUrl], ...customOptions }
    : { ...coreOptions, ...buildOptions, ...customOptions }

  const verifier = new MessageProviderPact(verifierOptions)

  return verifier.verify()

So as the pactUrl is specified, it omits the

buildOptions

therefore it doesn't verify any pending pacts, wip pacts, or have any consumer version selectors. The verification when the provider is being changed and ensuring it's not breaking any existing pacts is great, it's when the pact url is specified and there are multiple providers in this one repository.

I don’t believe it will consider the

provider

option and instead verify whatever is in the url, when doing an url based verification it isn’t acting as a filter, as you are expecting, I would need to check the source. there is a consumer filter

pactffi_verifier_set_consumer_filters

where you can filter consumers, again not sure if that takes effect if the pact url is set, or just on dynamically fetched pacts, we would need to check the pact-js verifier code and the rust verifier code.

so when the pact url is used, I assume this is being invoked by a webhook?

Yes. Each provider in this single repository has its own webhook. This URL is an example of the PACT_URL being provided by the webhook invoked triggerd https://pact-broker.localhost.io/pacts/provider/provider-a/consumer/consumer-a/pact-versi[…]3e074effa27e9192f566/metadata/Y3ZuPWNiZGNjZmEyJnc9dHJ1ZQ.

Are you receiving one single webhook?

Yes, each provider in this repository has its own webhook defined and will specify in the url the provider it is so provider-a or provider-b. I am receiving 1 webhook triggered pipeline with the above url. So no mention anywhere in the pipeline being triggered of

provider-b

.

provider opt of

provider-b

and your expectation is that it shouldn’t run a test?

Yes exactly that. The above url is being specified and running the 'webhook verify flow' in our pipeline. It is running each of the unique provider verification test files in the repo with the pact url. The provider that has a different matching name than what's in that URL (i.e. url says provider-a but provider-b is executing too) or within the contract file itself is being executed and failing. `provider-a`verification is running as expected and passing but

provider-b

is also being executed and failing.

how do you isolate the verifications between the two separate providers, that reside in the same codebase are they using the same file?

So they are two completely separate verification test files that use the options specified https://pact-foundation.slack.com/archives/C9VBGLUM9/p1689001131963479?thread_ts=1689000821.597169&cid=C9VBGLUM9 and the only major difference is the provider name and any messageProviders/stateHandlers. Unless I am missing a simple way to support multiple providers in the same project each with their own unique webhook (1 webhook per provider) then I think there may be only one solution. When the pact url is specified there needs to be manual filtering of which provider verification test files to run as it doesn't look like the pact url provider or the provider within the consumer contract are checked when the provider is ran.

ahh so any time one of two webhooks fire, both verification jobs fire, because they hit a single pipeline task. Gotcha. So for now I would use a webhook param to identify the provider name and only trigger the relevant one of the two verification jobs. For it pact side - Would probably need to get the thinking hat on as to what the user facing behaviour would or should be the case if. you provide a pact url for a provider, but specify a different provider • should the test fail, pass or warn? its quite an assumption that only one of the three would be relevant as people may have separate use cases, some may want to know as they may have a misconfigured webhook.

Hmmm yes. Thanks, I'll solve it for my company but I'd be interested to see if it could progress somehow in to the pact verification flow itself. However it works, if the contract specified provider != the provider being ran, it should have a flow that doesn't execute the incorrect expectations on a different provider 😄

I feel like this is a bug in the pact core. If you get given a contract file that is not for you, you should warn and ignore it I reckon

Yeah, it’s probably just not a scenario that has been considered. The

provider

name was previously only used as a selector to discover pacts in the broker, and not for much else

In 2000, everything was a monorepo. We have come full circle

haha

I’m not sure that filter is the right fit, because filtered results (shouldn’t) be publishable

that would be my assumption too, that filtered shouldn’t be publishable. I’d probably start by testing both the rust and ruby core via the cli, as to what happens in this use case (with provider name diff to that in a file), both with a local file and remote file and see if that makes any diff

that would be assumption too, that filtered shouldn’t be publishable.

It definitely shouldn’t, because if you filter the contract, you’re not adhering to the expectations of the consumer.

filtered consumer imo should be okay, filtered desc not so okay

I think they’re different kinds of filter.

yessum, one being a filter of contracts, one being a filter of interactions within contracts.