https://pact.io logo
Join Slack
Powered by
Hi All, In our pact scenario we have multiple serv...
# pactflow
c
Hi All, In our pact scenario we have multiple services and one of them (ServiceC) is both provider and consumer. We keep running into the following error when trying to run the pipeline for ServiceC: The cross contract comparison between the pact for the version of ServiceA currently in dev (610554d ******************) and the oas for version b461002***************** of ServiceC is currently being executed. Enable polling for the can-i-deploy command to retry until all results are available. See https://docs.pactflow.io/docs/cid-polling-bdc for details. ##[error]PowerShell exited with code '1'. Usually the error resolves itself by re-running the pipeline for ServiceC without running any other pipeline. For example last time this error happened, we had the following situation in Pact logs: Computer says no ¯\_(ツ)_/¯ CONSUMER |C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT# -----------------------|----------|-----------------------|------------|----------|-------- ServiceA | 610554d... | ServiceC | b461002... | ??? | ServiceC | b461002... | ServiceB | 3bce4a0... | true | 1 ServiceC | b461002... | ServiceD | 3bce4a0... | true | 2 After the re-run of ServiceC pipeline can-i-deploy succeeded and we had the following situation in Pact logs: Computer says yes \o/ CONSUMER |C.VERSION | PROVIDER | P.VERSION | SUCCESS? | RESULT# -----------------------|----------|-----------------------|------------|----------|-------- Admin | 610554d... | ServiceC | b461002... | true | 1 ServiceC | b461002... | ServiceB | 3bce4a0... | true | 2 ServiceC | b461002... | ServiceD | 3bce4a0... | true | 3 Any hints what causes this or how we can debug it?
m
Have you read the output above? It literally has the solution printed in it
c
Yes and we also have that pulling option activated but I've been asked by other team members why is this required and what causes this. Which results that can-i-deploy requires are not ready? The results of the other pacticipant? In our case when this happens the pipeline for the other pacticipant was executed long time before that.
m
right
so basically, when you use bi-directional contract testing, PactFlow generates comparisons for the provider contract against common versions (
mainBranch
, 
deployedOrReleased
etc.) of the consumer. This happens when a new consumer or provider contract is pushed. These have to happen asynchronously and within PactFlow, otherwise the publish command would be held up. In these cases, you would get this result. This is not a problem for pure Pact testing, because the verification is done when the provider runs a build, so the result is either not published or published - never in between
c
Does it mean that the more branches and environments we have (those that can be seen in the matrix view), the longer it takes for the results to become ready? We do cleanup our environments ourselves but we don't care about the branches. Do we have to cleanup the branches that no longer exist? Could this be the reason why we experience a lot of retrials during can-i-deploy? Our polling is currently set like this: --retry-while-unknown 3 --retry-interval 10 But the polling does not seem to be enough because often times we have to re-run our deploy pipeline due to failed can-i-deploy because the results are not ready even after the polling attempts.
m
I don't think all branches are included, no. The size of the pact files and OAS will be the main contributor. If you have many released versions that could be a part of it. 30s is a short window though given some of these comparisons might be queued. Some comparisons may take multiple or 10s of seconds each
c
In our case we have pact configured for 4 services only. And the most complex of them has an OAS with like 20 methods. Some of our team members express the concern, that if we run into performance issues already, with the few services and methods we have at this early stage of our project, how will those performance issues grow if we have like dozens of services and more complex OAS files. Right now, one of our services requires around 30 or more seconds to have its results ready for can-i-deploy. The concern is how this time might increase as our system becomes more complex, will we be faced with delays of minutes or tens of minutes? Also, you mentioned that the many released versions might contribute to our performance issues. Is there any strategy or approach to reduce the number of released versions inside Pactflow to counter the performance costs caused by them?
m
So if you are using 
record-deployment
then you are not using the releases - best to check if you are. Releases are for things like mobile, where you might have multiple versions live in the wild. Most of the time, you wouldn’t need this. See https://docs.pact.io/pact_broker/recording_deployments_and_releases
The concern is how this time might increase as our system becomes more complex, will we be faced with delays of minutes or tens of minutes?
I understand, I’d need to check with the dev team to fully understand if this is something to be worried about. How is your pipeline currently configured? At what point does the OAS get published? Perhaps a simple fix is to move it earlier into the pipeline, it sounds like you publish the OAS and immediately check 
can-i-deploy
.
c
> So if you are using 
record-deployment
then you are not using the releases - Yes, we are using record-deployment, so we shouldn’t have those released versions. > How is your pipeline currently configured? At what point does the OAS get published? We have two pipelines for each service. A build and a deployment pipeline. The publishing of the provider and/or consumer OAS/pact file happens in the build pipeline shortly after the actual build. The can-i-deploy step happens in the deployment pipeline that is triggered immediately after the build pipeline is done. So can-i-deploy is executed, depending on the service, around maybe 30..40 seconds after the publishing command. > Perhaps a simple fix is to move it earlier into the pipeline, it sounds like you publish the OAS and immediately check 
can-i-deploy
. The publishing command is executed around 15..20 seconds after the actual build, so maybe there is indeed some potential to place it a bit earlier and closer to the actual build. I will try to bring up this idea in the team. > I don't think all branches are included, no. What I see is that whenever we publish a new OAS version, there are new entries in the matrix view for many branches, many of them no longer existing in our DevOps repository. So, it seems that we have some old branches around in the Pactflow that cause unnecessary compatibility checks? The question is how are old branches supposed to disappear from Pactflow? Or are they supposed to remain there for recording reasons? And if they remain there won’t they cause more and more compatibility checks whenever new pact or OAS files are published?
m
What I see is that whenever we publish a new OAS version, there are new entries in the matrix view for many branches, many of them no longer existing in our DevOps repository. So, it seems that we have some old branches around in the Pactflow that cause unnecessary compatibility checks? The question is how are old branches supposed to disappear from Pactflow? Or are they supposed to remain there for recording reasons? And if they remain there won’t they cause more and more compatibility checks whenever new pact or OAS files are published? (edited)
I’ll follow this up. There is a new endpoint that can be used to delete old branches, perhaps that could be something to look into. Let me get back to you in a few days.
OK back on this - you’re right, we do actually check branches. This could explain the behaviour
If you used Github (or an SCM with hooks), something like this will work to clean up old branches from PactFlow:
Copy code
name: Clean up application branch in Pactflow

on:
  delete:
    branches:
      - "*"

jobs:
  delete-branch-in-pactflow:
    name: delete
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - uses: ruby/setup-ruby@v1

      - name: Clean up branch for application pactflow-application-saas in Pactflow
        run: script/ci/clean-up-pactflow-branch.sh
        env:
          PACTFLOW_TOKEN: ${{ secrets.PACTFLOW_TOKEN }}
          GIT_BRANCH: ${{ github.event.ref }}
The script (needs better error handling!):
Copy code
#/bin/bash

set -Eeuxo pipefail

APPLICATION="your-app"
PACTFLOW_URL="<https://YOUR_TENANT.pactflow.io>"

echo "Deleting branch ${GIT_BRANCH} for ${APPLICATION} in Pactflow"
ENCODED_GIT_BRANCH=$(echo "$GIT_BRANCH" | ruby -e "require 'erb'; puts ERB::Util.url_encode(ARGF.read.chomp)")
BRANCH_URL="${PACTFLOW_URL}/pacticipants/${APPLICATION}/branches/${ENCODED_GIT_BRANCH}"
curl -X DELETE $BRANCH_URL -H "Authorization: Bearer ${PACTFLOW_TOKEN}"
When the CLI is available, you could use that in the script instead
c
Thank you for your additional input. Our system has been stable after adjusting the retry parameters so far. If we encounter those errors again, I will try your latest suggestions.
🙌 1
m
We are also releasing changes this week that should speed up this process
👍 1
2 Views
Open in Slack
PreviousNext
Powered by