Starting to learn about pending pacts and work-in-...
# general
d
Starting to learn about pending pacts and work-in-progress pacts. Pending pacts - don't break the build for new pacts that have not yet passed verification. Work in progress pacts - run verification for the latest pact for any branch which is still pending, and which was not explicitly specified by the provider verification configuration. I was looking at using WIP to manage feature flag variations. But if WIP only runs for pending pacts, how do I make sure that a feature flag pact that previously passed verification will get verified in the future, when the provider changes?
👍 2
m
I missed this. Your summary of WIP/Pending is how I think about it
Good question on feature flags. @Beth (pactflow.io/Pact Broker/pact-ruby) any thoughts?
d
Great, thanks, looking forward to your thoughts @Beth (pactflow.io/Pact Broker/pact-ruby)
Hey @Beth (pactflow.io/Pact Broker/pact-ruby) checking in - any words of wisdom for us? Thanks!
Hey. This is becoming a real issue for us. Are there plans to address this?
👋 1
1
m
Hi David, I’ll raise it with the Pactflow team tomorrow for discussion
I’ve been thinking about this, and am not sure WIP pacts is the best approach. I was thinking one approach could be to treat each consumer+toggle as a separate contract and use the
record-deployment
feature to explictly select consumers deployed to an environment - in this case, that would represent a specific toggle
But that is bad for other reasons - for one, there will be a lot of applications making it hard to reason about
I think toggles might need to be supported as a first class concept
they might be another way though
d
So you would treat each variation of a consumer as a different consumer? I guess it's a workaround but ideally I'd like the Pact model to reflect the domain, which is that the same consumer can have multiple variations. One form of variation is over time, which is represented as versions. The other is over space, represented as feature toggled/configured variations (it may not be just toggles, it could be experiments or customers choosing different features, which I see a lot in the enterprise space). It seems like actually both variations over time and variations over space are both versions. It means that you can actually have multiple versions deployed simultaneously as well as released simultaneously
If you allow for multiple versions to be deployed to the same environment simultaneously via
record-deployment
and then have the verifications take that into account just as with releases, we'd have what we want
m
Yes, that is true and a good point
This might model more onto the
release
concept, originally conceived to support mobile use case (i.e. you can have multiple versions of an application released to market at any point at time)
👍 1
d
You might argue that the distinction between release and deploy starts getting a little fuzzy or even non-existent. Sorry, that's a tough thing to say when you put so much work into it.
m
yes
d
We're still using tags because the JVM doesn't support deployments and releases. Based on my statement that feature toggles are different versions of the same consumer that can be deployed on the same environment at the same time, here's what I think could work. But I'm not sure if the current Pact libraries support this... • When version a pacticipant, the version identifier includes the feature toggle name as well as the git hash • When we verify a provider, we have to verify all versions of that provider with the same git hash (e.g. all active toggles) and publish each one as a separate verification • When we run consumer tests, we have to run for all versions of the provider with the same git hash and publish each one separately. Assuming the test passes, we publish the pact and tag with the branch name. • When we deploy a pacticipant to an environment, we tag all versions that have the same git hash, regardless of the feature toggle • When we run can-i-deploy for a pacticipant to a given environment, we need to check all versions tagged with that environment, not just the latest (not sure if we can do this. I don't see an
--all
option)
m
actually, the language I used consulting was that a “deployment” was when the code was shipped to an environment, and it was “released” when made available to a customer (i.e. just a step)
d
I often say you deploy code and you release features (when you turn the toggle on)
👍 1
Which calls out that even if a version of an application supports a toggle, it's not released to an environment until the toggle is turned on
👍 1
You might even say that really
can-i-deploy
should be
can-i-release
😎 1
And runs not when the code is deployed but when the toggle is getting turned on
If you don't use feature toggles, then those two concepts happen simultaneously, but they are still separate concepts
And of course, once a toggle is on, then every deploy has to continue to support that toggle's contract
m
yeah. I guess the challenge here is that the terms/language needs to work across concepts that have subtle variations (branches, toggles, …). So it might be hard to put that genie back into the bottle.
d
Yea, you discover your domain as you implement it, and of course things also evolve, but now you're stuck with a model that doesn't correctly represent the domain. That can create a mismatch that is hard to work with. It's a bummer.
👍 1
I can imagine first defining best practices working with what you've got, and then maybe building tooling that is backward-compatible with the existing system but better aligns with the domain. Not saying that's easy, just spitballing here
🤔 1
For example I described a possible approach using tags above. I believe it needs more refinement, but let's say I get it working. I can imagine building scripts that layer over the Pact API that do what I want for feature toggles so each team doesn't have to handle all the tricky details. That's sort of what I meant.
👍 1
m
BTW we’ll be picking up the feature branch/environments etc. work in the Pactflow backlog soon to drive implementation across the ecosystem. We kicked off some planning late last week for that. Just thinking out loud, I wonder if this would be a better approach than WIP for you. As you describe above, you simply tag all variations of your feature variations (versions) with a tag. On the provider side, you can take advantage of the
latest
selector set to
false
. This would have the effect of always verifying all versions with that tag. You would need to remove the tag from the version afterwards once undeployed, which is the messy bit. I think releases is probably going to be the better tool once you can take advantage of it. From that page: • `latest`: true. Used in conjuction with the
tag
property. If a
tag
is specified, and
latest
is
true
, then the latest pact for each of the consumers with that tag will be returned. If a
tag
is specified and the latest flag is not set to
true
, all the pacts with the specified tag will be returned. (This might seem a bit weird, but it’s done this way to match the syntax used for the matrix query params. See https://docs.pact.io/selectors).
(I have to duck off to a meeting, but have popped a reminder in my calendar to discuss with the team tomorrow)
d
ok duck away, talk more
I was thinking something like what you describe. But, I don't want to tag an environment with a toggle until it's enabled. So I'd want to automate applying tags when I roll out toggles. Will play around with this and share what I learn when I learn it
👍 1
By the way, the eBay team is looking for guidance on how to work with feature toggles, so I am going to write something up. I realized, I should just write it up for PactFlow docs, and then point the eBay folks there. So expect another PR in the next few days 🙂. I know it would be better to get one done at a time, but there is a lot of pull for feature toggle guidance... My guidance will be using tags, rather than deploys and releases, since we can't use those yet. Is that OK?
🙏 1
party parrot 2
Making good progress on this doc. One area where I feel my understanding is a bit fuzzy is what happens when you publish a new version of a pact and kick off a webhook to run provider verification. I get how you can indicate which pact version needs to be verified. But how do you know what provider version to use when running the verification. We clearly don't want to run verification for all provider versions. So which ones are we supposed to run verification for? Let's say we deploy a provider to staging, preprod and prod. Should we find out what version is active for each of those environments, check out that version, and run verification? Now let's add feature toggles onto this. Now staging, preprod and prod will have multiple provider versions. So instead of running three tests we run six or nine or twelve or... Am I missing something? Thanks!
b
@David Van Couvering (DVC) - eBay sorry, I've only just seen this thread and it looks like Matt has answered a lot of you questions. What is still outstanding that I can help with?
But how do you know what provider version to use when running the verification. We clearly don't want to run verification for all provider versions. So which ones are we supposed to run verification for?
if you use the new webhook, it chooses the versions for you.
d
The new webhook - do tell? I wasn't aware of a new webhook
b
it verifies against the main branch and all currently deployed/released versions.
well "new" last year
Should we find out what version is active for each of those environments, check out that version, and run verification?
yes, it does exactly that
de-duplicated
d
Well, the challenge is not finding which pacts to verify against. The challenge is you have to actually build, start up and run different versions of the provider, one for each version that is deployed or released
b
yup
it kicks off a build for each one
d
Ah...
b
with the git sha of each provider.
d
I have to run, my son is ready to watch TV with me, but I'll take a look thanks!
b
👍🏼
there are still some improvments to be made to that workflow, but it's a big incremement to what we had before.