I'm working with a client on their contract testin...
# general
b
I'm working with a client on their contract testing implementation, and we're currently doing CDCT with Pact. I could use some views on how and where to make the consumer and provider test steps part of the respective pipelines. Services deploy to a DEV environment multiple times a day through a PR build process. At some point, a release candidate is put together for deployment into a QA environment. Not every PR leads to a new artefact version, hell, most of them don't (i.e., the version in the pom.xml), and I'm not even entirely sure that every RC does (but I'm checking that at the moment). I would love to see both consumer contract generation and publishing tests, as well as provider verification tests, run as often as possible, i.e., on every PR build, no matter what the change is. The earlier we can catch integration issues, the better.. However, I'm not sure if it is possible to publish changes to a contract without an artefact version bump? If that's not possible, I think we need to start having a serious talk with the various dev teams to get their versioning in order.. I'd love to hear some experiences from others about the frequency with which they run their contract tests, what that brought them and what things they had to keep in mind. How much do you really shift to the left with your contract tests?
šŸ‘‹ 1
t
This versioning problem is actually exactly the use case that I wrote absolute-version to solve - it allows you to get between-release version numbers that are both human readable and correct. (some teams just use the git sha, but I found that less nice because they're not really human readable)
ā˜ļø 4
It currently works off git tags, but it would be easy to extend it to read pom.xml, especially if that would be useful to you.
What I usually do is: • Generate version numbers with absolute-version • Every commit is contract tested • Every commit is deployed to dev • Every release is tagged and deployed to <other environments>, assuming that the can-i-deploy passes
ā˜ļø 1
m
However, I’m not sure if it is possible to publish changes to a contract without an artefact version bump? If that’s not possible, I think we need to start having a serious talk with the various dev teams to get their versioning in order..
šŸ˜†
Using the git commit SHA as the contract version is definitely the easiest first step, but Tim’s absolute version utility is a big improvement.
The issue here is the usual POM versioning strategy is semver or ā€œthis is the thing we are working on and then will releaseā€ version vs what Pact needs - which is a unique identifier per code change (which is why the git SHA is a good default).
I’d love to hear some experiences from others about the frequency with which they run their contract tests, what that brought them and what things they had to keep in mind. How much do you really shift to the left with your contract tests?
it should really be every single build, and also run on dev machines prior to commit. In the case of the provider, it should be tested against a locally running version of the software with external dependencies stubbed out (i.e. a so called ā€œhermetic serverā€) (image taken from https://docs.pact.io/getting_started/testing-scope)
šŸ’Æ 1
ā˜ļø 1
šŸ™Œ 1
In some organisations, the provider API is run on technology stacks that can’t be run locally for various reasons (e.g. vintage technology like IBM WebSphere or large runtimes that are just not practical). In those instances, you’ll need to run the Pact tests against a test environment - this is a bit harder to manage and provide general advice for, but the general rule would be ā€œas often as you canā€ :)
b
Thanks @Matt (pactflow.io / pact-js / pact-go) and @Timothy Jones, really appreciate your input. Running the tests (both consumer and provider parts) locally isn't the issue, we're doing that already (yay for Spring and mocking external dependencies). The versioning is the biggest challenge here. I'll bring that input back to the teams. Thanks again!
šŸ™Œ 1
m
Ah perfect! then you've already solved the hard technical problems, this should be the easy bit (famous last words)
b
Yeah, because convincing others to change their ways of working has always been the easy part of consulting šŸ˜‰ I'll keep you posted.
šŸ˜† 2
t
My original plan was to have absolute-version available natively for the JVM too - so if you would find it useful, but don't want a node tool, let me know what the needs are and I'm happy to build it (it's not a substantial amount of work)
b
Thanks @Timothy Jones for the offer, I really appreciate it. You're right in that I prefer not to introduce a NodeJS tool (or rather, the runtime itself), but I'm going to try and let them use commit hashes first to make them feel the pain of that, hopefully that'll lead to them doing better on their versioning šŸ™‚
I might ask you again about this at some point, though!
šŸ‘ 1
Hey @Matt (pactflow.io / pact-js / pact-go) coming back to this, how can I easily add the commit hash to the contract version number? It looks like that's the way to go for now.. But I can't find how to in the docs. Also, can you confirm that the hash performed to see if a contract has been updated in the broker does not include the version number? I.e., the contract isn't actually seen as 'new' unless some expectation changes?
Keep in mind we're using the
pact-jvm-provider-maven
plugin to publish consumer contracts
t
The version number isn’t in the contract file, so it’s not considered for the hash
I don’t know the answer to the other part of your question though, sorry
b
Ah I think I got it...
<projectVersion>${project.version}-${git.shorthash}</projectVersion>
t
That… looks like maven, yes šŸ˜…
Thanks @Timothy Jones!
t
Welcome!
b
Now to test if it actually works but for that I need to wait until the ops people provide me with a working broker šŸ™‚
šŸ™Œ 1
t
The contract hash in the broker is a straight up hash of the contract file. The broker is actually pact agnostic- you could use it for any other file
b
Oh that's nice to know!
t
I can recommend pactflow for that in good conscience, because I don’t work for them. But my experience is that it is definitely easier than maintaining your own.
(Of course, your situation may be different)
b
I'd be 100% in favour of Pactflow, too, I recommended my client using that but for reasons we're going the OSS route for now
ā¤ļø 1
(long story)
Hmm.. `${git.shorthash}`isn't resolved so that is appended to the contract version literally.. Not exactly what I was looking for!
Is that a plugin thing?
I've found this but the shortcodes are different: https://github.com/git-commit-id/git-commit-id-maven-plugin
@Matt (pactflow.io / pact-js / pact-go) do you happen to know how I can get Pact to recognize that
${git.shorthash}
shortcode as seen here (under Publishing Pact files to a Pact broker)?
m
I have no idea sorry Bas, but I reckon @uglyog might. Ron, any ideas? That being said, I like to encourage use of the Pact CLI if you can: https://docs.pact.io/implementation_guides/cli#pact-broker-cli Maven is….not fun
u
Just reading the comments. I think either Maven doesn't replace the expression if the system property is not defined, or it does not support expressions in that form. The best thing to do would be pass it on the command line (i.e. add
-Dpact.projectVersion=...
)
Oh, I see your issue. You've copied the example. That assumes something is setting
git.shorthash
system property. That is not the default Maven behaviour (Maven knows nothing about Git)
šŸ’” 1
Try adding
-Dgit.shorthash=...
to your command line (replacing the
...
with the actual Git SHA, of course)
m
Ah! That makes sense! Thanks Ron. So basically, define outside of maven however you wish (Bash, Powershell, scripts) and pass as a system property. Does Maven support environment variables in the same places as system props?
u
Does Maven support environment variables in the same places as system props?
Please refer any Maven questions to the Maven people
šŸ˜† 1
t
https://github.com/git-commit-id/git-commit-id-maven-plugin <-- you can use this to get
git describe
output, which is very similar to what absolute-version does (the main difference is that absolute-version is semver compatible)
šŸ‘ 1
b
Cheers folks! I thought it maybe was a feature I didn't know of. Looks like I'll have to take a closer look at that plugin (or see how to pass it through the CLI). I'd like to stick to Maven (yes, I said that) because so much is managed through Maven in this org I'm working with, probably easier to educate people that way.
t
I'd like to stick to Maven (yes, I said that)
You don't have to explain yourself to us šŸ˜‰
šŸ˜† 2
m
Cheers folks! I thought it maybe was a feature I didn’t know of. Looks like I’ll have to take a closer look at that plugin (or see how to pass it through the CLI). I’d like to stick to Maven (yes, I said that) because so much is managed through Maven in this org I’m working with, probably easier to educate people that way.
No judgement 😬
🤣 1