i had a question that I think I know the answer to...
# general
j
i had a question that I think I know the answer to but I'm not 100% sure.. so I'll ask it here and then provide what I think the answer is and if someone can correct me if any of that is wrong? Question What's the point of using pact if you use openapi.json files that are validated to be correct and/or are used to generate the stubs/code and you can guarantee to never introduce breaking changes (ie you follow semver.. any new MAJOR version results in a new URL (/my-api/v1, /my-api/v2).. Answer There is little to no benefit in using pact. with openapi.json as the source of truth, and a tool like optic which is able to compare the current openapi.json with the previous openapi.json and tell you if you are making any breaking changes, and a guarantee to never introduce breaking changes, you are able to release new APIs without breaking consumers, you can monitor logs (watch for the User-Agent field to work out who the consumer is... ie which service is the consumer that is still using the old version - so you know who to talk to) and you have basically no need to use pact in terms of consumer/producer validations/checks
in fact, wouldnt monitoring logs be a more accurate way to determine if someone is still using an old version? when using pact, could you end up in a situation where a repo has submitted pacts as the consumer 2 years ago, has had no updates, and in fact has been abandoned and is no longer used - but pact reports it as a consumer that you need to make sure upgrades?
b
Is
openapi.json
a schema thing? Because schemas aren't contracts, so you'd lot be missing coverage and reasoning power.
j
i would say semantic versioning and guaranteeing never to introduce breaking changes (if you do, thats a new URL) combined with openapi as the definition of a contract dont forget: you're reading a blog post about pact from pact themselves.. i've found little info about pact from anyone else other than pact.. seems a bit bias
b
You didn't really answer the question, but if you have some way of guaranteeing those things, then I think you probably don't need Pact? I'm not aware of any tooling or systems that do, so it'd be great if you could share some 🙏
this 1
As an aside, that blog post is inspired by a bunch of community contributions, that became so commonly repeated that they were extracted out as a reference document. It feels a bit ad hommy to dismiss them as propaganda.
j
sorry, what didnt i answer? i've never heard of openapi.json discussed as a "schema thing" .. so i guess i couldnt really provide an answer to that question?
in terms of the tooling im using
the app is .net, im using microsoft package validation to ensure that the public interfaces (ie public classes, methods, properties, etc) cannot be changed unless the major version number of the application is incremented.. and those public interfaces are being used to expose the web api
the openapi.json is written manually in pieces (ie each path / etc is its own file.. rather than trying to manage it all in a single big file) redocly is used to combine that into a single file
optic is used to guarantee the openapi.json does not change in a breaking way without a major version increment
and the testing is done using a client generated from the openapi.json (so as so as a field is changed in the openapi.json, if that field also isnt changed in the implementation then the tests fail and it cant be deployed)
m
They are all reasonable things to do. Ultimately, however, these are still static schemas we are talking about. In the linked article, the following points would still be valid concerns: 2, 3, 6 (and possibly others, I’m being generous 😛 ). The other point to make - which I should probably add to that article - is that building APIs this way is a pain. Who wants to maintain several major versions of an API just because one day you decided to rename a field? If you know that field isn’t used by anyone, just remove it and move on. If you adopt the “no breaking (schema) changes” model, you are forced to support multiple versions of an API with all of the support/maintenance that goes with it
b
(when I search for "openapi.json" I don't get any exact matches, but I think it's OAS (OpenAPI Spec), which answers it 👍 it's spec by schema, not by example)
👍 1
y
you can guarantee to never introduce breaking changes (ie you follow semver.. any new MAJOR version results in a new URL (/my-api/v1, /my-api/v2)..
You now have to maintain individual endpints
are used to generate the stubs/code
You don’t know exactly which fields a consumer needs to perform its function, so you have to assume your entire surface area of your api is in use all the time (which is rank)
i would say semantic versioning and guaranteeing never to introduce breaking changes (if you do, thats a new URL) combined with openapi as the definition of a contract
That is alot of onus on your developers, plus all you ensure if you never remove fields, leading to bloated responses or you have two versioned apis which is also nasty
you are able to release new APIs without breaking consumers, you can monitor logs (watch for the User-Agent field to work out who the consumer is... ie which service is the consumer that is still using the old version - so you know who to talk to)
a user agent can only tell you who called what endpoint, not what they need from that endpoint plus you can only check this post deployment, pact affords you this pre-deployment
and the testing is done using a client generated from the openapi.json (so as so as a field is changed in the openapi.json, if that field also isnt changed in the implementation then the tests fail and it cant be deployed)
Yes but a client may never use any of the fields (they may only use one) - adding an extra field shouldn’t break your client, if they don’t use it. Removing a field, will cause the client team to realise they can’t perform what they need to, and then they have to discuss it with the provider team
optic is used to guarantee the openapi.json does not change in a breaking way without a major version increment
This will force you into bumping your versions when in reality no-one might be using that field and therefore you can safely remove without a major version bump
You didn’t really answer the question, but if you have some way of guaranteeing those things, then I think you probably don’t need Pact? I’m not aware of any tooling or systems that do, so it’d be great if you could share some 🙏
This is a good point. If you have other mechanisms that provide you confidence, fair enough, but that doesn’t discount other mechanisms.
There is little to no benefit in using pact. with openapi.json as the source of truth, and a tool like optic which is able to compare the current openapi.json with the previous openapi.json
I completely disagree, Pact will provide you field level visibility pre-deployment and allow you to make better decisions about your versioning strategy for your providing api, hopefully mitigating the requirement to maintain multiple code paths
I think a combination of spec-first design, which aims to look at provider drift (provider impl drifting from spec) coupled with consumer driven contract testing to provide field level usage to providers, is incredibly valuable. Augmenting it with a full Pact provider verification, whereby the contracts are replayed against your provider (pre deployment) affords you the strongest guarantees
j
That is alot of onus on your developers, plus all you ensure if you never remove fields, leading to bloated responses
actually: they do remove fields.. the reason you can guaranteeing never to introduce breaking change is due to the tooling because you are made aware of breaking changes, and forced to increment MAJOR - which also brings with it a new URL (major version number are also in the URL - so that people have to explicitly upgrade to the next breaking version) - you never break any existing client so this is covered in 2 ways: 1. optic reports on breaking changes as visible in the openapi.json files 2. .net package validation reports on breaking change as visible in all public classes, methods, properties, etc so you are forced to follow semantic versioning every single time. there is no exception, there is nothing to consider.. you dont need to wonder if someone is using that field and only increment the major version number when someone is (that would appear inconsistent to people outside your team.. when you delete a field, sometimes you increment the major version and sometimes you dont?) .. you follow the same way of incrementing version numbers all the time and you dont really need to think about it
Removing a field, will cause the client team to realise they can’t perform what they need to, and then they have to discuss it with the provider team
i think the only time anyone talks with anyone else is: when as a provider you have a previous major version still running because consumer ABC still hasnt migrated to the latest major version.. this is one thing im really struggling with: why do consumers dictate what a provider can do and cause some changes a provider tries to make as not possible?? you're the provider (one), there could be hundreds of consumers - if you want to remove a field from your code, your api, and you're fine to continue keeping the previous major version running to allow people to upgrade - consumers shouldnt have any say in that / how you do that.. yes, i get it: if you just go and remove a field out of the blue, you will break consumers.. in this instance where someone isn't following semantic versioning and is not keeping the previous major version running while people migrate: pact makes sense..
seperate-ish question: can you be in a situation where a consumer application that hasnt had a change to it in 4 years but as a provider you believe they are still using field XYZ (because pact is telling you).. before removing XYZ, you try and get in touch with that consumer to discover the team no longer exists, the app is no longer running anywhere in your environment and you've just gone on a wild goose chase?
Augmenting it with a full Pact provider verification, whereby the contracts are replayed against your provider (pre deployment) affords you the strongest guarantees
if you know for a fact that your endpoint will always work for a particular set of fields/etc (because if it doesnt: that update is pushed to a new URL version), this isnt needed, is it? the consumers have confidence in the fact they will not break
y
seperate-ish question: can you be in a situation where a consumer application that hasnt had a change to it in 4 years but as a provider you believe they are still using field XYZ (because pact is telling you).. before removing XYZ, you try and get in touch with that consumer to discover the team no longer exists, the app is no longer running anywhere in your environment and you’ve just gone on a wild goose chase? (edited)
Theoretically yes. The broker isn’t aware about the states of your application, beyond what you tell it. The onus would be on the consumer team, to record-release-ended or record-undeployment of their application from an environment and at minimum delete associated pacts from their main branch. If the team creating the pacts disbands, and the application is removed from an env, the above should be adhered to.
> if you know for a fact that your endpoint will always work for a particular set of fields/etc (because if it doesnt: that update is pushed to a new URL version), this isnt needed, is it? > > the consumers have confidence in the fact they will not break Your clients now need to upgrade their applications, to point to the new url. and you have to maintain that old url for as long as people are using it. You claim you are going to ascertain that from logs (post deployment)
There is certainly value in doing some instrumentation of code, such that you could get some runtime information about consumers/providers and utilise that to understand where gaps existing in your contract coverage. CT doesn’t exist in a bubble, it complements many other development activities. If you’ve found what works for you team, that is great, we aren’t trying to sell you something you don’t need. Many people do find value, without us needing to show them it, but not everyone will be converted, that is fine