Hey Everyone, I am a QA at my company and new to Pact. Our company uses a microservice environment. Apparently, they tried to implement Pact in the past and it didn't catch on. I wanted to research a bit more to see if anything has changed. Here were some of the concerns that someone listed out for me from my company and I was hoping someone could help me understand if they are valid concerns still: • its consumer-oriented design being inconvenient to work with • Having services written in ruby, python, javascript and golang adds additional complexity. I believe there are pact libraries available for all of those languages, but I don't know anything about the non-ruby ones. • It's not well-suited to APIs where we don't control both ends of the implementation, e.g. APIs we provide to clients

I'm an SDET building my company's pact implementation from scratch, and so far my experience with it has been: • The consumer-oriented design is very convenient to work with, because pact effectively produces a contract automatically from the way I use mocks anyways in my consumer unit tests, and this ensures my contract only specifies the things my consumer actually cares about, leaving the provider full leeway in modifying things that are not yet in use. (I was aware of this criticism myself before getting into Pact and I don't really understand it other than maybe people who didn't understand the best-practices behind consumer-oriented contract testing were just doing it wrong.) • Pact, in my experience so far, seems very language independent. There are lots of neat libraries for different languages for building up the contract automatically in your consumer, but I've found that outside of that, I've been using the CLI tools for everything else: for uploading and verifying pacts. • This is true, it is not designed for APIs where you don't control both ends, but my company (as many likely do) uses a reverse-proxy front-end with all our public apis. By writing consumer contract tests into this, we can effectively test our public apis, since our reverse-proxy service does very little logic and passes-through public requests to the back-end services.

Thank you, thank you! the more im researching, the more I like it, but it's very difficult to convince people who are hesitant to try something new! I need to really look at their concerns (as you are helping me to do) to figure out if they are valid or just knee jerk reactions

I think one of the tricks to helping people understand the consumer-oriented nature of it is that most people have this impression that that means you make a change to the consumer, and it changes the contract, and then the next time you try and build the provider it breaks because of a mistake the consumer made. This is what I mean about people doing it wrong though. Ideally, if you're following something like gitflow, you make all your changes in branches, test those branches, and only if the tests pass can you merge them. Well, by the same philosophy, you block merges to both consumers and providers if either breaks the contract between them. So even though the consumer defines the contract, it still has to verify the contract against the provider's master branch before it can be merged to the consumer's master branch, so you can be sure you won't break the provider.

thank you for explaining this, i do think this is actually a misconception amongst my teammates!

🌮 for @json! Very well explained. A couple of pieces of personal opinion: Many many many teams do provider-driven development, which just seems mad to me. If you didn't have the consumer, you wouldn't need the provider. So why is the provider specifying the contract? It doesn't know what the consumer needs, only the consumer does. My personal opinion is that developing this way costs everyone time, and leads to unnecessary frustration and friction between teams - even ignoring contract testing, I don't think developing this way makes sense.

It's not well-suited to APIs where we don't control both ends of the implementation, e.g. APIs we provide to clients

The general case is if you can't know who the consumers are, consumer-driven contract testing is not a good fit. Pact can work if you don't control the implementation, but still know who they are (and can ask them to send you a pact file). It does depend, though- if you're also providing an API client library to your customers, then you can contract test that library. @json’s point about breaking changes being a common point of confusion is spot on. Using Pact will mean you need to think about when you should ask compatibility questions (do I want to fail the master build if the master consumer is not compatible? Often, no). I don't personally use gitflow (and I try to avoid branches, depending on the team), but you can still use pact very effectively with gitflow, github-flow or trunk based - you just need to be clear about when and what you expect to be compatible. Something that may have changed since you last looked at it is that the broker has better support for telling pact which contracts you expect to fail the build. Have a look at pending-pacts and work-in-progress pacts here https://docs.pact.io/pact_broker/advanced_topics/pending_pacts

ruby, python, javascript and golang

These are all languages that pact supports 😎 . They share the same core, so anything you can do in one will work in the others.

Also, if everyone is in a provider-first workflow, BDCT was added to make the transition away from that mentality easier. But if you can start with consumer-driven, that'd be even better 👌

Teams who are doing provider-first development almost always have mocks. The question I would have is "how are you checking that your mocks work like the real thing?" - Pact makes that check very easy.

What a great thread - thanks all for your answers!

You all are really helping me understand benefits. Another coworker has expressed concerns saying

I believe my main concern with Pact was that you're not getting end-to-end tests, cause the frontend is still left out.
In our solution, that's still a pretty big chunk that's not getting tested and while you can say that the interfaces between services work okay (and you can probably even contract test the interface the frontend uses), the code in the frontend itself doesn't get verified.
In addition, I see an overlap between Pact tests and controller/requests specs/test. For a good amount of APIs, we're already verifying their behaviour based on the input -> output, so I'm wondering what Pact offers here in addition.

contract tests are not meant to replace end to end tests, but end to end tests are slower and less reliable than lower level tests. This is why the classic image of the test pyramid is so important. If you're not familiar with it, it looks like this:

Yea, I'm going to explain that this would not replace the cypress tests ive written. I thought they understood that but maybe i need to clarify. I totally agree and understand what you are saying @json thanks for giving me better language to explain it!

I'm not exactly sure what they mean by that, I suppose it depends on how they get the specs they're testing. That's what pact does, is it uses the pact broker to communicate a spec as defined in the consumer back to the provider for verification, so you can asynchronously verify that your specs are always accurate. If your team already has another system for doing the same thing, then maybe you don't need pact after all, but I don't really know enough to say

i dont believe we do, but i need to get clarify on what our specs are currently doing

we're already verifying their behaviour based on the input -> output, so I'm wondering what Pact offers here in addition.

Are you also verifying that the input you're testing the controller with is what your client actually sends?

Further to @json’s pyramid point, and the Pact footprint confusion, here's a picture I prepared earlier yay sometimes the visual helps a bunch more than just trying to explain with words. Here, Pact is just the blue boxes. The purple would be Cypress in your case, and ideally not pushing past that network boundary, to keep them fast and less flaky. The better the low-level coverage is, the less exhaustive the end-to-end tests need to be.

@Timothy Jones and @Boris you have really helped me feel more confident that I am on the right path towards doing what our team needs. I think the misconceptions about Pact have really held us back. I wish I had a bit more technical knowledge to actually implement it myself 😜. Have you all found it hard to coordinate cross team collaboration when certain teams own an endpoint?

One of the important things to remember is Pact is just a tool. It can help prompt & drive conversations, because it highlights things that aren't working, and gaps that might've just been assumptions otherwise.

This is a good point. Yea, I think this phase of teaching people how it actually works is probably really critical. I'm not so technically savvy and pretty new to my company so I'm treading lightly!

Pushing/forcing people to use a tool like this, begrudgingly, has always ended up in needing to promote the ownership and buy-in with now-annoyed people.

Yea, this is really good to bring up. Even if a manager forces everyone to do it, people being disgruntled to do something they don't understand or believe in will not work out well in the long run.

I find it's easiest to build in with empowered teams - when they already drive their own backlogs, support their own prod issues, etc. If the organisational culture is more siloed, or blamey, or has deferred responsibility (e.g. testing team, secops gatekeeping, etc), it's harder for people to want to own things.

I'm getting the sense that our leadership is really open and supportive. I get good vibes all around from teams, so I'm hopeful 🤞

Amazing if you have top-down support 💖

yes, my team is very open to trying it (minus one person..this is the person i want to convince 🙂 )

In my company's case, we use Data Dog synthetic tests in prod environments, and we have simple e2e smoke suite and health suites which we run after each deploy to any environment to check the deploy was successful. The trick is keeping any sort of e2e testing or full-environment testing to a minimum though, because of how slow and expensive they are.

I wouldn't point pact tests at a live server. You can do it, but managing provider states might be painful

@Matt (pactflow.io / pact-js / pact-go) ah yes, i wasn't thinking of pointing pact tests to a live env. I was asking whether people do pact AND also run e2e tests against a live env. A few devs on my team are pushing for running e2e against a live env and it was something I was told not really to do (or do VERY sparingly).

I tend to only run smoke-level tests in prod (ensuring connectivity), and have prod-shaped non-prod (e.g. staging).

It’s more just that the live envs get so messy that the tests end up being flakey from changing data etc

yeah, unless you have something like a test account in prod that is relatively isolated from real data, it's not really reasonable to expect that in prod

Yeaaaa that’s what our infra dev was telling me!