Hey 😄 . I'm implementing Contract Based Tests for a project im working on. We are using our self hosted pact-broker (also considering switching to pactflow). Im wondering if anyone knows ways to gather some metrics. The /metrics endpoint gives some general data, but i was wondering if its possible to separate them per service or per integration? We are looking for failure rate.

Not at this stage sorry Gustav, but we are looking into it (see the other thread in general the other day)

We are trying to measure success for the CBTs that we already created, to see if they are helping at all, before we scale our operation. We already have detailed metrics for our FE and BE e2e tests that we measure per service daily. We combine them with the amount of bugs found per team. Then they are compared to teams which don't have those tests. The results are presented to the management mostly. We map the data over time to see if the tests make any difference at all in preventing bugs in the higher environments. We were thinking of doing something similar for Contract Based Tests. Maybe compare CBT failure rate against the amount of bugs found per service?

Thanks for sharing, that’s helpful to know

Yes. We have different teams for development and test automation. Then we have a different team for CBT testing. I have sent out a survey to the developers in the teams where we implemented the CBTs, but its been hard getting any results back. Its been 2 months since we added the first batch of CBTs. Maybe its too soon?

Interesting. Usually the Devs would write the contract tests. Who's writing them now? Are you using Pact or Pactflows bidirectional testing approach?

We are using both. Between internal services we use pact. With external services we intend to use pactflow with bidirectional (though its still in the Proof of concept phase). We created the CBT team to lay down some initial tests and infrastructure in our pipelines to make it work. The initial plan was for the devs to pick it up eventually, which is slowly happening now as well. Certain teams are more interested than others, depending on what type of services they own. We have quite a few internal and external integrations to cover, and just thinking if its worth scaling the CBT team to make it happen quicker.

We created the CBT team to lay down some initial tests and infrastructure in our pipelines to make it work.

that’s chefkiss , it’s a great way to help set good patterns/practices/habits (pact aside, any tool/practice etc.)