blobwave Hey Alex.
I’ll be honest, I have been pretty disappointed with the state of API Testing tool, with the intent of mitigating Provider drift, which is a common pain point for users and organisations invested in API Documentation as a source of truth.
Even for code-generated definitions, the potential for implementation to drift from the spec, is possible.
Over in the PactFlow camp, this is something we have wanted to address, and as we were acquired through SmartBear, it formed a natural part of the API delivery story, alongside other tools such as OpenAPI/Swagger.
We have a roadmap item here
https://github.com/pactflow/roadmap/issues/31
I personally would love to see a tool similar to Dredd, point and click against a deployed API, that could give you coverage details of your api description. I can see many arguments for it being able to hook in at a unit test level, but also see value in it being able to healthcheck deployed systems.
I feel like a API Description format aware Pact verifier, that could read an OpenAPI document for example and play those requests against a provider implementation. Converting the OpenAPI description into an intermediary format, a Pact file, might allow for the addition of matchers/provider states for fine grained control over the verification properties, and allow for an easier entry point. (It may be easier to convert an openapi file format to a pact file, that hacking about at Pact itself).
I’ve messed about with a few spikes before with OpenAPI descriptions with custom annotations for Pact to allow for generating Pacts from an OAS. You can also indirectly do that today OpenAPI -> Postman Collection -> Pact File with some open source tooling