I'm trying to add v4 support to PactNet and I've g...
# libpact_ffi-users
a
I'm trying to add v4 support to PactNet and I've got in a real mess when I try to use it for real at work. Not every pact will be upgraded to v4 at once (we can't enforce that), so we'll have a mix of: • v2 pacts with HTTP interactions • v3 pacts with HTTP interactions • v3 pacts with message interactions • v4 pacts with HTTP and/or message interactions The verifier gets in a serious mess when this happens. It starts sending the HTTP requests to the
/pact-messages
endpoint as if they're async requests, which then fails because obviously those are in a completely different format to what's expected. What am I supposed to do in that situation?
🤔 1
m
As in, you have a Pact file that has different versioned interactions in them? Or different consumers are on different versions of the pact spec?
a
The latter, yeah - some consumers have like v2 HTTP pacts and some have v4 pacts, but the verifier picks up the v2 pact and treats all the HTTP interactions like they're messaging interactions and hammers the provider with a bunch of invalid requests
m
ok that scenario makes sense to me, it sounds like it might be a bug then.
👍 1
I’m just on a call, but later tonight I’ll see if I can repro in Go/JS or something
a
Yeah cool. I've started putting together a small example in C# as well, although having it in another language would be even better just to confirm it's not just the way I've implemented something
👍 1
Looking through the code, I think this is to do with how I've implemented it, but in a way that I think might need a change in the FFI approach. For messaging interactions, there needs to be a REST API running with a
/pact-messages
endpoint available which returns the message body when given a magic string. That's really onerous to put that requirements on everyone using message pacts though, so instead PactNet includes a little API which starts transparently to the user and implements the
/pact-messages
endpoint. The problem is that the FFI expects just one API which does both the 'real' endpoints and the
pact-messages
one, but as I say the burden that places on the user isn't really workable. So PactNet currently calls the
pactffi_verifier_set_provider_info
FFI method and points it at the little built-in messaging API. Problem is that this makes the FFI forward all interactions there as if it's the real API. I think it should work more like the provider states endpoint - when you call
pactffi_verifier_set_provider_state
you can pass in an absolute URL, so in theory the provider states endpoint could be on a totally different API running on a different port (or even a totally different host if you wanted to get crazy). I could really do with a way of running a small built-in API on a different port to the main API and having all the
/pact-messages
calls sent over to that API instead. I can't really mandate that users have to implement that endpoint themselves on their 'normal' API, nor can I write something which adds it on for them.
That approach works fine in v3 messaging pacts btw, because v3 pacts can't contain both HTTP and messaging interactions, so even though HTTP interactions would get forwarded to the
/pact-messages
endpoint and break, in practice they don't because there never are any.
Oh and also it works fine because prior to v4 we mandated that anything using messaging had to have a different name, whereas from v4 we're allowing the same name for both HTTP and messaging interactions. From v4, hitting the pact broker for
Orders API
might give you back v2 HTTP, v3 HTTP and v4 HTTP+Messaging, which is what causes the problem with the in-built messaging handler API
So yeah, if there was some kind of
pactffi_verifier_set_messaging_endpoint
or something which you could provide with an absolute URL and it would call for all messaging interactions, that would solve the problem with consumers that have both HTTP and messaging interactions
y
That's really onerous to put that requirements on everyone using message pacts though, so instead PactNet includes a little API which starts transparently to the user and implements the
/pact-messages
endpoint.
This sounds nice, I like your approach to the end user experience, it really showed in the migration from v3 -> v4 Your proposal sounds sensible 🙂
whereas from v4 we're allowing the same name for both HTTP and messaging interactions.
Yeah I actually haven't given that a whirl yet, but yeah that is a key feature
From v4, hitting the pact broker for
Orders API
might give you back v2 HTTP, v3 HTTP and v4 HTTP+Messaging, which is what causes the problem with the in-built messaging handler API
Yeah I assumed the pact verifier (whether via FFI or CLI) would just handle this, and therefore my recommendation would be to a user stuck on a v2 ruby-backed provider language but generating v3/v4 specs would be to use the verifier cli. Obviously that involves the user knowing more about the lifecycle and appropriate options, than it neatly wrapped up in the DSL Appreciate you putting together the examples 🙂
a
Nah, the verifier CLI would be broken in the same way
y
I'll let Ron mull over the implementation detail, as I have many yaks to shave, but sounds good and will be awesome to see this in pact-net
Nah, the verifier CLI would be broken in the same way
ty for confirming, no-one has mentioned it so far, although a few have mentioned it caused them pain in v3 and below, needing separate named providers for message pacts, I haven't heard anyone mention your comments, so you are probably bleeding edge, just as you had proper use case tests in production. The team (PactFlow engineers) have been thinking about a way we can build up a set of examples for the core features (so v4 multi interaction would be one)
this should help hopefully maintainers implement in a client, language, and for us hopefully to demonstrate the ffi method lifecycle in some reference implementation examples
a
Say you have an API called
Orders API
with 2 endpoints: •
POST /api/orders
for creating orders •
GET /api/orders/{id}
for getting an order by ID and when you create an order, it also sends out a pubsub message to say that an order has been created. That's your provider API. Then you have 2 consumers: • Basket API - this calls
POST /api/orders
to create the order when a user clicks Buy. It is a v2 or v3 pact with only HTTP interactions • Fulfillment API - this listens to the pubsub topic for when orders are created, and calls the
GET /api/orders/{id}
endpoint to retrieve the new order details in order to trigger the actual shipping of the order. It's a v4 pact with both messaging and HTTP interactions inside When the Orders API hits the pact broker to perform a verification, it'll get both of those pacts back and start trying to verify them. In order to do that, the Orders API must have an endpoint called
/pact-messages
so that the verifier CLI/FFI can post a magic string to it and get a message body back. The problem is that the real Orders API doesn't have that endpoint, because that's a testing implementation detail. So, instead we create a little built-in API which has that endpoint so the user doesn't have to, and that's running on a different port to the main Orders API. This doesn't work though, either via the FFI or CLI, because you can only supply one URL. Either you give it the real Orders API URL, and then the messaging interactions fail because there's no
/pact-messages
endpoint, or you give it the messaging API URL, and then the regular HTTP interactions fail because it's just a little stub service for handling messaging stuff. So what you really need is to be able to provide two different URLs. One for the 'real' API, where all the HTTP interactions should be sent, and one URL for all the messaging interactions to be handled (which may be different to the HTTP URL, but doesn't have to be). In other languages you can probably monkey-patch the messaging endpoint onto the real API at test time from the test library itself, but you can't do that in quite a few languages. So either the library has to have a little built in API to handle them, or the user has to manually implement that endpoint (which is non-trivial). That situation would've never happened with v3 pacts, because you can't have a pacticipant called
Orders API
which has both HTTP and messaging interactions. They'd have to have different names and be in different pacts files, but from v4 that same name can now contain both, and that's what necessitates something that can handle the two types differently.
I've just found
pactffi_verifier_add_provider_transport
which I think can be used to do what I want above
👍 1
y
https://github.com/pact-foundation/pact-reference/commit/9b1b7965f0b97a8c7d11f2d8bb6bbeaa5a79e482#diff-afeb3e0e20ecfe56b3[…]f87602daaaa7c07de3dba859 now you mention this, it brings back memories of when I first trialled out plugins with Tien's PHP example, there was some oddities, I scattered a brain dump in some code comments, and sure I posted about it somewhere but can't find the issue or slack message about it, popping out for lunch shortly
👍 1
a
Yeah so I think adding a transport is the way it would be done, but to do that I'm going to have to make a breaking API change in PactNet no matter what I think. When you create the verifier for the example above then you need to add both a HTTP transport and a messaging transport, but the API of PactNet currently only allows you to do one or the other, because in v3 pacts it wasn't allowed to do both (and I didn't know v4 allowed it at the time I wrote it like a year ago)
m
I could really do with a way of running a small built-in API on a different port to the main API and having all the
/pact-messages
calls sent over to that API instead. I can’t really mandate that users have to implement that endpoint themselves on their ‘normal’ API, nor can I write something which adds it on for them.
You’re right, this is part of the guide we need to write for maintainers (this is how it’s done in both JS and Go, and how i’ve recommended others do it. We discussed a callback approach, but trust me (and more so Ron), it makes the FFI much more low level and running an HTTP service is just much simpler to do).
Yeah so I think adding a transport is the way it would be done, but to do that I’m going to have to make a breaking API change in PactNet no matter what I think. When you create the verifier for the example above then you need to add both a HTTP transport and a messaging transport, but the API of PactNet currently only allows you to do one or the other, because in v3 pacts it wasn’t allowed to do both (and I didn’t know v4 allowed it at the time I wrote it like a year ago)
Mind elaborating on why it would be a breaking change in the .NET DSL? In JS/Go, the REST API / proxy is completely transparent to the user. We just register one port for messages and another port for HTTP
I’ve just found
pactffi_verifier_add_provider_transport
which I think can be used to do what I want above
yes, that’s the way around it. In fact, you’ll need to use this if you verify a plugin also because it will use a different scheme (e.g.
grpc
)
This is why we are writing a maintainer guide! The raw ingredients (functions) are all there, but how you mix them together (the recipe) matters
Sorry for this confusion. It almost certainly would be in a thread in this channel, now long lost to history.
a
I've added an RFC to the PactNet channel to detail why this is a breaking change. I've got it working locally now though with that breaking change
👍 1
m
Nice! I had a quick pass now, but it’s late and I’ve already had a negroni. I’ll review with fresh eyes next week, and also compare against JS/Go for inspo
have a great weekend!
👍 1
Also, thanks Adam - I think this is all really helpful. We should include a section on this (setting up an intermediate API, how to set multiple transports etc.) in the maintainer guide we’re about to pick up, what do you think @uglyog?
u
Just reading through all this, a maintainer guide would be a way to describe how to deal with this problem.
👍 1
m
Good timing then 😉