https://pact.io logo
Join Slack
Powered by
we're looking at using Pact to give us nice test s...
# pact-js
a
we're looking at using Pact to give us nice test separation between our front-end clients and our back-end, which provides a GraphQL API for them to use. at the moment, from having experimented with the GraphQLInteraction class in the Pact JS library, it appears as if fairly complete GraphQL query strings need to be provided in order for the Pact mock API server to return the correct response. i have managed to get this working by formatting the query in precisely the right way (e.g. 
... on
rather than 
...on
, etc.), according to the GraphQL client library we are using (urql), but i feel that this approach seems a little brittle currently. IMO, with GraphQL, the 'contract' is the strongly-typed schema the technology provides, rather than the precise way in which a client interacts with it. putting this another way, if i make a small change to the query a given client makes, i don't feel that should be regarded as a change of contract, yet so far as i can see, this is how the Pact GraphQLInteraction stuff is currently set up. i was wondering whether, rather than providing a full GraphQL query string, it would be possible to build some sort of strongly-typed schema-based resolver for providing the mock API functionality? this could leverage existing tools (e.g. https://www.graphql-code-generator.com), and still capture interactions for playback against the server logic (in the spirit of Pact), but the developer experience might be somewhat nicer and more idiomatic.
m
Yes
Pact is a specification by example testing framework
I don’t see Pact ever supporting the kind of proposal you have, but I do see that as a viable approach for the bi-directional contract testing feature of Pactflow
What you’re proposing is more of a schema comparison approach, which is different to the way Pact works and will likely ever work
a
thanks for your thoughts. given what you say, it sounds like Pact might not be quite the right tool for testing GraphlQL APIs in this way.
i may look into building something more well-suited to its features.
m
no problems
It’s something we’ll explore for sure in Pactflow
I’d also suggest having a read of https://pactflow.io/blog/schemas-are-not-contracts/ so you can understand the challenges in your approach
(It’s not to say you shouldn’t do it, but just be aware of the elements of contract testing you’ll miss out on)
a
i will, thanks. i think the main issue i can see with regarding a specific example as canonical is that in GraphQL, it's possible for a client to request things such as field name mappings (e.g. 'firstName' from 'name'), and i see these as not fundamentally changing the contract between the two systems; they're really just metadata of a request.
in practical terms, these sorts of things would normally be handled by the GraphQL server library in use (e.g. Apollo), so application code would not be affected by a change of field name mapping, ergo regarding it as a different example would seem wrong.
m
Can you please elaborate on this? How does the evolution of this happen? So a provider can rename a field and old referencew are resolved by Apollo?
a
no
when consuming a GraphQL API, consumers can describe the 'shape' of the data they want to be returned
this includes the selection and names of fields, as well as more complex things
if, for some reason, a given consumer wants to change the name of a field it receives from the API (as with the name->firstName example i gave earlier), this is something which would be transparent to the provider implementation
as typically it would be handled entirely by GraphQL server libraries such as Apollo
my point is that this sort of consumer-specific change is not a change of contract, IMO, but the string-matching approach taken by GraphQLInteraction in Pact JS currently regards it as such.
m
if, for some reason, a given consumer wants to change the name of a field it receives from the API (as with the name->firstName example i gave earlier), this is something which would be transparent to the provider implementation
I don’t see how this wouldn’t break any type of contract. Either, the provider has the field or it doesn’t. If it gets renamed in the process by the consumer, sure, that’s up to them and is outside the scope of any contract test (Pact included)
But I think you’re saying, in GraphQL you can change the query which affects the client side (e.g. a field alias). That’s an unfortunate side effect of Pact’s approach, yes
a
yes; in practical terms, if i write a test that causes a client to make a GraphQL API call which is subsequently mocked / stubbed / recorded by Pact, this is effectively a snapshot of the actual text of the query, which feels brittle to me for the reasons i've outlined.
a subsequent change to that query which has no practical impact on any other system should not then be regarded as a change of contract, IMO.
m
I think that’s fair, but it’s hard to make Pact perfect in this case to suit all types of APIs
a
or more accurately, i think that currently Pact does not support GraphQL in an idiomatic way
m
I think this is an edge case that is more of a theoretical issue than a real one.
or more accurately, i think that currently Pact does not support GraphQL in an idiomatic way
this is definitely true
I would prefer to see us properly handle the AST rather than the current string based approach
Plugins will be where this gets properly solved
This will be a content type plugin
a
interesting - i think we already have the tooling required to support this sort of behaviour, though i'm not 100% sure how well this would fit with the recording of a given interaction.
👍 1
there's a slight tension between the requirement to mock a GraphQL API in an elegant way, and the recording of a specific interaction for playback against a provider
m
Yes, we’re not a general “take a schema, make a mock server” tool
a
no, i understand that
m
Oh, totally, just agreeing with you 😛
a
i've built something which pretty much does exactly that with our schema, but i was particularly interested in using Pact to avoid the horror of full-stack / 'E2E' tests (and for a few other reasons)
👍 1
i'll have a think about whether i want to separate out the concerns of automated front-end testing and 'pure' API testing, as i think this may go some way towards solving this slight messiness
my main concern with the former being conciseness, elegance and expressiveness for developers, and with the latter being precision / correctness, i think.
anyway, thanks for your thoughts, i'll stop taking up your time now!
m
not at all - anytime!
this is helpful for me/us as we look to solving GraphQL better going forward
a
well, we do quite a bit of automated testing with TS and a GraphQL API from various clients, so by all means yell if you want to chat about anything. I'm no expert in GraphQL (which is a deep technology), but i've encountered quite a lot of the common 'gotchas', i think
🙏 1
Open in Slack
PreviousNext
Powered by