Hello! I feel a bit confused about using provider states. Even after reading documentation, seeing examples of test applications on the web and trying to find related answers about that - even after that, it is tough to consolidate naming and describe a little algorithm of his naming. So, as I can see, if we are mocking the repository layer (or even the business service layer that is directly referenced to the repository layer) - we should set a human-readable provider state. For example, for Task Manager API and test

GET

and

POST

methods, we should name the provider states as

Task with existing ID is available

and

Task with the correct body is created

. But, how should we call the provider state of checking unauthorized scenarios or bad requests (with invalid request body)? And which name of the provider state is better in the point of view of Pact best practices, the first one or

post-task-success

and

get-task-success

?

I would definitely tend towards human readable, and describe a state that the provider can represent before a request is made. • "Task with existing ID is available" sounds ok, although "existing ID" is hard for me to understand, but it might be very meaningful for your domain. • "Task with the correct body is created" sounds more like an outcome or action, rather than a declarative state. If you take it from a ReST perspective, it might be easier to think about. You want to say "my provider is in this state, so when my consumer asks it to perform an action, my intended outcome will be achieved". The same state could be useful for several actions. A

POST

may only need a parent resource to exist beforehand, so that's all you need to specify. The same state may work for a

GET

or

DELETE

of the parent, too.

Also, you probably don't need bad requests in your pact. Pact tests are consumer driven - which means they cover the interactions that the consumer will actually send.

It seems a bit more challenging than I imagined. So, I created a diagram of your answers. Can you please check if it is correct? Also, I want to ask, should we provide a separate state when we cannot find the object by id? @Matt (pactflow.io / pact-js / pact-go) Matt, please, can I ask you for an additional answer to this question?

I agree with the comments from Andras and Tim

Thanks, Matt!

I'd be interested to know what kind of invalid input you expect your API client to produce. That might remove a third of your cases, which can also be caught be Pact.

One thing to consider is whether or not you want to test all of the authorization errors per endpoint, or just a representative sample on a key endpoint (usually, auth is universally applied so I’d tend to only run these once, and not across the entire set of API endpoints)

Also, it would be surprising if all the endpoints handle auth by themselves, so the general "unauthorised" contract should be the same test, regardless of endpoint. Depends on your auth model, but I usually don't duplicate it for every endpoint/verb combination.

If your client code is designed so that you can’t send a bad request, that might be hard to test properly. You can still of course have a scenario somewhere that can deal in a generic

4xx

or

5xx

error (auth errors probably warrants a separate error handling to validation or system outages). That can probably be done outside of Pact though, because it might be hard to produce a

5xx

on the provider side

The "bad request" response should still be handled, but probably in provider-level (functional?) tests, rather than contract tests. I.e. you want it to respond appropriately if a new consumer makes bad requests (e.g.

curl

)

Authorization sets on an entire controller, so I cannot imagine how we can test it once. Only if check it for GET method, and delete from every endpoint, for example

Can your consumer actually send a request like that, though? An ID like that usually (at least, in my code) comes from the provider, and is sent back without modification (optionally parsed into a strong type, too).

Pact is not about testing that your API behaves as specified, it's about testing that your API behaves as the client actually uses it

oh yeah, that might be the missing piece 🙏

So you don't need to cover cases that your client doesn't send

So, is that conclusion correct? • We should cover only each case that client sends to our provider; • Check unauthorized responses only once per endpoint (e.g. for GET method); • Give names to provider states as generic as we can, but complete them as human-readable as possible; • Cover with Pact case for bad request, where expect a validation error object from a provider - is okay

We should cover only each case that client sends to our provider;

Yes, and all response types (eg both 404 and 200 for

GET /some/item

)

Check unauthorized responses only once per endpoint (e.g. for GET method);

Yes, usually - generally you need one response type per client behaviour. For example, if your client treats all 401s the same, then you don't need tests for invalid token vs expired token, etc. But, if the server responds with a reason that prompts the user differently (ie, if the client relies on a specific field value eg

reason: "expired_token"

to switch some behaviour), then you'll want a test for each response

Give names to provider states as generic as we can, but complete them as human-readable as possible;

Yes

Cover with Pact case for bad request, where expect a validation error object from a provider - is okay

Yes. Examples might include

username: "      "

, which maybe the user can enter, might return

400: {reason: "Your username must not contain spaces" }

, or maybe

username: "bob"

->

{reason: "Your username is too short" }

In a case like that, if the behaviour is "show the reason to the user", then you don't need to test each reason that the provider might reject a username, you just check that the response is

400 { reason: <string> }

Great, everything is understood but requires a lot of practice. Thank you very much @Timothy Jones @Boris @Matt (pactflow.io / pact-js / pact-go)!

Awesome! Feel free to ask more questions when/if you have them

Hello, guys! Could we continue our discussion about provider states with ideas from my colleagues? My colleagues propose to introduce the one God state that sounds like “`default`” and introduce something another; only if we need to change the response to the client from mocked domain service (repository layer). In “_standard_” conditions, will we behave in the following situation: • Verifying for HTTP 401 Unauthorized status code - will be done on the layer of Authorization middleware (C# language). So if the client does not send the Authorization header in his request - we regard that as “`an unauthorised user`” scenario and respond with HTTP 401. So, the provider doesn't have any mocks of domain services in that case (or even if it has, for us, it doesn't matter), and we don't see sense in introducing another provider state. • Verifying for HTTP 400 Bad Request - will be very similar to the first one, but in that case, a client should pass the Authorization header on his request, and we automatically assign it to this user admin role. And if given other body doesn't meet controller / DTO validation - we expected this response from the provider. And even in this case, execution doesn't execute to mocked domain services - so we can say that it doesn't matter what is mocked on this layer during that verification on the provider side. • Verifying for HTTP 200/201 Success/Created - but in this case, it is the most complicated because the main idea from my colleagues is if it a just positive case and we don't have any specific response to the client - mock all domain services that are using for verification on the provider side to himself all methods in positive cases. So, in conclusion, I need your help to answer the two main questions: Is that correct from the point of view of Pact conception and in theory, is that implementable on .NET with Moq library (will we not use only the last mock for each domain services, as I write before - because that will be a lot, for each domain service for all himself methods to mock positive scenarios)? Thanks

I don't know much about the .NET ecosystem, so some of those questions might be better placed in #C9UTHV2AD 😅

Thank you, Andras! I understand; so, if that is correct from the point of view of global conception - it should be implemented just like that

As very general advice, yes 🙂 I would just try it, and see what problems you end up with. Nothing is really perfect on the first attempt 😇

Remember that Pact is not about describing the whole API, but just the parts your client needs. Usually you don't need to test 400 bad request, unless the client is expected to receive and understand that (eg some designs with validation might be done this way)

Yes, that's already clear to me after our previous discussion. Thank you, Timothy

Ah, sorry. I didn't check which discussion was above 😅

Andras has also written about newer Pact versions and multiple provider states. So that is definitely what I need to figure out. But, if we are talking about the default provider state, I mean the provider state with the name exactly

default

, not

"user with ID exists who has task 1 open and task 2 closed"

. Just

default

, without a shortcut or, better, an alias for getting a specific user. But, if we rethink that process, the default aligns only for fetching all users or getting one by id, so a provider state with a name default isn't enough. Instead, we should inject a new provider state with a different name (maybe as

"user with ID exists who has task 1 open and task 2 closed"

). If moved back to Authorization, so in our contact tests, we implemented him by adding additional middleware here, such like

services.AddAuthentication(AuthenticationHandler.Schema)
            .AddScheme<AuthenticationSchemeOptions, AuthenticationHandler>(AuthenticationHandler.Schema,
                _ => { });

and

public sealed class AuthenticationHandler : AuthenticationHandler<AuthenticationSchemeOptions>
{
    public const string Schema = "Test";

    public AuthenticationHandler(IOptionsMonitor<AuthenticationSchemeOptions> options, ILoggerFactory logger,
        UrlEncoder encoder, ISystemClock clock) : base(options, logger, encoder, clock)
    {
    }

    protected override Task<AuthenticateResult> HandleAuthenticateAsync()
    {
        AuthenticateResult result;

        var authorizationHeader = Context.Request.Headers.Authorization;
        if (string.IsNullOrWhiteSpace(authorizationHeader))
        {
            result = AuthenticateResult.Fail(
                new UnauthorizedAccessException("To access API, you must provide a Bearer token."));

            return Task.FromResult(result);
        }

        var claims = new[] { new Claim(ClaimTypes.Name, "Test user"), new Claim(ClaimTypes.Role, "admin") };

        var identity = new ClaimsIdentity(claims, Schema);
        var principal = new ClaimsPrincipal(identity);
        var ticket = new AuthenticationTicket(principal, Schema);

        result = AuthenticateResult.Success(ticket);

        return Task.FromResult(result);
    }
}

For me, I don't like the name

default

as it is ambiguous - I would confuse it with "the state that a blank provider would start in". But the concept of "this state actually covers most of the situations I want to test" is pretty helpful.

Agree. We should think about a name that says, "we mocked all domain services for all positive scenarios, that's also compatibly to test all 401 unauthorized and 400 bad request requests"

This is where the granular states really help. With one state, you would go:

state: "user with ID=10 exists who has task 1 open and task 2 closed"
GET /users/10
returns {/* user 10 */}

and

state: "user with ID=10 exists who has task 1 open and task 2 closed"
GET /users/10/tasks
returns [{/* id=1, open */}, {/* id=2, closed */}]

with granular states, you could say:

state: ["user with ID=10 exists"]
GET /users/10
returns {/* user 10 */}

and

state: ["user with ID=10 exists", "user 10 has task ID 1, which is open", "user 10 has task ID 2 which is closed"]
GET /users/10/tasks
returns [{/* id=1, open */}, {/* id=2, closed */}]

this has clear advantages when you want to add new concepts (eg, "task 2 has comments from user 12")

Granular states look fantastic and don't confuse like just

default

I usually just don't provide any provider state in that case. What you're saying then is "it doesn't matter what state the provider is in, I expect request X to respond with Y"

I usually just don't provide any provider state in that case

I am a bit surprised. Is that possible?

Apologies, I don't think I understood your 401 question

Sorry, my guilt is related to the general difficulty of our enterprise system. It seems like we can just don't provide any provider state in that case, also if it's possible

Is that possible?

Yep. The point of provider states is to set up some precondition. If you don't have any preconditions, you don't need to use them. If you prefer, a related pattern is to use empty states - like "there are no users", which you implement to do nothing during setup or teardown. This is a neat way to use the states as documentation.

Great, I understand. Can you also help me define the provider state for testing entity creation (POST method)? Something like

"user with ID=10 does not exist"

is a good one? Whereas, for other methods (GET, PUT and DELETE), we can adapt your granular states example

"user with ID=10 does not exist"

This is exactly what I do.

Sorry, I don't understand you completely. What does it mean to teardown for each state? As I understand, for the state that is representing no-state or, in other words, nothing preparing / mocking on the provider side - such as the "there are no users" state we should do nothing before pact verification of that test case. So, what does it mean to do a teardown or reset the server on such states? Can you please explain more?

Apologies, I wasn't clear. Pact might verify the interactions in any order. So, if you have interactions with no state, you need to make sure that the interactions that do have state are torn down. Eg:

start server with mock repos that are empty

"user 10 exists"
setup - mock user repo layer so that user 10 exists
teardown - replace mocked user repo layer with empty mock

Perfect, everything is clear!

You're welcome!