Hi all, I'm working on contract testing for a message broker scenario. My consumer has a modular structure with a handler->transformer (simplified). I've realized that running the test with a mocked transformer dependency won't cover some cases from the transformer. Alternatively, I'm thinking about extracting necessary assertions directly into the contract test. This way, I can manually check the contract's compliance with specific aspects of the application's functionality, including those from the transformer and other modules. However, this approach might lead to a disconnection from the actual app logic and could be prone to errors if the logic changes. Which approach is more aligned with the best practices of contract testing using Pact?

In case you haven’t seen, these docs help you understand how to write / think about message pact tests: https://docs.pact.io/getting_started/how_pact_works#non-http-testing-message-pact

Thank you for your insights. To provide a clearer picture based on the example from the link you shared, I'm working with a receiveProductUpdate function that serves as a port in this architectural paradigm. This function is also the focal point of the Pact test from the consumer's perspective. In my scenario, the function's logic extends through multiple services (on the port level) which are integrated via dependency injection. This leads to a critical decision point: should I test receiveProductUpdate by mocking its dependencies, or should I analyze all the interconnected services, extract their requirements, and then, instead of directly invoking receiveProductUpdate in the pact consumer test, formulate assertions based on these extracted requirements? An alternative could be to conduct an integration test, but this is quite complex and may not be feasible in all scenarios. Example in Go with 3 different options:

_ = pact.VerifyMessageConsumer(t, message, productServiceWrapper)

//cheking only productService, missing compliance with TransformerService and other dependencies
func productServiceWrapper(m dsl.Message) error {
    productMessage := *m.Content.(*Product)

    productService := NewProductService(mockTransformerService)

    err := productService.receiveProductUpdate(productMessage)
    if err != nil {
        return err
    }

    return nil
}

//use real transformer and turn it to integration test
func productServiceWrapper(m dsl.Message) error {
    productMessage := *m.Content.(*Product)

    productService := NewProductService(transformerService)

    err := productService.receiveProductUpdate(productMessage)
    if err != nil {
        return err
    }

    return nil
}

// bring all the requirements from services here, without calling the service directly
func productServiceWrapper(m dsl.Message) error {
    productMessage := *m.Content.(*Product)

    // Validate date format (e.g., "2006-01-02")
    if _, err := time.Parse("2006-01-02", productMessage.Date); err != nil {
        return errors.New("invalid date format")
    }

    // Check if status is either "active" or "deleted"
    if productMessage.Status != "active" && productMessage.Status != "deleted" {
        return errors.New("status must be 'active' or 'deleted'")
    }

    // Additional validations according to services...

    return nil
}

I would tend to go with Option 1, provided that there is decent test coverage (collaboration tests) between the

NewProductService

and

TransformerService

.

The transformer is just another service in the same code repo, so it's internal. What do you think about the third option? Is it viable at all? Full disclosure: in our company, the ESTEDS decided to go with the third option, which seems controversial to me, and that's why I decided to check it here. Can we conclude that it's a bad practice?

If I understand option 3, it’s not executing the code that would be executed in real life and thus runs the risk of testing the wrong thing, or if the code is refactored in a way that changes behaviour, it won’t be detected. So yes, I would say it’s not ideal. I can imagine potentially why you’d want to go this way and given enough guarding/other tests it could work, but in general I think it’s risky

Thank you for the clarification. I appreciate that.

No worries!

Hi @Matt (pactflow.io / pact-js / pact-go) After studying the documentation and extensive internal discussions about balancing a narrow scope in consumer tests with meaningful verification results, we've developed an approach that we hope could address these concerns. We recognize that responses in consumer-provider interactions often depend on business logic executed in lower layers of our application. To tackle this, we've come up with the idea of centralizing business logic requirements into a validation function. This is how it looks when evolving the code example from previous posts

// Checking only productService
func productServiceWrapper(m dsl.Message) error {
    productMessage := *m.Content.(*Product)

    productService := NewProductService(mockTransformerService)

    err := productService.receiveProductUpdate(productMessage)
    if err != nil {
        return err
    }

    return nil
}

func (p *ProductService) receiveProductUpdate(product Product) error {
    err := validate(product)
    if err != nil {
        return err
    }

    p.transform(product)
    p.post(product)
}

func validate(product Product) error {
    // Validate date format (e.g., "2006-01-02")
    if _, err := time.Parse("2006-01-02", product.Date); err != nil {
        return errors.New("invalid date format")
    }

    // Check if status is either "active" or "deleted"
    if product.Status != "active" && product.Status != "deleted" {
        return errors.New("status must be 'active' or 'deleted'")
    }

    // Additional validations according to services...

    return nil
}

This approach aims to enhance our consumer tests with additional validation layers, reflecting the downstream business logic requirements. We believe this could keep our test scope narrow while ensuring the tests remain effective and relevant. What do you think about this approach? Can we go as far as calling only the validate() function in a Pact test instead of the handler, considering it's used in production and maintained by the developers?

Nice! You could possibly still call

receiveProductUpdate

and mock/stub the downstream services, but that would be roughly equivalent anyway. So the answer is probably “yes” from me, with caveats like “as long as you test the other bits”

hi Guys, thanks for this thread, very useful. On this use-case context, I have a doubt @Matt (pactflow.io / pact-js / pact-go) please. Can I use the validate directly? since it is maintained by developer's production code and no risk of missing implemented requirements.

// Checking only productService
func productServiceWrapper(m dsl.Message) error {
    productMessage := *m.Content.(*Product)
    err := validate(productMessage)
    if err != nil {
        return err
    }

    return nil
}

so it generates the contract using production requirements code logic, without mocking (option 1) or using the dependency services (option 2).

As I said above, yes, assuming you have other good tests in place

thanks @Matt (pactflow.io / pact-js / pact-go)