Pact Foundation #avro

Ali Ustek

11/18/2022, 1:05 PM

I've not made much progress, I had a chat with @Matt (pactflow.io / pact-js / pact-go), he mentioned that matchers work in Rust only and I've started to familiarise myself with Rust, as I am Java/Scala dev

🙌 2

Yousaf Nabi (pactflow.io)

11/18/2022, 1:14 PM

I think this PR might be useful, it exposes matching rules via the FFI functions https://github.com/pact-foundation/pact-reference/pull/229 rather than needing to be re-written in the consuming language

Matt (pactflow.io / pact-js / pact-go)

11/30/2022, 11:29 PM

FYI I thought a feature request existed but it did not: https://pact.canny.io/admin/board/feature-requests/p/plugin-avro

Yousaf Nabi (pactflow.io)

12/14/2022, 10:38 AM

This was the avro file I was using for my little hello world app, just so we can have a look at the structure

Copy code

{
  "type": "record",
  "name": "Item",
  "namespace": "example.avro",
  "fields": [
    {
      "name": "name",
      "type": "string"
    },
    {
      "name": "description",
      "type": [
        "string",
        "null"
      ]
    },
    {
      "name": "price",
      "type": [
        "double",
        "null"
      ]
    }
  ]
}

👍 1

Ali Ustek

12/14/2022, 11:13 AM

@Yousaf Nabi (pactflow.io) here is my cannibalised protobuf to avro plugin skeleton https://github.com/austek/pact-avro-plugin/tree/rust_main

🙌 3

Ali Ustek

02/12/2023, 8:20 PM

I've put a skeleton of the avro plugin together, https://github.com/austek/pact-avro-plugin

PactAvroPluginService

is the Plugin implementation. I've lots of questions but here is a small list before I can continue 1- what is supposed to be in the ConfigureInteractionResponse, what is the content of the each Interaction? is it what ever the user put into the test for the consumer? 2- I've tried a dummy Interaction but getting

While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either that the input has been truncated or that an embedded message misreported its own length.

, so far I don't know why 3- What is plugin supposed to do? get a request, get a response and make the mock server respond with that response given the request? 4- where do content matchers/generators get into picture?

Ali Ustek

02/16/2023, 4:48 PM

I'm trying to add the avro schema to PluginConfiguration, test log shows it on the payload but final pact file doesn't have it

pact_test.log avro-consumer-avro-provider.json

👋 2

Ali Ustek

02/16/2023, 4:48 PM

what am I missing?

Ali Ustek

02/28/2023, 11:36 PM

@uglyog Hi, in

CompareContentsRequest.actual

and

CompareContentsRequest.expected

are of type

ByteString

. how are the values of those fields calculated. Can I just use

actualBody.getContent.toStringUtf8

to and use it to build the Avro message?

Ali Ustek

02/28/2023, 11:36 PM

Or is it the plugin populating them somewhere else that I have to do?

Ali Ustek

03/07/2023, 11:00 AM

In Match Context, is the path for arrays supposed to contain the index?

Ali Ustek

03/07/2023, 11:03 AM

In Content Matching response, what is supposed to be in the

ContentMismatch.actual

and

ContentMismatch.expected

Ali Ustek

03/14/2023, 9:49 AM

@Yousaf Nabi (pactflow.io) my understanding is that the plugin only needs to be available locally. meaning it doesn't need to be uploaded to the pact broker, am I right?

✅ 1

Ali Ustek

03/14/2023, 9:51 AM

Also, Do I need to enable V4 driver on the provider test? if so, how?

Ali Ustek

03/14/2023, 10:02 AM

My provider test is failing but I don't see logs from the plugin

Ali Ustek

03/14/2023, 10:02 AM

Copy code

09:42:14.391 [main] DEBUG a.c.d.pact.provider.ProviderVerifier - found class @au.com.dius.pact.provider.junitsupport.Provider("order-provider") @au.com.dius.pact.provider.junitsupport.loader.PactBroker(url="<http://localhost:9292>", host="${pactbroker.host:}", port="${pactbroker.port:}", scheme="${pactbroker.scheme:http}", tags={${pactbroker.tags:}}, consumerVersionSelectors={@au.com.dius.pact.provider.junitsupport.loader.VersionSelector(tag="${pactbroker.consumerversionselectors.tags:}", latest="${pactbroker.consumerversionselectors.latest:}", consumer="${pactbroker.consumers:}", fallbackTag="")}, consumers={${pactbroker.consumers:}}, authentication=@au.com.dius.pact.provider.junitsupport.loader.PactBrokerAuth(username="${pactbroker.auth.username:}", password="${pactbroker.auth.password:}", token="${pactbroker.auth.token:}", headerName="Authorization"), valueResolver=au.com.dius.pact.core.support.expressions.SystemPropertyResolver.class, enablePendingPacts="${pactbroker.enablePending:false}", providerTags={${pactbroker.providerTags:}}, providerBranch="${pactbroker.providerBranch:}", includeWipPactsSince="${pactbroker.includeWipPactsSince:}", enableInsecureTls="${pactbroker.enableInsecureTls:false}") class com.collibra.example.pulsar.avro.PactPulsarProducerTest
09:42:14.393 [main] DEBUG a.c.d.pact.provider.ProviderVerifier - found method [@au.com.dius.pact.provider.PactVerifyProvider("Order Created") public au.com.dius.pact.provider.MessageAndMetadata orderCreatedEvent()]
09:42:14.399 [main] DEBUG a.c.d.pact.provider.ProviderVerifier - Found methods = [public au.com.dius.pact.provider.MessageAndMetadata com.collibra.example.pulsar.avro.PactPulsarProducerTest.orderCreatedEvent() throws java.io.IOException]
    generates a message which
      has a matching body (FAILED)
      has matching metadata (OK)

Failures:

1) Order Created: generates a message which has a matching body

    1.1) body: / Actual body '�name-1��|A�������?�street nameItem-1Item21' is not equal to the expected body '�name-1��|A�������?�street nameItem-1Item-2'


09:42:14.720 [main] DEBUG a.c.d.p.p.DefaultTestResultAccumulator - Received test result 'Failed(results=[{attribute=body, description=Actual body '�name-1��|A�������?�street nameItem-1Item21' is not equal to the expected body '�name-1��|A�������?�street nameItem-1Item-2', identifier=/, diff=null, interactionId=857a381662884d96adb1cb05bc50788bf9d1ed65}], description=Body had differences)' for Pact order-provider-avro-consumer and Order Created (Pact Broker <http://localhost:9292/pacts/provider/order-provider/consumer/avro-consumer/pact-version/d5c261bfc261542df7e0d1f0ba81810ef94bb192/metadata/c1tdW2N2XT05>)
09:42:14.723 [main] DEBUG a.c.d.p.p.DefaultTestResultAccumulator - Number of interactions #1 and results: [Failed(results=[{attribute=body, description=Actual body '�name-1��|A�������?�street nameItem-1Item21' is not equal to the expected body '�name-1��|A�������?�street nameItem-1Item-2', identifier=/, diff=null, interactionId=857a381662884d96adb1cb05bc50788bf9d1ed65}], description=Body had differences)]
09:42:14.724 [main] DEBUG a.c.d.p.p.DefaultTestResultAccumulator - All interactions for Pact order-provider-avro-consumer have a verification result
09:42:14.724 [main] WARN  a.c.d.p.p.DefaultTestResultAccumulator - Skipping publishing of verification results as it has been disabled (pact.verifier.publishResults is not 'true')
09:42:14.725 [main] DEBUG a.c.d.p.p.j.PactVerificationStateChangeExtension - afterEach for interaction 'Order Created'

Ali Ustek

03/14/2023, 11:20 AM

I've found the problem with my actual body and fixed it using trial and error. However, I don't see logs from the Avro plugin, currently it's using the

MessageAndMetadata

target and using matcher

<http://au.com|au.com>.dius.pact.core.matchers.Matching#matchBodyContents

not the matcher from the plugin what am I missing?

Ali Ustek

03/14/2023, 11:22 AM

https://github.com/austek/pact-avro-plugin/blob/feat/e2e_tests/modules/examples/pr[…]va/com/collibra/example/pulsar/avro/PactPulsarProducerTest.java

Yousaf Nabi (pactflow.io)

03/14/2023, 12:05 PM

can you try with trace logs? if it doesn’t find the plug-in or there is an issue it falls back to the pact framework for matching

Ali Ustek

03/14/2023, 4:23 PM

I don't see test trying to load the plugin at all

provider_test.log

Ali Ustek

03/14/2023, 4:25 PM

on consumer side following are needed,

Ali Ustek

03/14/2023, 4:25 PM

Copy code

Map.entry("pact:avro", schemasPath),
                                Map.entry("pact:record-name", "Order"),
                                Map.entry("pact:content-type", "avro/binary"),

Ali Ustek

03/14/2023, 4:25 PM

do I need to populate them on provider side too>

uglyog

03/16/2023, 6:05 AM

@Ali Ustek I have released Pact-JVM 4.5.3 will a fix for your issue

👌 1

🙏 1

gratitude thank you 1

Ali Ustek

03/16/2023, 11:19 PM

Hi @uglyog when running the provider test in Plugins

compareContents

method

request.getPluginConfiguration.interactionConfiguration

has no configuration, did I miss something?

Yousaf Nabi (pactflow.io)

03/21/2023, 4:56 PM

https://github.com/asyncapi/avro-schema-parser

👀 1

Tõnis Ojandu

03/24/2023, 5:32 PM

Hi all, Looking to contribute possibly. We here at Sympower are currently doing our Pulsar Avro async interaction contract testing with a temporary in-house band-aid tooling where we employ Avro Java POJO generators and Jackson JSON serialization. However, ideally we would prefer to move to pure Avro at some point. Especially, since we plan include python solutions to the mix. What is the state of Avro plugin development currently? I see there is some initial activity, but we are not sure how far things are along behind the scenes. We are hesitant at blindly starting any development from scratch, because there can already be something far along. However, if there is something ongoing, then we are open to supporting this effort.

🙌 2

Ali Ustek

11/29/2023, 6:55 AM

@uglyog @Matt (pactflow.io / pact-js / pact-go) Do I need to keep the version of the plugin up-to-date at https://github.com/pact-foundation/pact-plugins/blob/main/repository/repository.index#L752

👋 1

Yousaf Nabi (pactflow.io)

08/19/2024, 2:50 PM

spotted another avro plugin in the wild https://github.com/vootelerotov/avro-schema-as-pact-contract-plugin that makes three avro plugins now • https://github.com/austek/pact-avro-plugin • https://github.com/praveen-em/pact-avro-plugin

👁️ 1

Nathan Deamer

01/08/2025, 4:19 PM

Hi All, I've been using the pact-avro-plugin, but have ran into an issue where I have a single provider to a single consumer relationship - where the consumer can consume 2 different records (Order, Customer) I get this error

Copy code

`body: $ Plugin Avro Schema configuration item with key '65304033840260accc36fc52520ed0ff' is required`

I've built a sample repo showing the problem here: https://github.com/nathandeamer/pact-sample-avro It seems to be because the

metadata.plugins.configuration

in the consumer pact file only gets populated with 1 of the 2 schemas (Disclaimer: This is my first time using avro and pact plugins).