This message was deleted.

Hi, regarding the first question, the Pulsar function acts just like a consumer subscribed to multiple topics. What specifically is your question here? For question 2 there are some limitations around the type: https://github.com/apache/pulsar/blob/master/pulsar-functions/api-java/src/main/java/org/apache/pulsar/functions/api/Function.java#L27-L30

@Alex Katebi, if your Function subscribes to multiple topics, then each of those topic must have schemas that are compatible with one another, i.e. the same schema. As Alexander pointed out, the Pulsar consumer feeding the Function will attempt to deserialize the incoming messages into the defined input class. If it cannot, then the messages will never be delivered to/processed by the Function

So for the input type one can use json object string and process it with org.json package. In this way I can handle any message types.

Yes, you can use a schema type of String for the topics and perform the deserialization yourself inside the Function.

Initially I was thinking Avro but that is not possible and not as flexible

If you use

byte[]

then your function can consume any message. Maybe that is a path forward for you. I have a number of functions that rely on this. They consume

byte[]

, check the Message properties for a type hint we set, then do deserialization based on the type hint. In our case its protobuf.

If you are using a JSON based schema, then the String schema type will suffice. If you want to use Avro then you will need to use the byte[] schema type and ensure that your functions have access to the Avro library and type definition so they can deserialize the messages internally. You can also use the generic object class as well

I like both suggestions. However I also thought about another. Graalvm has a JavaScript engine. It can run NodeJS applications natively. So I will wrap the Pulsar Functions with JavaScript and run my application logic in JavaScript.

Good luck. That sounds like a lot of work just to deserialize json

I think I will opt for simpler solutions that you guys suggested. I will try to test both solutions and finally pick one. Thanks for your valuable inputs!