MohammadReza Shahmorady
08/10/2025, 9:12 PMPedro Mázala
08/11/2025, 9:00 AMBasically, we can’t have an operator with more than two inputs.Is this a restriction by your team? What inputs are you using?
The solution that comes to mind is to join the first two topics,
then join the result with the third topic.If all of them come from the same broker, you can comma split the topic names and Flink will read from all of them in a single shot.
MohammadReza Shahmorady
08/11/2025, 9:09 AMPedro Mázala
08/11/2025, 9:23 AMPedro Mázala
08/11/2025, 9:26 AMKafkaSource<OrderEvent> kafkaSource =
KafkaSource.<OrderEvent>builder()
.setBootstrapServers(bootstrapServers)
.setProperties(properties)
.setTopics("a,b,c,d,e,f,g,h")
.setGroupId("###")
.setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.EARLIEST))
.setValueOnlyDeserializer(new EventDeserializerSchema())
.build();
My object OrderEvent may be 8 different types here. I can even achieve this without Kryo to ensure compatibility in the futureMohammadReza Shahmorady
08/11/2025, 9:28 AMPedro Mázala
08/11/2025, 10:13 AMAndreas Bube
08/12/2025, 6:39 AMItemSold, ItemPicked, ItemShipped, etc.
Let's say I need to perform an operation that involves all three of the above. In this case, I would create a union type with an appropriate name, e.g., ItemSoldPickedShippedUnion (naming is hard, right?!). This union type contains all relevant fields from each of the concrete input records.
Then I'd map each record input stream to the union type. For example:
var itemSoldUnionStream = itemSoldStream
.map(ItemSoldPickedShippedUnion::fromItemSold);
var itemPickedUnionStream = itemPickedStream
.map(ItemSoldPickedShippedUnion::fromItemPicked);
var itemShippedUnionStream = itemShippedStream
.map(ItemSoldPickedShippedUnion::fromItemShipped);
var itemSoldPickedShippedUnionStream = itemSoldUnionStream
.union(itemPickedUnionStream)
.union(itemShippedUnionStream);
// do stuff with itemSoldPickedShippedUnionStream
(I left out calls to .uuid, .keyBy etc for brevity)
I prefer having concrete types, as opposed to generic ones, as it makes the intention of the code more explicit. It's more code and more verbose, but when using Java record types, the boilerplate is kept to a minimum.Andreas Bube
08/12/2025, 6:50 AM