Hi folks,
I’m currently working on a POC using Flink for transforming Kafka events.
Our setup involves consuming events from Kafka, applying transformations, and producing the results back to Kafka. Since we have multiple event types with varying structures, I’m evaluating which approach would be best — DataStream API, Table API, or SQL.
At the moment, I’m exploring Flink SQL because it’s easier for end developers and faster to integrate. For each event type, I’ve created a Kafka source table and defined a corresponding view to apply transformations before inserting the results into separate Kafka topics.For more complex transformations, we plan to implement custom UDFs within Flink SQL.
Question: Given this setup, which approach would be most suitable — DataStream API, Table API, or SQL — especially when dealing with multiple event types and varying transformation complexity?