I’m trying to write a Flink SQL query that is doin...
# random
t
I’m trying to write a Flink SQL query that is doing
UNION ALL
for 80 different (kafka) tables - each one has filters, and different schema. Since the schema is different for each table, I guess I need to add to the
SELECT
of each query -
CAST(NULL AS STRING) AS ..
for each field missing, so all SELECT statements will be aligned with the same types and number of fields - any better way you can think of 🙂 ? I’m wondering if there is a better / readable / best practice way to do it. The purpose of this query is to do stateless operations (filtering only).
d
Have you tried writing all of the topics to single sink directly (after applying filters + type cast) instead of doing union then writing to sink?
t
@Dave Filonuk Not yet. But will I still be able to have a single query? I want to avoid a query for each kafka topic.