Hi folks, i am already running starrocks on k8s where i use the broker load job to write data into a primary key table which is working really well for me
I have a new usecase where i want to create a realtime primary key table for which i was exploring:
• routine load
• stream load
• flink connector
• kafka connect
i am getting columns from different sources in a single kafka topic, so for an instance lets say i have 5 different producers producing in the same kafka topic and each of the 5 different producers write data for different columns of the table
now the problem i am facing with routine load is that with a single routine load job with all columns mentioned, it writes null for the columns which dont have data in a particular kafka offset/msg, so i tried running multiple routine load jobs on the same kafka each with specified columns from the 5 producers, in this case also when the msg with col1 and col2 is read by routine load job for col3 it writes col3 as null
how can i design this using the above 4 streaming option's
Thanks