Hi everyone, I have KafkaSource real-time data wh...
# troubleshooting
f
Hi everyone, I have KafkaSource real-time data which I read as DataStreamMyClass. I want to join this stream data with another data which comes from database query (select id, name from mytable where user_id= ...). You can find my below code what I tried. It doesn't print joined dataset to STDOUT. The flink web ui show that joined apply function finishes. How can i join two data and print as STDOUT.
Copy code
KafkaSource<MyClass> kafkasource = KafkaSource.<MyClass>builder()
                .setBootstrapServers("localhost:9092")
                .setTopics("myflink")
                .setStartingOffsets(OffsetsInitializer.latest())
                .setValueOnlyDeserializer(new JsonDeserializationSchema())
                .build();

DataStream<MyClass> text = env.fromSource(kafkasource , WatermarkStrategy.noWatermarks(), "KafkaExactlyOnce");

DataStream<AssetModel> anotherDataStream = env.fromElements(
                new AssetModel("1234567", "Slack"),
                new AssetModel("1234568", "Flink"),
                new AssetModel("1234569", "Spark"));

DataStream<String> joined = kenanStream.join(anotherDataStream)
            .where(KenanBillMapper::getGsm)
            .equalTo(AssetModel::gsm)
            .window(GlobalWindows.create())
            .apply(new JoinFunction<KenanBillMapper, AssetModel, String>(){
                @Override
                public String join(MyClass first, SeconClass second) throws Exception {
                    return first.getGsm() + ","+ first.getBillAmount() + "," + second.myname();
                }
    });

joined.print();
j
Remember,
GlobalWindows
with the default
NeverTrigger
will not produce any output unless a custom trigger is added.
f
Should I have processing time in customer trigger or how can I proceed in this case ?