In the flink code, I am reading data from a Kafka topic and sinking the results to postgresDB. This works fine.
If now I want to check if there exists an entry in the database for the same message that came in from Kafka and then decide if I want to update the row or insert a new row, how can I achieve this?
s
Soumya Ghosh
03/17/2023, 8:51 AM
Well, one approach could be we use primary key clause in flink jdbc sink DDL.
If you want to update selective columns in target table, then use a lookup join with target table - this can help you identify which records will get inserted and which one get updated.
a
Abhinav sharma
03/17/2023, 9:05 AM
But if I am using a datastream, how can I achieve this?