Hi all, I have Flink (v1.17) reading a number of t...
# troubleshootingg
Gordon Murray
07/28/2023, 10:47 AMHi all, I have Flink (v1.17) reading a number of tables from Mariadb (10.6) on AWS RDS, performing some quick aggregations, then I'm sinking data back in to Mariadb and its mostly working well.
However, I can only get INSERT working, It doesn't look I can UPSERT or OVERWRITE with this JDBC Connector (flink-connector-jdbc-3.1.0-1.17.jar).
I'd like to update or replace data in the Mariadb sink and not add new records. Can anyone recommend an approach here?
g
Giannis Polyzos
07/28/2023, 11:32 AMhttps://nightlies.apache.org/flink/flink-docs-release-1.17/docs/connectors/table/jdbc/#key-handling
the connector does support upserts
g
Gordon Murray
07/28/2023, 11:35 AMFlink uses the primary key that was defined in DDL when writing data to external databases. The connector operates in upsert mode if the primary key was defined, otherwise, the connector operates in append mode.Ah, Looks like I need to define a primary key. My destination table so far was just 1 field to store a value. I'll update my approach, thank you!
π 1
Gordon Murray
07/28/2023, 1:39 PMIts working well now with a primary key added, records are being updated nicely. Kinda obvious in hindsight π Thank you for the pointer in the right direction.
π 1