Hi team, I'm using flink to write to delta lake tables using the flink-delta-connector. Does anyone know how to increase the commit frequency so that files can be written more frequently? The only way now as I'm aware is to decrease the commit interval. Are there other ways ?

Apart from decrease the commit interval you can try these 2 approach 1. Reduce watermarks interval

env.getConfig().setAutoWatermarkInterval(<value>);

2. Delta Lake allows you to set a target file size, and Flink can be configured to commit when the data size reaches this target.

tableProperties.put("delta.targetFileSize", "<number>");