Hey team! Ran into something interesting with checkpoint intervals on the CDC → Iceberg pipeline and wanted to get your take. Our cluster is configured to checkpoint every 30s in

docker-compose

. For this particular pipeline, I wanted a shorter interval, so I added:

execution.checkpointing.interval: 3s

under the

pipeline:

block in the pipeline YAML, assuming the job-level setting would override the cluster-level configuration as it normally does in Flink. However, that didn't happen—the job continued checkpointing every 30s regardless of the value in the YAML. After digging into it, it looks like this isn't really a case of the cluster configuration taking precedence. The Flink CDC pipeline YAML doesn't currently support

execution.checkpointing.interval

as a pipeline option, so the setting is simply ignored. The only way I was able to override it was by passing it through the CLI during submission:

-D execution.checkpointing.interval=3s

While looking into this, I also came across FLINK-34834, which proposes adding a

flink-config:

block to the pipeline YAML to allow pipeline-level overrides of

flink-conf.yaml

. That's essentially what I was trying to achieve here. For reference, our setup: • Flink 2.2.1 (Java 21) • Flink CDC 3.6.0 • MySQL Pipeline Connector 3.6.0-2 • MySQL 8.0 / Driver 8.4.0 • Iceberg 1.11.0 • Nessie 0.108.1 • Trino 482 Has anyone else run into this, or is there a recommended way to configure job-specific Flink properties directly through the CDC pipeline YAML?