Hey all, I have a question regarding scheduling flink jobs in batch mode. For some datasets, we receive files in batches (every hour/day). We can either run a streaming job in application mode and monitor directories in S3 using FileSource. Or we can run a flink cluster in session mode and submit flink jobs periodically. Running a streaming application for batch files is not an efficient use of resources especially for low frequency datasets so we think it is better to run flink cluster in session mode and use it to process multiple batch data sources . Using Flink K8s Operator, we can create flink cluster in session mode using

FlinkDeployment

CRD and session job using

FlinkSessionJob

CRD. But we have to use an external scheduler which can create

FlinkSessionJob

periodically. Wouldn't it be better if we can have another CRD

FlinkSessionCronJob

which can fire up jobs periodically using Kubernetes native scheduler? Or how do you generally schedule flink jobs for batch workloads?