In k8s flink operator, is there a way to submit a flink session job using cmd, not Rest API? I would like to have 1 job manager with multiple flink batch job submissions (to be specific i am running iceberg compaction). When submitting a batch job using

flinksessionjob

CRD, I am seeing [FLINK-24883] Use flink web ui to submit the jar throw Job client must be a CoordinationRequestGateway. This is a bug exception. Exception occurs here, but TL;DR is that the batch job needs a long-lived client, but flink jobs submitted via Rest API doesn’t have that.

Collecting results require a client to continuously fetching the result till the end or explicitly closes the iterator (causing the job to be cancelled). If client goes away without fetching all the results or terminating the job it will hang forever. Submitting by web UI does not fulfill this requirement (as it does not have a long existing client) so we're not supporting collect in web UI. I'll give out a more proper exception message and refine the document of collect.

If you have a session cluster running, you could try the regular Flink client to submit to it (

bin/flink

) I haven’t tested this bit it should work as long as you can connect to the cluster from the client machine