Hi all, Flink Kubernetes Operator Hadoop config related. 1. When I submit the Flink job bellow, the TaskManager could not be created due to error: """ Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 3m39s default-scheduler Successfully assigned magnetar/statemachine-taskmanager-1-24 to minikube Warning FailedMount 97s kubelet Unable to attach or mount volumes: unmounted volumes=[hadoop-config-volume], unattached volumes=[hadoop-config-volume flink-config-volume kube-api-access-ft56w flink-logs]: timed out waiting for the condition Warning FailedMount 91s (x9 over 3m39s) kubelet MountVolume.SetUp failed for volume "hadoop-config-volume" : configmap "hadoop-config-statemachine" not found""" 2. The HadoopConfMountDecorator.java (https://github.com/apache/flink/blob/master/flink-kubernetes/src/main/java/org/apa[…]/kubernetes/kubeclient/decorators/HadoopConfMountDecorator.java) says that it would create one, but no error log found in the JobManager. Could you help on this?

apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkDeployment
metadata:
  name: statemachine
  labels:
    type: flink-native-kubernetes
spec:
  image: zhiyut/statemachine:2.0
  # image: flink:1.16
  flinkVersion: v1_16
  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "2"
  serviceAccount: flink
  podTemplate:
    apiVersion: v1
    kind: Pod
    metadata:
      name: pod-template
    spec:
      containers:
        # Do not change the main container name
        - name: flink-main-container
          volumeMounts:
            - mountPath: /opt/flink/log
              name: flink-logs
          env:
            - name: MT_TOKEN
              value: <###>

      volumes:
        - name: flink-logs
          emptyDir: { }
  jobManager:
    resource:
      memory: "2048m"
      cpu: 1
  taskManager:
    resource:
      memory: "2048m"
      cpu: 1
  job:
    jarURI: local:///opt/flink/usrlib/StateMachineExample.jar
    entryClass: org.apache.flink.streaming.examples.statemachine.StateMachineExample
    parallelism: 2
    upgradeMode: stateless

Please don't repost to the channel

The problem is mitigated it by passing an existing ConfigMap to flinkConfiguration

kubernetes.hadoop.conf.config-map.name

. [In Flink Kubernetes Operator] When generating the YAML for JobManager, it also creats the configmap 1. Try to get the configmap for Hadoop a. Flink Kubernetes Operator detects if flinkConfiguration

kubernetes.hadoop.conf.config-map.name

is set, the configmap used. b. It detects the config file in list [$HADOOP_CONF_DIR, $HADOOP_HOME/ect/hadoop, $HADOOP_HOME/conf], and the first existence will be used. c. If configmap is got, add to the YAML of JobManager 2. If the configmap exists, the configmap will be created. [In JobManager] When generating the YAML for TaskManager, 1. Try to get the configmap for Hadoop a. Flink Kubernetes Operator detects if flinkConfiguration

kubernetes.hadoop.conf.config-map.name

is set, the configmap used. b. It detects the config file in list [$HADOOP_CONF_DIR, $HADOOP_HOME/ect/hadoop, $HADOOP_HOME/conf], and the first existence will be used. c. If configmap is got, add to the YAML of TaskManager It is something strange to get the HADOOP config in the operator. For the operator does not care about the HADOOP, on the Flink job itself cares about the HADOOP. Is there some way to get the HADOOP env from the image for both JobManager and TaskManager?? I have a scenario. Each Flink job may access different YARN clusters. So I build separate images for each YARN cluster, and the HADOOP envs are set correctly in the image.