how do I add secrets via environment variable to a flink native operator? I have this in my FlinkDeployment:

spec:
 flinkConfiguration:
   metrics.reporter.datadog.apikey: ${DATADOG_API_KEY}
[...]
          - name: DATADOG_API_KEY
            valueFrom:
              secretKeyRef:
                name: {{ .Values.flink.ddApiKey.secretName }}
                key: ddApiConfigString

but I get this exception: [] - Could not instantiate MetricReporter datadog. Reporting might not be exposed. java.lang.IllegalArgumentException: API key: ${DATADOG_API_KEY} is invalid

The Flink Kubernetes Operator won't substitute the '${}' variables, so DataDog is getting the value "${DATADOG_API_KEY}". Did you try to create a secret like this?

apiVersion: v1
kind: Secret
metadata:
  name: datadog-api
type: Opaque
stringData:
  DATADOG_API_KEY: "<your-real-key>"

spec:
  containers:
    - name: flink-main-container
      env:
        - name: DATADOG_API_KEY
          valueFrom:
            secretKeyRef:
              name: datadog-api
              key: DATADOG_API_KEY

You could then use env.java.default-opts.jobmanager and env.java.default-opts.taskmanager (or env.java.opts.all) in the config yaml like this:

env.java.opts.all: "-Dmetrics.reporter.dghttp.apikey=${DATADOG_API_KEY}"

thanks, I'll keep that in mind! my

env.java.opts.all

wasn't working due to a malformed entry. so anything prefixed with a

-D

entry will also be considered for the

config.yaml

in Flink?

I did a search for an example config.yaml with env.java.opts.all being used:

spec:
  flinkConfiguration:
    jobmanager.memory.process.size: "2048m"
    taskmanager.memory.process.size: "8192m"

    env.java.opts.all: >
      -Dservice.name=my-flink-service
      -Denv=${ENVIRONMENT}
      -Dfeature_x_enabled=true

hmm it seems to work for job and task manager, but not for the flink kubernetes operator pod (the one that does the auto scaling). I tried FLINK_PROPERTIES, env.java.opts.all, setting it in a config file but it doesn't get picked up by the operator pod 😞