Howdy again. I am trying to configure the Prometheus integration for Flink. Using the operator, I have defined the following properties in my config. I have also ensured the Prometheus JAR was included in the base image:

flinkConfiguration:

collect-sink.batch-size.max: "8388608"

taskmanager.numberOfTaskSlots: "2"

metrics.reporters: prom

metrics.reporter.prom.factory.class: org.apache.flink.metrics.prometheus.PrometheusReporter

metrics.reporter.prom.port: "9999"

Do I need to use the podTemplate attribute of the FlinkDeployment spec to add the additional port mapping? EDIT I figured it out. The config properties mentioned in the docs were incorrect. And I added an additional container port using the podTemplate param in the spec.

What was the problem with the doc?

@Gyula Fóra I ended up reading this page and didn't realize it was a blog post which is several years old: Flink and Prometheus: Cloud-native monitoring of streaming applications. The Metric Reporters section of the docs has the correct config property name :

metrics.reporter.prom.factory.class

@Zachary Piazza What’s the container port you added? I’m also integrating operator metrics with prometheus, so like to know about it. Thanks!

@RICHARD JOY Sorry didn't see your tag. Here is what I added in my FlinkDeployment spec to enable Prometheus metrics:

---
apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkDeployment
metadata:
  namespace: cdc
  name: cdc
spec:
  flinkVersion: v1_17
  flinkConfiguration:
    metrics.reporter.prom.factory.class: org.apache.flink.metrics.prometheus.PrometheusReporterFactory
    metrics.reporter.prom.factory.port: "9249"
    metrics.reporters: prom
  podTemplate:
    spec:
      containers:
      - name: flink-main-container
        ports:
        - name: prometheus
          containerPort: 9249
...

@Zachary Piazza, Along with this, should we also add prometheus service monitor or pod monitor for each flink application? So that prometheus can scrape the metrics.

@Gyula Fóra can I use prometheus pushgateway with my argocd k8 operator app, the doc say it is pushed with well known metrics reporter of flink, so I just need to append prefix

kubernetes.operator

on this configs: https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/#prometheuspushgateway