Hi folks, Is it possible to add custom labels in serviceMonitor? I was about to enable Prometheus exporter. The original chart doesn’t have a place to add a label which used to match with our data-agent. cc @bulky-electrician-72362

{{- if and .Values.serviceMonitor.create .Values.global.datahub.monitoring.enablePrometheus -}}
apiVersion: <http://monitoring.coreos.com/v1|monitoring.coreos.com/v1>
kind: ServiceMonitor
metadata:
  name: {{ printf "%s-%s" .Release.Name "datahub-gms" }}
  labels:
    {{- include "datahub-gms.labels" . | nindent 4 }}
  {{- with .Values.serviceMonitor.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  endpoints:
  - port: jmx
    relabelings:
    - separator: /
      sourceLabels:
      - namespace
      - pod
      targetLabel: instance
  selector:
    matchLabels:
      <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
      <http://app.kubernetes.io/name|app.kubernetes.io/name>: datahub-gms
{{- end -}}

Hey @famous-florist-7218, the current templating doesn't support customer labels other than the ones on

gms

Thank you so much blob excited

You can add it to

.datahub-gms.serviceMonitor.annotations

Hi @bulky-electrician-72362, do you know this error? I’m trying to enable ServiceMonitor for datahub-gms and front-end. Here is the value in yaml file.

serviceMonitor:
    create: true
    annotations:
      release: prometheus-operator

Error log:

client.go:521: [debug] Patch Deployment "cdg-acryl-datahub-actions" in namespace data
client.go:521: [debug] Patch Deployment "cdg-datahub-frontend" in namespace data
client.go:521: [debug] Patch Deployment "cdg-datahub-gms" in namespace data
client.go:512: [debug] Looks like there are no changes for Ingress "cdg-datahub-frontend"
client.go:512: [debug] Looks like there are no changes for Ingress "cdg-datahub-gms"
client.go:250: [debug] Created a new ServiceMonitor called "cdg-datahub-frontend" in data

client.go:512: [debug] Looks like there are no changes for ServiceMonitor "cdg-datahub-gms"
upgrade.go:434: [debug] warning: Upgrade "cdg" failed: cannot patch "cdg-datahub-frontend" with kind Service: Service "cdg-datahub-frontend" is invalid: spec.ports[2].name: Duplicate value: "jmx"
Error: UPGRADE FAILED: cannot patch "cdg-datahub-frontend" with kind Service: Service "cdg-datahub-frontend" is invalid: spec.ports[2].name: Duplicate value: "jmx"
helm.go:84: [debug] cannot patch "cdg-datahub-frontend" with kind Service: Service "cdg-datahub-frontend" is invalid: spec.ports[2].name: Duplicate value: "jmx"

hey, this seems like a bug in our helm chart, it double adds jmx to frontend, can you share the values.yaml's frontend part?

@bulky-electrician-72362 sure.

datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
    tag: "v0.9.3"
  labels:
    release: prometheus-operator
  serviceMonitor:
    create: true
    annotations:
      release: prometheus-operator
  # Set up ingress to expose react front-end
  ingress:
    enabled: true
    className: "gateway-internal"
    enabled: true
    annotations:
      {}
      # <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
      # <http://kubernetes.io/tls-acme|kubernetes.io/tls-acme>: "true"
    hosts:
      - host: <http://xxxxxx.com|xxxxxx.com>
        # redirectPaths:
        #   - path: /*
        #     name: ssl-redirect
        #     port: use-annotation
        paths:
          - /
  # OIDC auth based on <https://datahubproject.io/docs/authentication/guides/sso/configure-oidc-react>
  oidcAuthentication:
    enabled: true
    provider: google
    clientId: xxxxxx
    clientSecret: xxxxxx
    # only needed if you would like to store the client secret in secret
    # clientSecretRef:
    #   secretRef: auth-oidc-secret
    #   secretKey: auth-oidc-client-secret
    extraEnvs:
      - name: AUTH_OIDC_EXTRACT_GROUPS_ENABLED
        value: "true"
  resources:
    # We usually recommend not to specify default resources and to leave this as a conscious
    # choice for the user. This also increases chances charts run on environments with little
    # resources, such as Minikube. If you do want to specify resources, uncomment the following
    # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
    limits:
      cpu: 1.0
      memory: 2G
    requests:
      cpu: 0.5
      memory: 1G
  nodeSelector:
    cloud-managed/team: cloud-data
    cloud-managed/capacitytype: on-demand
    cloud-managed/nodetype: high-mem

Are you upgrading from an older version?

Yup, I’ve upgraded from datahub 0.9.1

.Values.global.datahub.monitoring.enablePrometheus

is enabled right?

Yes.

could you run

kubectl get -o yaml svc cdg-datahub-frontend -n <namespace>

to check the list of ports there?

@bulky-electrician-72362

ports:
  - name: http
    nodePort: 32288
    port: 9002
    protocol: TCP
    targetPort: http
  - name: jmx
    nodePort: 32631
    port: 9100
    protocol: TCP
    targetPort: jmx

glad it's working. did you delete port from the service? and let helm update it?

Yes. I did that way.

The metric names seems good, but they won't show up until there is measurment on it. so until there is a get on it