Hi DataHub people! While setting up DataHub Project in our AWS EKS cluster, we're running into an issue we donb't really understand. Following this guide; https://datahubproject.io/docs/deploy/aws, we got the prerequisites running. However, the setup in the datahub helm chart itself gives an issue:

Error: UPGRADE FAILED: failed to create resource: Deployment.apps "datahub-datahub-frontend" is invalid: spec.template.spec.containers[0].env[11].valueFrom.secretKeyRef.key: Required value helm.go:84: [debug] Deployment.apps "datahub-datahub-frontend" is invalid: spec.template.spec.containers[0].env[11].valueFrom.secretKeyRef.key: Required value

What kind of value is it looking for? Is it some helm setting, but I can't seem to find that setting in the helm chart settings: https://artifacthub.io/packages/helm/datahub/datahub Is it looking for EKS cluster settings? Does anyone have a pointer for me, where should I set this setting and what value is it looking for? Thanks! Dennis

Did you set up the secrets according to this guide?: https://datahubproject.io/docs/deploy/kubernetes#quickstart

Hi Ryan, yes, secrets are available and I am referring to them from the helm chart configs.

So it's trying match up to a value in this template: https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-frontend/templates/deployment.yaml Specifically:

spec.template.spec.containers[0].env[11].valueFrom.secretKeyRef.key

and not finding a value to match to the required field. This gets pulled from the secrets. Are you using the default values.yaml?

I was trying to set it up using our own RDS cluster, so I disabled the mysql bit in the prerequisites project values and added this bit in the datahub values:

global:
  sql:
    datasource:
      host: "<http://my-rds-host.eu-central-1.rds.amazonaws.com:3306|my-rds-host.eu-central-1.rds.amazonaws.com:3306>"
      hostForMysqlClient: "<http://my-rds-host.eu-central-1.rds.amazonaws.com|my-rds-host.eu-central-1.rds.amazonaws.com>"
      port: "3306"
      url: "jdbc:<mysql://my-rds-host.eu-central-1.rds.amazonaws.com:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8>"
      driver: "com.mysql.jdbc.Driver"
      username: "my-user"
      password:
        secretRef: mysql-secrets
        secretKey: mysql-root-password

This is specifically for the DataHub Frontend pod, so I don't think that modification to the prerequisites would cause this error: https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-frontend/README.md Based on the number in the array it seems like it's either missing the ElasticSearch authentication secrets or the Metadata Service Auth secrets depending on what you have configured.