https://datahubproject.io logo
Join Slack
Powered by
Quick one, how would I follow <this> if I've previ...
# all-things-deployment
f
Quick one, how would I follow this if I've previously used the public helm chart?
How do I pass these to 
values.yaml
?
datahub-frontend:
...
extraEnvs:
- name: AUTH_OIDC_ENABLED
value: "true"
- name: AUTH_OIDC_CLIENT_ID
value: your-client-id
- name: AUTH_OIDC_CLIENT_SECRET
value: your-client-secret
- name: AUTH_OIDC_DISCOVERY_URI
value: your-provider-discovery-url
- name: AUTH_OIDC_BASE_URL
value: your-datahub-url
Is the easiest way to do this fork the public helm chart?
b
We typically use "extraEnvs" as you have shown above^^
In our values.yaml:
Copy code
datahub-frontend:
    enabled: true
    image:
      tag: "head"
    extraEnvs:
      - name: AUTH_OIDC_ENABLED
        value: "true"
      - name: AUTH_OIDC_CLIENT_ID
        value: <client id> 
      - name: AUTH_OIDC_CLIENT_SECRET
        valueFrom:
          secretKeyRef:
            name: oidc-secret
            key: oidc-secret
      - name: AUTH_OIDC_DISCOVERY_URI
        value: <discovery uri> 
      - name: AUTH_OIDC_BASE_URL
        value: <base url> 
      - name: AUTH_OIDC_SCOPE
        value: <scope> 
      - name: AUTH_OIDC_USER_NAME_CLAIM
        value: <claim>
f
So I'd fork the helm chart and update? I can't patch in-flight?
presumably should pass secrets separately as env variables?
e
All the subcharts have ability to add extra without having to fork the helm charts
The one ^ John posted above is set in the values.yaml file you use to override
So for secrets see the example above, you simply set the secret name and key that you will refer to to get the secret value.
Before running the helm chart you need to manually create k8s secret by using this command
Copy code
kubectl create secret generic oidc-secret --from-literal=oidc-secret=<<secret-value>> --namespace <<namespace>>
If you deployed before without explicitly setting values.yaml
you can create your own values.yaml and input it during helm upgrade by running the following
Copy code
helm upgrade --install -f path-to-your-values.yaml datahub datahub/datahub
Your input values.yaml will overlay on the default values.yaml defined within the chart
So by adding extraEnvs in your input values.yaml, it will be applied in the final rendered charts
f
🙇
Got this error for the yaml:
Copy code
Error: failed to parse values.yaml: error converting YAML to JSON: yaml: line 3: mapping values are not allowed in this context
It doesn't like the extraEnvs, thought it might be the ... but then I get:
Copy code
Error: failed to parse values.yaml: error converting YAML to JSON: yaml: line 4: did not find expected key
e
can you share your values.yaml file? feel free to obfuscate actual values
f
Copy code
datahub-frontend:
  ...
  extraEnvs:
      - name: AUTH_OIDC_ENABLED
        value: "TRUE"
        - name: AUTH_OIDC_CLIENT_ID
          value: XXXX
        - name: AUTH_OIDC_CLIENT_SECRET
          value: XXXX
        - name: AUTH_OIDC_DISCOVERY_URI
          value: <https://accounts.google.com/.well-known/openid-configuration>
        - name: AUTH_OIDC_BASE_URL
          value: <http://localhost:9002>
Looking at it now reckon I should have copied the rest of it as well eh? 😅
Getting the same issue for the same line though:
Copy code
# Values to start up datahub after starting up the datahub-prerequisites chart with "prerequisites" release name
# Copy this chart and change configuration as needed.
datahub-gms:
  enabled: true
  image:
    repository: linkedin/datahub-gms
    tag: "v0.8.16"

datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
    tag: "v0.8.16"
  # Set up ingress to expose react front-end
  ingress:
    enabled: true
  extraEnvs:
    - name: AUTH_OIDC_ENABLED
      value: "TRUE"
        - name: AUTH_OIDC_CLIENT_ID
          value: 123
        - name: AUTH_OIDC_CLIENT_SECRET
          value: 123
        - name: AUTH_OIDC_DISCOVERY_URI
          value: <https://accounts.google.com/.well-known/openid-configuration>
        - name: AUTH_OIDC_BASE_URL
          value: <http://localhost:9002>

datahub-mae-consumer:
  image:
    repository: linkedin/datahub-mae-consumer
    tag: "v0.8.16"

datahub-mce-consumer:
  image:
    repository: linkedin/datahub-mce-consumer
    tag: "v0.8.16"

datahub-ingestion-cron:
  enabled: false
  image:
    repository: linkedin/datahub-ingestion
    tag: "v0.8.16"

elasticsearchSetupJob:
  enabled: true
  image:
    repository: linkedin/datahub-elasticsearch-setup
    tag: "v0.8.16"

kafkaSetupJob:
  enabled: true
  image:
    repository: linkedin/datahub-kafka-setup
    tag: "v0.8.16"

mysqlSetupJob:
  enabled: true
  image:
    repository: acryldata/datahub-mysql-setup
    tag: "v0.8.16"

datahubUpgrade:
  enabled: true
  image:
    repository: acryldata/datahub-upgrade
    tag: "v0.8.16"
  noCodeDataMigration:
    sqlDbType: "MYSQL"

global:
  graph_service_impl: neo4j
  datahub_analytics_enabled: true
  datahub_standalone_consumers_enabled: false

  elasticsearch:
    host: "elasticsearch-master"
    port: "9200"

  kafka:
    bootstrap:
      server: "prerequisites-kafka:9092"
    zookeeper:
      server: "prerequisites-zookeeper:2181"
    ## For AWS MSK set this to a number larger than 1
    # partitions: 3
    # replicationFactor: 3
    schemaregistry:
      url: "<http://prerequisites-cp-schema-registry:8081>"
      # type: AWS_GLUE
      # glue:
      #   region: us-east-1
      #   registry: datahub

  neo4j:
    host: "prerequisites-neo4j-community:7474"
    uri: "<bolt://prerequisites-neo4j-community>"
    username: "neo4j"
    password:
      secretRef: neo4j-secrets
      secretKey: neo4j-password

  sql:
    datasource:
      host: "prerequisites-mysql:3306"
      hostForMysqlClient: "prerequisites-mysql"
      port: "3306"
      url: "jdbc:<mysql://prerequisites-mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8&enabledTLSProtocols=TLSv1.2>"
      driver: "com.mysql.cj.jdbc.Driver"
      username: "root"
      password:
        secretRef: mysql-secrets
        secretKey: mysql-root-password

  datahub:
    gms:
      port: "8080"
    mae_consumer:
      port: "9091"
    appVersion: "1.0"
Error: failed to parse values.yaml: error converting YAML to JSON: yaml: line 19: did not find expected key
Got it working with changed indentation:
Copy code
datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
    tag: "v0.8.16"
  # Set up ingress to expose react front-end
  ingress:
    enabled: false
  extraEnvs:
    - name: AUTH_OIDC_ENABLED
      value: "TRUE"
    - name: AUTH_OIDC_CLIENT_ID
      value: 123
    - name: AUTH_OIDC_CLIENT_SECRET
      value: 123
    - name: AUTH_OIDC_DISCOVERY_URI
      value: <https://accounts.google.com/.well-known/openid-configuration>
    - name: AUTH_OIDC_BASE_URL
      value: <http://localhost:9002>
e
awesome
note you don’t need to copy over the whole values.yaml
you just need values.yaml with
Copy code
datahub-frontend:
  extraEnvs:
    - ...
since it will overlay it against the values.yaml in the chart
f
Haha dw found other issues now 😅
b
@fancy-fireman-15263 What are you seeing?
f
I couldn't test OIDC with localhost for some reason so I'm going to sort the subdomain first (datahub.curve.com) from AWS route 53 to GKE in GCP where I've spun datahub up
b
Got it
Open in Slack
PreviousNext
Powered by