https://datahubproject.io logo
Join Slack
Powered by
Hi i'm deploying datahub on kubernetes. Datahub gm...
# all-things-deployment
b
Hi i'm deploying datahub on kubernetes. Datahub gms and frontend are failing. Logs of gms:
Copy code
2021-12-27 04:59:13.599:INFO::main: Logging initialized @4991ms to org.eclipse.jetty.util.log.StdErrLog
WARNING: jetty-runner is deprecated.
         See Jetty Documentation for startup options
         <https://www.eclipse.org/jetty/documentation/>
2021-12-27 04:59:14.192:INFO:oejr.Runner:main: Runner
2021-12-27 04:59:16.986:INFO:oejs.Server:main: jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 1.8.0_302-b08
2021-12-27 05:00:03.685:INFO:oeju.TypeUtil:main: JVM Runtime does not support Modules
2021-12-27 05:00:13.284:WARN:oeja.AnnotationParser:main: Unrecognized ASM version, assuming ASM7
2021/12/27 05:01:14 Command exited with error: signal: killed
s
How did you deploy to Kubernetes? Using this guide https://datahubproject.io/docs/deploy/kubernetes? If yes, can you share your 
values
file for helm (after MASKING sensitive information? What version of DataHub did you deploy?
b
datahub_values
Copy code
# Values to start up datahub after starting up the datahub-prerequisites chart with "prerequisites" release name
# Copy this chart and change configuration as needed.
datahub-gms:
  enabled: true
  image:
    repository: linkedin/datahub-gms
    tag: "v0.8.20"

datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
    tag: "v0.8.20"
  # Set up ingress to expose react front-end
  ingress:
    enabled: false

datahub-mae-consumer:
  image:
    repository: linkedin/datahub-mae-consumer
    tag: "v0.8.20"

datahub-mce-consumer:
  image:
    repository: linkedin/datahub-mce-consumer
    tag: "v0.8.20"

datahub-ingestion-cron:
  enabled: false
  image:
    repository: acryldata/datahub-ingestion
    tag: "v0.8.20.0"

elasticsearchSetupJob:
  enabled: true
  image:
    repository: linkedin/datahub-elasticsearch-setup
    tag: "v0.8.20"

kafkaSetupJob:
  enabled: true
  image:
    repository: linkedin/datahub-kafka-setup
    tag: "v0.8.20"

mysqlSetupJob:
  enabled: true
  image:
    repository: acryldata/datahub-mysql-setup
    tag: "v0.8.20.0"

datahubUpgrade:
  enabled: true
  image:
    repository: acryldata/datahub-upgrade
    tag: "v0.8.20.0"
  noCodeDataMigration:
    sqlDbType: "MYSQL"

global:
  graph_service_impl: elasticsearch
  datahub_analytics_enabled: true
  datahub_standalone_consumers_enabled: false

  elasticsearch:
    host: "elasticsearch-master"
    port: "9200"

  kafka:
    bootstrap:
      server: ""
    zookeeper:
      server: ""
    ## For AWS MSK set this to a number larger than 1
    # partitions: 3
    # replicationFactor: 3
    schemaregistry:
      url: ""
      # type: AWS_GLUE
      # glue:
      #   region: us-east-1
      #   registry: datahub

  neo4j:
    host: "prerequisites-neo4j-community:7474"
    uri: "<bolt://prerequisites-neo4j-community>"
    username: "neo4j"
    password:
      secretRef: neo4j-secrets
      secretKey: neo4j-password

  sql:
    datasource:
      host: ""
      hostForMysqlClient: ""
      port: ""
      url: ""
      driver: "com.mysql.cj.jdbc.Driver"
      username: ""
      password:
        secretRef: mysql-secrets
        secretKey: mysql-root-password

  datahub:
    gms:
      port: "8080"
    mae_consumer:
      port: "9091"
    appVersion: "1.0"

#  hostAliases:
#    - ip: "192.168.0.104"
#      hostnames:
#        - "broker"
#        - "mysql"
#        - "elasticsearch"
#        - "neo4j"

##  Add below to enable SSL for kafka
#  credentialsAndCertsSecrets:
#    name: datahub-certs
#    path: /mnt/datahub/certs
#    secureEnv:
#      ssl.key.password: datahub.linkedin.com.KeyPass
#      ssl.keystore.password: datahub.linkedin.com.KeyStorePass
#      ssl.truststore.password: datahub.linkedin.com.TrustStorePass
#      kafkastore.ssl.truststore.password: datahub.linkedin.com.TrustStorePass
#
#  springKafkaConfigurationOverrides:
#    ssl.keystore.location: /mnt/datahub/certs/datahub.linkedin.com.keystore.jks
#    ssl.truststore.location: /mnt/datahub/certs/datahub.linkedin.com.truststore.jks
#    kafkastore.ssl.truststore.location: /mnt/datahub/certs/datahub.linkedin.com.truststore.jks
#    security.protocol: SSL
#    kafkastore.security.protocol: SSL
#    ssl.keystore.type: JKS
#    ssl.truststore.type: JKS
#    ssl.protocol: TLS
#    ssl.endpoint.identification.algorithm:
yes using the guide u shared version 0.8.20
s
Are there enough resources in the cluster? Is this some cloud provider or local kubernetes cluster?
b
pod is getting enough resources
Hi does this mean its not getting enough resources or allocated resources or not enough Last State: Terminated Reason: OOMKilled Exit Code: 255 Started: Mon, 27 Dec 2021 122808 +0530 Finished: Mon, 27 Dec 2021 123040 +0530
s
It means memory is not enough
b
how can increase the memory through values
s
b
Frontend is up after increasing the resources.
s
Can you share 
kubectl get pods
output?
b
Still backed is not up . Its not showing any error in logs and in running from long time
s
This seems to be restarting. Can you check the pod events to see what is happening?
b
no logs after this line
Copy code
ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.8ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.8ANTLR Tool version 4.5 used for code generation does not match the current runtime version 4.8ANTLR Runtime version 4.5 used for parser compilation does not match the current runtime version 4.8%
s
How much memory are you running GMS with? Can you up that to 1GB and try running it again?
b
512
its working now thanks
3 Views
Open in Slack
PreviousNext
Powered by