https://datahubproject.io logo
Join Slack
Powered by
# all-things-deployment
  • m
    👋 Hello, team!
  • m
    Hello, I am trying to deploy Datahub on Kubernetes(EKS) and having an issue with exposing the front-end using an ALB. I have followed the isntructions in the documentation to set up the load balancer but when I do a kubectl get ingress, the address is blank. Can someone help on this issue?
  • b
    Hi Team, I am trying to install datahub on a kubernetes cluster --- and I am hitting the following helm certificate error
  • b
    image.png
  • b
    Error: looks like "https://helm.datahubproject.io/" is not a valid chart repository or cannot be reached: Get "https://helm.datahubproject.io/index.yaml": x509: certificate signed by unknown authority
  • c
    Hi team How can we clean up all the metadata from datahub ?
  • b
    Does anyone have an metrics on how much horsepower you need and the performance expectations around having 5 million assets in DataHub? Or any other useful metrics on sizing and perf?
  • r
    What is the minimum elasticsearch version supported by Datahub.
  • b
    can I import all/partial company active directory groups/users to datahub?
  • s
    Hi team, I run datahub docker quickstart local
  • s
    image.png
  • s
  • s
    this address is unaccessable
  • f
    Hello guys a quick one, is it possible to setup datahub user authenitication via JumpCloud?
  • g
    IMG_20220602_205951.jpg
  • r
    kubectl logs prerequisites-kafka-0 -p
    131631.27
    131631.29 Welcome to the Bitnami kafka container
    131631.30 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-kafka
    131631.30 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-kafka/issues
    131631.30
    131631.31 INFO ==> * Starting Kafka setup *
    131631.57 WARN ==> You set the environment variable ALLOW_PLAINTEXT_LISTENER=yes. For safety reasons, do not use this flag in a production environment.
    mkdir: cannot create directory '/bitnami/kafka/config': Permission denied
  • g
    Hello folks. Is there someone using DataHub deployed on OpenStack?
  • r
    😮
  • c
    Hi team I have upgraded the datahub images to 0.8.35 and charts to 0.2.78 version but while doing helm install getting error
  • c
    Anyone @square-activity-64562 @dazzling-judge-80093 @big-carpet-38439 Can help me ?
  • b
    Thank you..
  • f
    Our organization does not allow node port service or ingress in production as we need ssl at the pod level. The best way to do is by adding side car container as proxy to the front end container(I'll be using nginx container). Problem here is that default datahub helm chart does not allow us to add extra containers from values.yaml it only allows for initcontainers. Can someone help on this please?
  • s
    SNAG-1323.png
  • e
    bumping ^
  • e
    Hi, is confluent-schema-registry still a required dependency today?
  • s
    I have tried configuring the values.yaml file follow the instructions in the docs, but I haven't been able to get past the installing prerequisites steps. Based on the output kubectl, it looks like elasticsearch never becomes healthy because each node end up waiting on itself.
    Copy code
    {"type": "server", "timestamp": "2022-08-01T03:24:53,159Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1", "message": "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [elasticsearch-master-0, elasticsearch-master-1, elasticsearch-master-2] to bootstrap a cluster: have discovered [{elasticsearch-master-1}{gFEfwJeySC2UFqqgikaOxg}{vBXThqFsSgu2KjwL9rLW0A}{172.17.0.3}{172.17.0.3:9300}{cdfhilmrstw}]; discovery will continue using [] from hosts providers and [{elasticsearch-master-1}{gFEfwJeySC2UFqqgikaOxg}{vBXThqFsSgu2KjwL9rLW0A}{172.17.0.3}{172.17.0.3:9300}{cdfhilmrstw}] from last-known cluster state; node term 0, last-accepted version 0 in term 0" }
{"type": "server", "timestamp": "2022-08-01T03:24:54,243Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1", "message": "path: /_cluster/health, params: {wait_for_status=green, timeout=1s}",
  • s
    I'm going to try and skip the prerequisites and hope the jobs run as part of datahub install and not prerequisites (as I'm not using any of them).
  • f
    but I dive into 
    datahub-elasticsearch-setup-job
    . Its log is so weird.
    Copy code
    > kl datahub-elasticsearch-setup-job--1-bb5n9
/usr/local/bin/dockerize: line 1: syntax error: unexpected word (expecting ")")
    Note: I deployed DataHub components on EKS which can be mixed ARM and Non-ARM based NodeGroups in the same cluster.
  • c
    Hey @brave-businessperson-3969 and to anyone that might be able to help, I know this was a while ago, but I was wondering if you could provide me the steps you've made to set up datahub with the proxy server? Thanks
  • a
    @bumpy-needle-3184 Thanks for the advice, I’ve already done all this. Actually I need just to enable this https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-frontend/values.yaml#L48 It’s annotation in datahub-frontend helm subchart When I install it as you describe - 
    helm install datahub datahub/datahub
    - I always get this annotation commented out, just like it looks in a file in your repository I’m new to helm, I tried to download the repository https://github.com/acryldata/datahub-helm locally, to a folder on my computer, and install the datahub from there using helm’s local install:
    Copy code
    cd ~/github/datahub-helm-master/charts/datahub
helm install datahub .
    but before it I need to install prerequisites:
    Copy code
    cd ~/github/datahub-helm-master/charts/prerequisites
helm install prerequisites .
    and here I got an error like I described in the head of this thread
    Copy code
    sk-MB:prerequisites kk$ helm install prerequisites .
Error: INSTALLATION FAILED: An error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: elasticsearch, neo4j, neo4j-community, mysql, cp-helm-charts, kafka
sk-MB:prerequisites kk$ helm dependency build
Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "<https://helm.elastic.co>" chart repository
...Successfully got an update from the "<https://confluentinc.github.io/cp-helm-charts/>" chart repository
...Successfully got an update from the "<https://equinor.github.io/helm-charts/charts/>" chart repository
...Successfully got an update from the "<https://neo4j-contrib.github.io/neo4j-helm/>" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "eks" chart repository
...Successfully got an update from the "datahub" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Error: can't get a valid version for repositories kafka. Try changing the version constraint in Chart.yaml
    Maybe this is not the right way? All I need is to install datahub using helm with this annotation enabled https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-frontend/values.yaml#L48
1...4950515253Latest