Good day, I had a question regarding launching DataHub locally. I am trying to switch some ports around using the dockerfile so that no ports conflict with the ones I'm currently using with my existing project, but when I edit the dockerfile everything crashes. Does anyone have a port modification guide using docker-compose? Thanks!

Hi team We have upgraded the datahub to 0.8.39 version but somehow the datahub -mae-consumer is giving error Like application failed to start: Parameter 1 of constructor in com.LinkedIn.metadata.Kafka.hook.siblings.siblingassocuateHook required a bean named Entutyaspectdeo that could be found

Hello, I wanna ask a question. Is there any reference to connect Datahub into Elasticsearch cluster? I tried to change the ELASTICSEARCH_HOST env to array ["es-host-1:9200", "es-host-2:9200", "es-host-3:9200"] but it didn't work, I think the syntax is incorrect. (Fyi, I deploy Datahub on Kubernetes)

Hello guys, is there a way to integrate Slack with Datahub? Asking so users can search through slack channel/chat and get the result link pointing to Datahub as a response?!? Any help or tip is appreciated, thanks!

Hi all, we've recently ramped up traffic to our Datahub deployment and are seeing some issues handling the load in several dimensions(ingestion from the kafka emitter has relatively high latency, UI is starting to get a bit laggy, and we sometimes get errors in the UI loading large lineage graphs). Before we start digging into the various components and where things are getting bottlenecked, I was curious to hear from the community on how they have scaled their deployments and what issues they had to work through as load increased. Any feedback/insight is appreciated!

Is there a doc for automating infra deployment using terraform?

Good day, I'm having issues with the docker-compose file while following the quickstart guide; If I run the command

datahub docker quickstart

everything runs fine, but if I instead run

docker-compose up -d

with the attached file my ui becomes all wonky and unsuable. Has anypne ever experienced this?

Hi all, is there anyone meeting the following problem and knowing how to handle? Execution failed for task 'datahub web reactyarnInstall'.

Process 'command 'xx/datahub/datahub-web-react/.gradle/yarn/yarn-v1.22.0/bin/yarn'' finished with non-zero exit value 1

1.The following issue happened. Should I install scala manually? Or the command " ./gradlew build" will install it automatically?

Execution failed for task ':datahub-frontend:compileScala'.
> Compilation failed

2. After I install scala manually, it apprears the following info:

[Error] /datahub/datahub-frontend/build/src/play/routes/router/Routes.scala:23: type CentralLogoutController is not a member of package controllers
[Error] /datahub/datahub-frontend/build/src/play/routes/router/Routes.scala:27: type TrackingController is not a member of package controllers
[Error] /datahub/datahub-frontend/build/src/play/routes/router/Routes.scala:40: type CentralLogoutController is not a member of package controllers
[Error] /datahub/datahub-frontend/build/src/play/routes/router/Routes.scala:44: type TrackingController is not a member of package controllers

hello, I'm trying to count total number of user logins to the datahub. As I know .ds-datahub_usage_event-00000x file contains user activity events. I can get all events before 27th June 2022. However, events created after 27th June are not found in the elasticsearch db. Do new events stored into somewhere different place?

Hi all, I'm trying to understand the stability of datahub in production. It seems to have many infra dependencies, and I'm curious to know how frequently one would conduct maintenance/debugging in this?

Hi datahub community, for a POC, we have Datahub installed in Docker on a Linux VM. Currently we get often "out-of-memory" problems without running regular data ingestion. Now I would be interested to know what your experiences are for running Datahub in a stable production environment? Is anyone also using Docker on a VM? Is there any experience for running datahub in Azure AKS?

Hi Team, Do we have grafana dashboard template readily available for datahub-gms ?

Hey again there 🙂 I'm facing still some issues : while deploying on AKS clustter i did a deplloyment on a namespace "ns1" it is workign well, and now i'm trying the installation in a second namespace "ns2" ans i'm stuck : The the seconde prerequisites helm install, the ES masters shoudl be in running but ain't ! Still in pending, no logs on those pods , i don't get the point. If you have any idea by any chance 😕

Now it's time to play with Ingress to play with all that 😉

Hey guys, what is better and why to deploy with AWS managed services and Confluent cloud for kafka or in kubernetis cluster ?

i had use “docker save” and “docker load”，get all the images which datahub need， how can I start datahub offline with no internet？

i use commond “datahub docker quickstart --quickstart-compose-file ./docker-compose-without-neo4j.quickstart.yml” ，it still down the image. i already have this image , but have no internet.

Hi there, I got a deployment question re Acryl Data, hope it’s fine to ask here: Is it possible to deploy Acryl on premise?

Hello, I deployed datahub with docker quickstart and I read datahub-gms does not support SSL - is this true ? How can this be configured with an SSL certificate? to be specific I want that the application on port 9002 works with https - how is this doable?

Does datahub requires Kafka service ?

So, as a first we deploy datahub with datahub docker quickstart. How do we terminate the process? And we should always start using quickstart? Thanks

if i want to enable TLS encryption for MSK kafka between client and broker what i need to change in values.yaml

Datahub frontend is saml service or ws federation endpoint?

Hi team I am facing an issue when I am trying to create new recipe for snowflake or any other source system The page is not getting loaded I ran the restore indices job as well Still no change it is not getting loaded I am using 0.8.38 version

Hello guys, I need the technical architecture digram of datahub, is it possible to find it from somewhere?

While deploying datahub v0.8.38 using helm , services are failing with errors: GMS ERROR:

Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dataHubAuthorizerFactory': Unsatisfied dependency expressed through field 'entityClient'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'javaEntityClientFactory': Unsatisfied dependency expressed through field '_entityService'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'entityAspectDao' defined in com.linkedin.gms.factory.entity.EntityAspectDaoFactory: Unsatisfied dependency expressed through method 'createEbeanInstance' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanServer' defined in com.linkedin.gms.factory.entity.EbeanServerFactory: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.ebean.EbeanServer]: Factory method 'createServer' threw exception; nested exception is java.lang.NullPointerException
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)

Datahub Upgrade Error:

022-07-12 07:56:35.388  INFO 1 --- [           main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-07-12 07:56:35.595 ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'upgradeCli': Unsatisfied dependency expressed through field 'noCodeUpgrade'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanServer' defined in class path resource [com/linkedin/gms/factory/entity/EbeanServerFactory.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.ebean.EbeanServer]: Factory method 'createServer' threw exception; nested exception is java.lang.NullPointerException

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanServer' defined in class path resource [com/linkedin/gms/factory/entity/EbeanServerFactory.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.ebean.EbeanServer]: Factory method 'createServer' threw exception; nested exception is java.lang.NullPointerException
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:658) ~[spring-beans-5.3.20.jar!/:5.3.20]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:486) ~[spring-beans-5.3.20.jar!/:5.3.20]
        at

pods status: Prerequisites are running fine

NAME                                                READY   STATUS      RESTARTS        AGE
datahub-acryl-datahub-actions-6598bff46-mdlcw       1/1     Running     6 (4m59s ago)   31m
datahub-datahub-frontend-7dbbd4644f-hqnfx           1/1     Running     0               31m
datahub-datahub-gms-996bbbfc9-2pgf7                 0/1     Running     5 (4m18s ago)   31m
datahub-datahub-upgrade-job-b9jlz                   0/1     Error       0               28m
datahub-datahub-upgrade-job-dp2qb                   0/1     Error       0               30m
datahub-datahub-upgrade-job-dr7p8                   0/1     Error       0               29m
datahub-datahub-upgrade-job-gb9w7                   0/1     Error       0               27m
datahub-datahub-upgrade-job-mxnh2                   0/1     Error       0               24m
datahub-datahub-upgrade-job-r4lv6                   0/1     Error       0               31m
datahub-datahub-upgrade-job-z7r9x                   0/1     Error       0               30m
datahub-elasticsearch-setup-job-pjtpm               0/1     Completed   0               34m
datahub-kafka-setup-job-c99cc                       0/1     Completed   0               34m
datahub-mysql-setup-job-nckpd                       0/1     Completed   0               32m
prerequisites-cp-schema-registry-555899678b-xxt69   1/1     Running     0               4m38s
prerequisites-kafka-0                               1/1     Running     0               39m
prerequisites-mysql-0                               1/1     Running     0               4m34s
prerequisites-zookeeper-0                           1/1     Running     0               4m34s

hi, i am trying to setup datahub on k8's with postgress, need help on it on sql section on values.yml which driver should i use?

Does datahub has the feature for the pod to pod encrypted communication?

Hi, I have quick question about the conection with ElasticSearch. With the property

ELASTICSEARCH_HOST

in the Docker-compose you can indicate the host of one of the masters of ES. A normal ES deployment has more than 1 master so that, in case it falls, the tool is still operative. The problem is that, as in Datahub you can only specify only one master, if that master falls, DataHub users won't be able to do anything until that node recovers from failing. Is there a way to specify 2 hosts of ES so that this problem is solved?? Thanks in advance!