@early-lamp-41924 Looks like there is a problem with the latest helm chart: https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/values.yaml#L71 That image does not exist (the latest I see is 0.8.28.1). Same deal with datahub-upgrade...

Hello. I am deploying datahub for Pinterest. I am trying to run

datahub docker quickstart --quickstart-compose-file ./docker/quickstart/docker-compose-without-neo4j.quickstart.yml

in my devserver and replace each dependencies to Pinterest infra. The mysql was successful The elasticsearch will be replace to OpenSearch

datahub-gms:
    container_name: datahub-gms
    environment:
    - ELASTICSEARCH_HOST=<http://vpc-xxxxx.amazonaws.com|vpc-xxxxx.amazonaws.com>
    - ELASTICSEARCH_PORT=443
    - ELASTICSEARCH_USE_SSL=true
    - ELASTICSEARCH_PROTOCOL=https
    - GRAPH_SERVICE_IMPL=<http://vpc-xxxxx.amazonaws.com|vpc-xxxxx.amazonaws.com>

Then I am getting errors like

2022/03/10 05:05:28 Received 200 from <https://vpc-xxxxx.amazonaws.com:443>
2022/03/10 05:05:29 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s
2022/03/10 05:05:29 Problem with dial: dial tcp: lookup broker on 127.0.0.11:53: no such host. Sleeping 1s
2022/03/10 05:05:30 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s
2022/03/10 05:05:31 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s
2022/03/10 05:05:32 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s
2022/03/10 05:05:33 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s
2022/03/10 05:05:33 Problem with dial: dial tcp 169.254.254.231:29092: connect: no route to host. Sleeping 1s
2022/03/10 05:05:34 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s
2022/03/10 05:05:34 Problem with dial: dial tcp 169.254.254.231:29092: connect: connection refused. Sleeping 1s
2022/03/10 05:05:35 Problem with request: Get "http:": http: no Host in request URL. Sleeping 1s

Do you know how to investigate the issue? Thanks!

Hi We are getting random Read timeout during mae, mce, gms builds. We have set below values in gradle.properties.

Hi For building datahub-frontend we need to whitelist the repository urls on our proxy. In the search i was able to find below urls. Please let me know if i am missing some and also do we need to whitelist all of these.

airflow.demo.datahub
ant.design
<http://company.looker.com|company.looker.com>
<http://crunchconf.com|crunchconf.com>
<http://datahubproject.io|datahubproject.io>
<http://developer.okta.com|developer.okta.com>
<http://developers.google.com|developers.google.com>
<http://github.com|github.com>
<http://graph.microsoft.com|graph.microsoft.com>
<http://jestjs.io|jestjs.io>
<http://joplinapp.org|joplinapp.org>
login.microsoftonline
<http://on.cypress.io|on.cypress.io>
<http://registry.npmjs.org|registry.npmjs.org>
<http://registry.yarnpkg.com|registry.yarnpkg.com>
slack.datahubproject
<http://stackoverflow.com|stackoverflow.com>
superset.demo.datahu
www.analyticsmania.c
<http://www.datahubproject.io|www.datahubproject.io>
<http://www.npmjs.com|www.npmjs.com>
<http://www.robotstxt.org|www.robotstxt.org>

Hi Guys. I'm trying to work with you Java Emmiter lib. Is it possible to send MCE event with several aspects? In your documentation https://datahubproject.io/docs/metadata-integration/java/as-a-library#usage example with only one aspect, but I want to push several, for instance, ownership and dataset properties. didn't find any option in the source code

helm noob question, is it possible to install Datahub via helm without referencing the datahub-prerequisite chart? (meaning, the supporting components will be separately created)

just to check on my understanding: if we're ingesting stuff via rest endpoint only, is MCE consumer pod needed? (I set MAE-consumer and MCE-consumer in GMS config to false) seems like datasets still gets created and indexed without the MCE consumer

Hello. I failed to set up cp-schema-registry correctly. What I was doing was run confluentinc/cp-schema-registry:5.4.0 with zookeeper setting. Then I tailed the docker logs

[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.5.6-c11b7e26bc554b8523dc929761dd28808913f091, built on 10/08/2019 20:18 GMT
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=<http://schema-registry-test-0a02a454.ec2.pin220.com|schema-registry-test-0a02a454.ec2.pin220.com>
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_212
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Azul Systems, Inc.
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/zulu-8-amd64/jre
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/etc/confluent/docker/docker-utils.jar
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=5.4.0-1041-aws
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=root
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/root
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=230MB
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=3456MB
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=234MB
[main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181 sessionTimeout=40000 watcher=io.confluent.admin.utils.ZookeeperConnectionWatcher@6f539caf
[main] INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
[main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
[main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
[main-SendThread(zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper-dynamic-schema-registry-dev-0a0c00b7/10.12.0.183:2181. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /10.2.164.84:33680, server: zookeeper-dynamic-schema-registry-dev-0a0c00b7/10.12.0.183:2181
[main-SendThread(zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server zookeeper-dynamic-schema-registry-dev-0a0c00b7/10.12.0.183:2181, sessionid = 0x500acfbbd7b5afa, negotiated timeout = 40000
[main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x500acfbbd7b5afa closed
[main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x500acfbbd7b5afa
[main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181 sessionTimeout=40000 watcher=io.confluent.admin.utils.ZookeeperConnectionWatcher@799f7e29
[main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
[main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
[main-SendThread(zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper-dynamic-schema-registry-dev-0a0c00b7/10.12.0.183:2181. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /10.2.164.84:33682, server: zookeeper-dynamic-schema-registry-dev-0a0c00b7/10.12.0.183:2181
[main-SendThread(zookeeper-dynamic-schema-registry-dev-0a0c00b7:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server zookeeper-dynamic-schema-registry-dev-0a0c00b7/10.12.0.183:2181, sessionid = 0x500acfbbd7b5afb, negotiated timeout = 40000
[main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x500acfbbd7b5afb closed
[main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x500acfbbd7b5afb
[main] ERROR io.confluent.admin.utils.cli.KafkaReadyCommand - Error while running kafka-ready.
org.apache.kafka.common.errors.TimeoutException: Timed out waiting for Kafka to register brokers in Zookeeper. timeout (ms) = 40000

Do you know what is the problem? Thanks

This message was deleted.

Hello today I am switching to my own MySQL instance and see an error Here is the diff

git diff
diff --git a/docker/quickstart/docker-compose-without-neo4j.quickstart.yml b/docker/quickstart/docker-compose-without-neo4j.quickstart.yml
index a3614d602..7a13a1fef 100644
--- a/docker/quickstart/docker-compose-without-neo4j.quickstart.yml
+++ b/docker/quickstart/docker-compose-without-neo4j.quickstart.yml
@@ -59,14 +59,14 @@ services:
     - ${HOME}/.datahub/plugins:/etc/datahub/plugins
   datahub-gms:
     container_name: datahub-gms
-    depends_on:
-    - mysql
     environment:
     - DATASET_ENABLE_SCSI=false
-    - EBEAN_DATASOURCE_USERNAME=datahub
-    - EBEAN_DATASOURCE_PASSWORD=datahub
-    - EBEAN_DATASOURCE_HOST=mysql:3306
-    - EBEAN_DATASOURCE_URL=jdbc:<mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8>
+    - EBEAN_DATASOURCE_USERNAME=xxx
+    - EBEAN_DATASOURCE_PASSWORD=xxx
+    - EBEAN_DATASOURCE_HOST=<http://schemaregistrydevdb.proxysql.pinadmin.com:3306|schemaregistrydevdb.proxysql.pinadmin.com:3306>
+    - EBEAN_DATASOURCE_URL=jdbc:<mysql://schemaregistrydevdb.proxysql.pinadmin.com:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8>
     - EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver
     - KAFKA_BOOTSTRAP_SERVER=broker:29092
     - KAFKA_SCHEMAREGISTRY_URL=<http://schema-registry:8081>
@@ -125,30 +125,30 @@ services:
     - KAFKA_BOOTSTRAP_SERVER=broker:29092
     hostname: kafka-setup
     image: linkedin/datahub-kafka-setup:${DATAHUB_VERSION:-head}
-  mysql:
-    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin
-    container_name: mysql
-    environment:
-    - MYSQL_DATABASE=datahub
-    - MYSQL_USER=datahub
-    - MYSQL_PASSWORD=datahub
-    - MYSQL_ROOT_PASSWORD=datahub
-    hostname: mysql
-    image: mysql:5.7
-    ports:
-    - 3306:3306
-    volumes:
-    - ../mysql/init.sql:/docker-entrypoint-initdb.d/init.sql
-    - mysqldata:/var/lib/mysql
   mysql-setup:
     container_name: mysql-setup
-    depends_on:
-    - mysql
     environment:
-    - MYSQL_HOST=mysql
+    - MYSQL_HOST=<http://schemaregistrydevdb.proxysql.pinadmin.com|schemaregistrydevdb.proxysql.pinadmin.com>
     - MYSQL_PORT=3306
-    - MYSQL_USERNAME=datahub
-    - MYSQL_PASSWORD=datahub
+    - MYSQL_USERNAME=xxx
+    - MYSQL_PASSWORD=xxx
     - DATAHUB_DB_NAME=datahub
     hostname: mysql-setup
     image: acryldata/datahub-mysql-setup:head
diff --git a/metadata-ingestion/src/datahub/cli/docker_check.py b/metadata-ingestion/src/datahub/cli/docker_check.py
index ea3944c10..003aa87ca 100644
--- a/metadata-ingestion/src/datahub/cli/docker_check.py
+++ b/metadata-ingestion/src/datahub/cli/docker_check.py
@@ -11,7 +11,7 @@ REQUIRED_CONTAINERS = [
     "kafka-setup",
     "schema-registry",
     "broker",
-    "mysql",
     "zookeeper",
     # These two containers are not necessary - only helpful in debugging.
     # "kafka-topics-ui",

It works when I use mysql image However, after I switched to my MySQL instance and nuke and restart the quickstart, I saw error like

18:06:08.571 [main] ERROR o.s.web.context.ContextLoader:313 - Context initialization failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'recommendationServiceFactory': Unsatisfied dependency expressed through field 'topPlatformsCandidateSource'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'topPlatformsCandidateSourceFactory': Unsatisfied dependency expressed through field 'entityService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanAspectDao' defined in com.linkedin.gms.factory.entity.EbeanAspectDaoFactory: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.entity.ebean.EbeanAspectDao]: Factory method 'createInstance' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ebeanServer' defined in com.linkedin.gms.factory.entity.EbeanServerFactory: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.ebean.EbeanServer]: Factory method 'createServer' threw exception; nested exception is java.lang.NullPointerException
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639)
	at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
	at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401)
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292)
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:921)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:554)
	at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:888)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:357)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1443)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1407)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:821)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:276)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:106)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:106)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.server.Server.start(Server.java:407)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:106)
	at org.eclipse.jetty.server.Server.doStart(Server.java:371)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	at org.eclipse.jetty.runner.Runner.run(Runner.java:520)
	at org.eclipse.jetty.runner.Runner.main(Runner.java:565)

Do you know what was going on? Thanks!

Hello here, Have you guys some tips to build the all project ? I mean it's a bit difficult from my POV to take the sources and build it. With a lot a modules, different languages and so, the jdkversion, the node version, the gradle version... Anyway i was wondering if you have some inputs or tutorial to do that properly, all that in a CI/CD pipeline purpose. Thanks for your inputs 🙂

Hello everyone: Just curious whether anyone has successfully built the docker image on AWS EC2 instances? I have had the image successfully built and running at local. But building the image on EC2 will always lead the following error:

#12 158.7 [main] INFO com.linkedin.pegasus.generator.DataTemplateGeneratorCmdLineApp - Generating 32 files
#12 158.9
#12 158.9 FAILURE: Build failed with an exception.
#12 158.9
#12 158.9 * What went wrong:
#12 158.9 Execution failed for task ':datahub-web-react:yarnInstall'.
#12 158.9 > Process 'command 'yarn'' finished with non-zero exit value 131
#12 158.9
#12 158.9 * Try:
#12 158.9 Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
#12 158.9
#12 158.9 * Get more help at <https://help.gradle.org>
#12 158.9 Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
#12 158.9 Use '--warning-mode all' to show the individual deprecation warnings.
#12 158.9 See <https://docs.gradle.org/6.9.2/userguide/command_line_interface.html#sec:command_line_warnings>
#12 158.9 17 actionable tasks: 17 executed
#12 158.9
#12 158.9
#12 158.9 BUILD FAILED in 2m 38s
------
executor failed running [/bin/sh -c cd datahub-src && ./gradlew :datahub-frontend:dist -PenableEmber=${ENABLE_EMBER} -PuseSystemNode=${USE_SYSTEM_NODE} -x test -x yarnTest -x yarnLint     && cp datahub-frontend/build/distributions/datahub-frontend.zip ../datahub-frontend.zip     && cd .. && rm -rf datahub-src && unzip datahub-frontend.zip]: exit code: 1

I also created an issue on GitHub: https://github.com/datahub-project/datahub/issues/4434 Any advices or insights is appreciated!

👋 Hello, team! I would like to add additional SQL Alchemy Dialect for Vertica - to be able to work with QuickStart Image and define automatic ingestion. I can do it from CLI command line, but I would like to have it in GUI. Is there a way to do so without rebuilding image:I did

# python3 -m pip install 'acryl-datahub[sql-profiles]'

# python3 -m pip install 'acryl-datahub[sqlalchemy]'

# python3 -m pip install sqlalchemy-vertica-python

And manage to execute the ingestion of object into Vertica. However it is not working in GUI - error is that Vertica-Python module cannot be found. Is there a way to add this module without rebuilding the docker image? 1> I woudl also like to execute SQL Profiling for SQL alchemy - but when I add profile.enable = True - I am gettig error. Any suggestions? Thanks

I've a question about configuration: When investigating the login mechanisms and how to secure DataHub, I found following setting in datahub-gms: # Encryption of DataHub Secrets # SECRET_SERVICE_ENCRYPTION_KEY=<your-AES-encryption-key> How and where this key is used (just to understand, I don't need the technical details)? And should I provide just a long random string (like a password) or is there anything else to consider?

noob question while learning to use helm: where should the imagePullSecrets definition to pull all the docker images from a (private repository) be specified?

Just curious, is it possible to run datahub without confluent-schema-registry?

I'm working with a team on installing DataHub in AWS following the documentation, and when trying to expose the frontend on an ALB we're getting this error:

Error: UPGRADE FAILED: error validating "": error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend.service.port.number): invalid type for io.k8s.api.networking.v1.ServiceBackendPort.number: got "string", expected "integer", ValidationError(Ingress.spec.rules[0].http.paths[0].backend.service): missing required field "name" in io.k8s.api.networking.v1.IngressServiceBackend]

We copied the ingress portion from the values.yml in this section: https://datahubproject.io/docs/deploy/aws#expose-endpoints-using-a-load-balancer Anyone run into this or know a solution?

hello all, currently i'm running datahub locally using docker and the quick start guide, i ingested my metadata and configured the rest. Now how can I deploy it so that my users can access it directly through a URL without additional configuration and of course without losing the metadata that I ingested ?

Hello I am working on setting up OAuth. We have Envoy webservice to authenticate the incoming and outgoing traffic. The username is passed to datahub via HTTP request header and cookies after a successful/cached login. For example, Header: x-forwarded-user=ben Cookie: OAuthUsername=ben Any suggestion on how to integrate with datahub web ui? Thanks

The datahub helm chart makes Frontend and GMS as separate deployments im wondering if anyone has tried to colocate both containers in the same pod, ie, FE as the primary container and GMS as a sidecar? I'm thinking of trying it out because occasionally, I get people complaining they can't login, and the logs in frontend suggest DNS lookup isn't successful at that point in time. And its a randomly happening issue.

hello,i have a question. how to add? I can't add an option

Has someone managed to deploy Datahub with Traefik reverse proxy? I am getting following errors when I am trying to login into the page:

Failed to log in! SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

Hello, I have a question regarding the kubernetes (AWS EKS) deployment, I followed the quickstart guideline from here, and use the command:

helm install prerequisites datahub/datahub-prerequisites

to install dependencies, but I keep on receiving errors from pods, is anyone know about this error or could help with that? Thanks a lot in advance.

Hi, a question on the DataHub Rest sink. https://datahubproject.io/docs/metadata-ingestion/sink_docs/datahub/#config-details We're using a self-signed certificate on our load balancer. It's al internal, so that's good enough for us. However, the recipe fails because of it. I cannot find any option to ignore the Certificate check, or to point to a trusted cert to verify against. Any ideas?

Hello! I’m curious if anyone has tried to deploy DataHub on Google Cloud Run. Is this possible or do I need to deploy using GKE?

I have two questions regarding the acryl-datahub python package on PyPI: • How can I build the package by myself from the DataHub source on Github • Why are there sub version like 0.8.31.3 or 0.8.31.5 with no correspoinding tag on Github and when/why are these sub versions generated and uploaded to PyPI? I have not found any information about this in the documentation.

Hey Team, on our deployed datahub we are unable to see most popular , recently viewed data, domains on homepage and also weekly active users are not updating on analytics page . What can be the reason (or have any extra configs to enable this) cc @gifted-kite-59905

Hi team, Should I use root user for mysql? I'm using Datahub via helm chart. If I create new mysql user and modify values.yaml and then deploy, is there any problem? https://github.com/acryldata/datahub-helm/commit/5b1c651d525f4c37bc737f4f8eef65511f7f38b5#diff-2af07fde488e4606b652c7[…]6c085bee6135afb3d26b696eaR90

Hi everyone, Is DataHub affected by this Spring Framework vulnerability? Thank you!

Hello I have a problem when I use this way to nuke all the metadata. 1. drop table metadata_aspect_v2 and drop table metadata_index. Run docker/mysql/init.sql in my mysql instance 2. delete all indices in my openserarch instance. Then run elasticsearch-setup upon it. 3. Restart the datahub-gms