DataHub #all-things-deployment

powerful-cat-68806

05/16/2023, 8:58 AM

Hi team. We have an ongoing issue with Redshift cluster ingestion We faced the error

Copy code

'/usr/local/bin/run_ingest.sh: line 40:  2722 Killed ( datahub ${debug_option} ingest run -c "${recipe_file}" '

several times. Each time, I’ve extended the memory & CPU for

acryl-datahub-actions

pod which came out finally to this enormous config:

Copy code

acryl-datahub-actions:
  enabled: true
  image:
    repository: acryldata/datahub-actions
    tag: "v0.0.7"
  resources:
    limits:
      memory: 120Gi
    requests:
      cpu: 32Gi
      memory: 100Gi

But still failing with similar error I’m using, from the UI, the CLI version

0.10.1.2

I’ve checked also the memory usage for this pod & I don’t see any issue Any idea? Cc: @modern-garden-35830 @icy-controller-68116

✅ 1

clever-magician-79463

05/16/2023, 1:47 PM

Hi All, I had a query. This is the scenario: I created 2 instances of datahub on different machines, call them datahub_1 and datahub_2. The mysql server is hosted on different dbs on RDS, call them datahub_primary and datahub_secondary for the resprective instances. I ingested some data in datahub_1, this data goes into the metadata_aspect_v2 table of datahub_primary db. Now i want to point datahub_2 to use the same metadata. Just copying the metadata_aspect_v2 table from primary to secondary db is not enough? I tried doing it but i still not see the data in datahub_2. Is there some other way of restoring the metadatas? This test was done by deploying both the instances through docker and I was trying to test the back and restore functionality of datahub. Can anyone please suggest why the above result was seen and if there is a better or a official way of restoring metadata and indices. Happy to take this offline if needed. Thanks a lot! Edit: I have tried directly pointing the db of datahub_2 to the same db of datahub_1 but no use. I created different dbs and copied the table but still no use

green-honey-91903

05/16/2023, 5:40 PM

Hey all, does anyone know if the datahub frontend when deployed with posgresql as the backend use an ORM to communicate with postgresql? If so what flavor?

elegant-nightfall-29115

05/16/2023, 10:05 PM

Hello all, I chatted on a few threads in #advice-data-governance, #ingestion, and #troubleshoot and have been told to bring my question here. My goal is to completely dissable UI Ingestion for all users. I tried to do so by setting the env variable

UI_INGESTION_ENABLED

in the GMS to

false

. That did not work. I am running v0.9.5. The exact declaration in the helm chart is below.

Copy code

datahub-gms:
  enabled: true
  image:
    repository: linkedin/datahub-gms
    tag: "v0.9.5"
  extraEnvs:
    - name: UI_INGESTION_ENABLED
      value: "false"

bland-gigabyte-28270

05/17/2023, 9:39 AM

Hi, is it possible to use Datahub with Amazon MSK Serverless? Meaning without Zookeeper.

prehistoric-scooter-11523

05/17/2023, 10:25 AM

hi, Im deploying latest Datahub helm chart into AWS EKS, and I got error in GMS, can someone helm me please to understand where the issue is ?

steep-vr-39297

05/18/2023, 7:32 AM

Hi, team. I have a question. Installation and testing were conducted in local according to the https://datahubproject.io/docs/advanced/monitoring/#enable-monitoring-through-docker-compose guide. I want to get the count metric information that the user searched.

metrics_com_linkedin_metadata_resources_entity_EntityResource_search_Count

I think this key is the one I'm looking for, but it doesn't appear in the search. Which key should I check?

echoing-alligator-27984

05/18/2023, 8:43 AM

Hi team, can I upgrade log4j from 2.17.0 to 2.17.1 in gradle? There are vulnerabilities in log4j 2.17.0 and gradle 7.3+ has fix it, but I cannot upgrade gradle because there are some deprecated methods in current gradle config file.

quick-megabyte-61846

05/18/2023, 9:46 AM

Hey is there any chance to manually trigger this job? https://github.com/acryldata/datahub-helm/blob/335c5812002ca8f3406725d97be47360717[…]datahub/templates/datahub-upgrade/datahub-system-update-job.yml I’m looking to revive my GMS pod, and I’ve found that updating the Helm chart to the latest version gets it working properly. However, after 30 days, the issue resurfaces. I’m wondering if there’s a solution to this problem. For more information, please refer to the link provided.

✅ 1

limited-forest-73733

05/18/2023, 11:52 AM

Hey team I deployed datahub 0.10.2 version. I am integrating airflow with datahub but unable to create connection using datahub_kafka_default. Config file:

chilly-boots-22585

05/18/2023, 12:25 PM

Hello support. I have running AWS EKS. Now i want to install datahub on it using this link. https://datahubproject.io/docs/deploy/aws For that i have created mysql rds and elastic-search and as per the documentation i have changed the values.yaml for datahub but when i run below command i am getting this.

Copy code

helm upgrade --install datahub datahub/datahub --values values.yaml

: lookup https on 10.100.0.1053 no such host. Sleeping 1s 2023/05/18 113800 Problem with request: Get https://datahub-es:dglk4904hghghDDHzD4%21@https<//search-datahub-star-es-mfdqbow3ca.eu-west-1.es.amazonaws.com443> dial tcp: lookup https on 10.100.0.1053 no such host. Sleeping 1s

chilly-boots-22585

05/18/2023, 12:26 PM

One thing do i need to run helm install prerequisites datahub/datahub-prerequisites
if i am using aws managed RDS and opensearch ?

plus1 1

chilly-boots-22585

05/18/2023, 1:06 PM

receiving this error as well and when i check pod

Copy code

helm upgrade --install datahub datahub/datahub --values datahub-values.yaml
Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition

chilly-boots-22585

05/18/2023, 1:06 PM

Problem with request: Get http://datahub-es:dglk4D4%21@https<//search-datahub-starburst-es-m2rippb3ca.eu-west-1.es.amazonaws.com443> dial tcp: lookup https on 10.100.0.1053 no such host. Sleeping 1s 2023/05/18 123444 Problem with request: Get http://datahub-es:dglk490hg4%21@https<//search-datahub-starburst-es-m2rih5h4wppbw3ca.eu-west-1.es.amazonaws.com443> dial tcp: lookup https on 10.100.0.1053 no such host. Sleeping 1s 2023/05/18 123445 Problem with request: Get http://datahub-es:dglk4904hgDDHzD4%21@https<//search-datahub-starburst-es-m2rih5h4fdqbow3ca.eu-west-1.es.amazonaws.com443> dial tcp: lookup https on 10.100.0.1053 no such host. Sleeping 1s 2023/05/18 123445 Timeout after 2m0s waiting on dependencies to become available: [http://datahub-es:dglk4gzD4!@https//search datahub starburst es m24w4afdqbow3ca.eu west 1.es.amazonaws.com443]

chilly-boots-22585

05/18/2023, 2:07 PM

can anyone help me with this issue https://datahubspace.slack.com/archives/CV2UVAPPG/p1684415199085129

✅ 1

stale-traffic-76901

05/18/2023, 2:28 PM

Hey everyone, I’m having this problem, and the docker service is active. Has it happened to you before?, thanks for the support

stale-traffic-76901

05/18/2023, 2:28 PM

logs, empty

bland-gigabyte-28270

05/19/2023, 4:44 AM

Not particularly about deployment, but what is the purpose of schema registry?

bland-gigabyte-28270

05/19/2023, 5:42 AM

Any guidelines on how to run Datahub in production? Which needs to be scaled, for example?

chilly-boots-22585

05/20/2023, 1:25 PM

Hello datahub support i am receiving this error when i change the dathub helm values.yaml with aws rds, aws elasticsearch. Also I am attaching pod logs as well.

helm upgrade --install datahub datahub/datahub --values datahub-values.yaml

Error: UPGRADE FAILED: pre-upgrade hooks failed: timed out waiting for the condition

Below error is when i run command kubectl logs -f datahub-elasticsearch-setup-job-qtthl
2_023/05/20 132357 Problem with request: Get https://datahub-es:dglk4904_4@https<//search-datahub-starburst-es-m2rihdqbow3ca.eu-west-1.es.amazonaws.com443> dial tcp: lookup https on 10.100.0.1053 no such host. Sleeping 1s_ _*2023/05/20 132358 Timeout after 2m0s waiting on dependencies to become available: [https://datahub-es:dglk490HzD_4@https//search datahub starburst es m2rih5h4w4ppba.eu west 1.es.amazonaws.com443]*_

chilly-boots-22585

05/20/2023, 1:26 PM

NOTE: I am not installing prerequisites because i am using AWS Services for it. Do i still need to install prerequisites? Please help me to resolve this issue i am trying to follow up with datahub support team from last 2 days.

cool-kitchen-48091

05/21/2023, 7:37 AM

Hey DataHub team, I am looking to build a support system for our internal entities and for that I would like to integrate some of the DataHub APIs in our domain. The problem starts when requesting HTTP requests I am getting a CORS not allowed from the DataHub host to my host, Both are deployed on a k8s pod. 1. What configurations do I need to add to the helm chart? 2. Where can I find the nginx configuration in the pod? 3. We can’t have ingress enabled in our pods I’ve been told that our organization disabled it but the flag is on, where can I check if it even works? This is what we currently have configured:

Copy code

datahub-frontend:
  enabled: true
  image:
    repository: linkedin/datahub-frontend-react
  ingress:
    enabled: true
    annotations:
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: nginx
      <http://kubernetes.io/cors-allow-origin|kubernetes.io/cors-allow-origin>: '*'
      # <http://kubernetes.io/tls-acme|kubernetes.io/tls-acme>: "true"
    hosts:
      - host: datahub-frontend.*.com
        # redirectPaths:
        #   - path: /*
        #     name: ssl-redirect
        #     port: use-annotation
        paths:
          - /

steep-vr-39297

05/22/2023, 2:23 AM

Hi Team, I use the key of

metrics_com_datahub_graphql_GraphQLController_searchAcrossEntities_Count

, the metrics will be counted when I search, is that correct? My purpose is to find a

metric key

that can check the activity indicator. I would appreciate it if you could tell me other

metric keys

that I can check other than those keys.

echoing-alligator-27984

05/22/2023, 3:08 AM

Hi team, I'm building datahub in an internal environment, it built failed with commons-text:1.8 in the plugin pegasus and 1.8 is not allowed to download due to vulnerabilities. I tried to put a new version in build.gradle in the root path(this works for log4j 2.17.0), but it does not work. I also tried other gradle files, but didn't make it. Does anyone know how to upgrade or exclude the commons-text jar in pegasus? Thank you!

bland-gigabyte-28270

05/22/2023, 7:32 AM

Hi, is there a way to customize the UI (the logo atleast) for DataHub?

✅ 1

chilly-boots-22585

05/22/2023, 1:44 PM

what could be the issue for this error

Copy code

2:38:24,715 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [datahub-frontend/conf/logback.xml] at [file:/datahub-frontend/conf/logback.xml]
12:38:24,810 |-INFO in ch.qos.logback.core.joran.action.TimestampAction - Using current interpretation time, i.e. now, as time reference.
12:38:24,810 |-INFO in ch.qos.logback.core.joran.action.TimestampAction - Adding property to the context with key="bySecond" and value="2023-05-22_12-38-24" to the LOCAL scope
12:38:24,810 |-INFO in ch.qos.logback.core.joran.action.TimestampAction - Using current interpretation time, i.e. now, as time reference.
12:38:24,810 |-INFO in ch.qos.logback.core.joran.action.TimestampAction - Adding property to the context with key="byDate" and value="2023-05-22" to the LOCAL scope
12:38:24,811 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
12:38:24,814 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
12:38:24,844 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - This appender no longer admits a layout as a sub-component, set an encoder instead.
12:38:24,844 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
12:38:24,844 |-WARN in ch.qos.logback.core.ConsoleAppender[STDOUT] - See also <http://logback.qos.ch/codes.html#layoutInsteadOfEncoder> for details
12:38:24,845 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
12:38:24,850 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
12:38:24,851 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
12:38:24,858 |-INFO in ch.qos.logback.core.rolling.FixedWindowRollingPolicy@3e62d773 - No compression will be used
12:38:24,861 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: /tmp/datahub/logs/datahub-frontend-2023-05-22_12-38-24.log
12:38:24,861 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [/tmp/datahub/logs/datahub-frontend-2023-05-22_12-38-24.log]
12:38:24,862 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
12:38:24,862 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
12:38:24,862 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
12:38:24,863 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
12:38:24,863 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@81d9a72 - Registering current configuration as safe fallback point
12:38:25 [main] INFO  o.p.play.store.ShiroAesDataEncrypter - Generated key: 52a53270b47f43a0
May 22, 2023 12:38:25 PM com.google.inject.internal.MessageProcessor visit
INFO: An exception was caught and reported. Message: com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'analytics.elastic.sslContext.sslProtocol'
com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'analytics.elastic.sslContext.sslProtocol'

Oops, cannot start the server.
com.google.inject.CreationException: Unable to create injector, see the following errors:

1) An exception was caught and reported. Message: No configuration setting found for key 'analytics.elastic.sslContext.sslProtocol'
  at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:173)

2) Could not find a suitable constructor in react.analytics.AnalyticsService. Classes must have either one (and only one) constructor annotated with @Inject or a zero-argument constructor that is not private.
  at react.analytics.AnalyticsService.class(AnalyticsService.java:37)
  while locating react.analytics.AnalyticsService
    for the 3rd parameter of react.controllers.GraphQLController.<init>(GraphQLController.java:51)
  while locating react.controllers.GraphQLController
    for the 11th parameter of router.Routes.<init>(Routes.scala:69)
  at play.api.inject.RoutesProvider$.bindingsFromConfiguration(BuiltinModule.scala:121):
Binding(class router.Routes to self) (via modules: com.google.inject.util.Modules$OverrideModule -> play.api.inject.guice.GuiceableModuleConversions$$anon$1)

2 errors
	at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543)
	at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:159)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
	at com.google.inject.Guice.createInjector(Guice.java:87)
	at com.google.inject.Guice.createInjector(Guice.java:78)
	at play.api.inject.guice.GuiceBuilder.injector(GuiceInjectorBuilder.scala:185)
	at play.inject.guice.GuiceBuilder.injector(GuiceBuilder.java:221)
	at play.inject.guice.GuiceApplicationBuilder.build(GuiceApplicationBuilder.java:156)
	at play.inject.guice.GuiceApplicationLoader.load(GuiceApplicationLoader.java:34)
	at play.api.ApplicationLoader$JavaApplicationLoaderAdapter$1.load(ApplicationLoader.scala:87)
	at play.core.server.ProdServerStart$.start(ProdServerStart.scala:51)
	at play.core.server.ProdServerStart$.main(ProdServerStart.scala:25)
	at play.core.server.ProdServerStart.main(ProdServerStart.scala)
Caused by: com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'analytics.elastic.sslContext.sslProtocol'
	at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:156)
	at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:174)
	at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:180)
	at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:180)
	at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:180)
	at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:188)
	at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:193)
	at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:250)
	at react.analytics.AnalyticsServiceModule.createSSLContext(AnalyticsServiceModule.java:83)
	at react.analytics.AnalyticsServiceModule.configure(AnalyticsServiceModule.java:63)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:61)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:344)
	at com.google.inject.spi.Elements.getElements(Elements.java:103)
	at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:173)
	at com.google.inject.AbstractModule.configure(AbstractModule.java:61)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:344)
	at com.google.inject.spi.Elements.getElements(Elements.java:103)
	at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:137)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103)
	... 10 more

chilly-boots-22585

05/22/2023, 1:45 PM

above issue is coming for pod datahub-datahub-frontend-6f975cc84-q55c6

wonderful-book-58712

05/23/2023, 5:58 AM

Hi Team we are planning to upgrade Datahub to latest version of Java . Please let us know if there are learnings on challenges we might face in doing so or best practices to make this effort successful . This we are trying to resolve open vulnerabilities

bland-gigabyte-28270

05/23/2023, 8:04 AM

Hi, where is the secret stored when you put it in Datahub UI? Is it secure, and is there any documentation about secrets?

rich-policeman-92383

05/23/2023, 8:38 AM

Hello With release v0.10.1 ES version < 7.9 are no longer supported. Can someone please confirm on below points: • What version > 7.9 are supported • Do we support ES 8.x