Hi! I need to change the front end of the datahub, by placing my company logo, for example. Is it possible to do this??

Hi, Can I build a datahub-actions image from the source code? If yes, how to do

Hey Community! Just curios to know. I've been able to configure OIDC using Google with Datahub. However, is there a way to configure OIDC with some other providers as well apart from Google,Okta and Azure ? We'll like to do it with WSO2.

Hi guys 🙂 Is there an updated Datahub Grafana Dashboard JSON somewhere? Cus most of the metrics of the JSON in the datahub repository (github) is unavailable in the current version Thanks!

Hello, I am deploying the Datahub on k8s with the helm chart, we want to access existing metadata in postgress. so have used existing postgress. I have pointed it to existing Trino's metastore PG, but it show error reg "metadata_aspect_v2" does not exist. I have ran the PG setup job but I guess by default it goes to "datahub" db only not the db that I have mentioned in global. Any help here will be appreciated

Hello guys, I have a concern about

datahub

default user. I’m looking for a friendly way to secure the DataHub frontend. The guideline was updated earlier this month (ref link) but some unexpected cases may occur. For example: • Let’s say Adam deploys DataHub with default configs just for testing. When accessing the UI, the login page will show up and he can login with the default credential. • After that, he sets up an alternative authentication method, OIDC for example. • The point is: In the next time, the frontend will take Adam to Home Page directly. He will no longer see Login Page. So he might forget to change the default credential. => The bad guy can leverage this problem to access the frontend via datahub user. And by default, its role is really powerful. The data can be stolen. My thoughts: • Update password for datahub user through the UI. • Warning message about potential vulnerabilities. (in case people forget the security issues). • Qualifying task to check security issues during deployment.

Hello! I’m deploying dathub with Helm. The pod running acryldata/datahub-gms:v0.9.6.2 is comes up fine but after inspecting the logs, I see I’m getting a number of GroupAuthorizationExceptions:

org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: mce-consumer-job-client
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: generic-mce-consumer-job-client
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: generic-mae-consumer-job-client
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: generic-platform-event-job-client
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: datahub-usage-event-consumer-job-client

these are all followed by a:

[ThreadPoolTaskExecutor-1] ERROR o.s.k.l.KafkaMessageListenerContainer$ListenerConsumer:140 - Fatal consumer exception; stopping container

Is this OK? I can’t tell if it’s affecting anything as I’m still unable to create or run an ingestion due to the issue referenced here We’re using our own Confluent Kafka but I have no access to the console myself without SRE/DevOps support. Is it possible that we need to permission our Kafka user to access these consumer groups?

Hello, teams. I understand that datahub does not support SAML 2.0, do you have any support plans?

Hi Team, After we successfully started the GMS and Frontend service, me and my team are trying to ingest some tables from a Postgres database. The ingestion job ran fine but we cannot see the tables in the UI. There is one more strange behavior, initially when we started to ingest the data from Postgres we were able to ingest one table, then we created few more tables but are not able to ingest only those tables. We restarted GMS, ran the ingestion job again, but then again we were only able to see the first table that we ingested initially and not the other ones which we created later on, in the UI. Any help please?

Good afternoon, does anyone know if any of the datahub images (gms, frontend or actions) support Docker Secrets?? I just don't think it is a good idea to have as plain text information like ES user/password, PostrgeSQL user/password/connection URL, .... Thanks in advance!!

This message was deleted.

Hi! I cannot fetch docker image

acryldata/datahub-postgres-setup:v0.9.6.1

while installing Datahub v0.9.6.1 from helm charts. Is it really missing or I have a local problem in fetching images? I can fetch v0.9.6.2 for that matter. Any thoughts?

Hi! Is there any guideline on architecture for multi-domain organization?

are users stored in Elasticsearch?

Hi The Datahub actions and ingestion images v0.9.6.1 are having a. Lot of critical and high vulnerabilities and most of them are coming from the pyspark site package resource path: "usr/local/lib/python3.10/site-packages/pyspark/jars/" can we update the pyspark packages or bump up the jars in pyspark site packages which are within this path to resolve the vulnerabilities. https://github.com/datahub-project/datahub/issues/6852 (metadata ingestion can we bump pyspark to latest version 3.3.1)

Also can we prioritise this issue in GitHub to resolve the critical and high vulnerabilities in Datahub images and the actions , ingestion image (majority of vulnerabilities are coming from site packages in pyspark jars) https://github.com/datahub-project/datahub/issues/7222 And https://github.com/datahub-project/datahub/issues/7204

Hi, I was told to ask in this channel instead of #getting-started, so here I am 😄 Can anyone help me?

Hey team any ETA for new release i.e. 0.9.7. Thanks

Hey there! I was hoping i could get a hand with connecting to our Kafka cluster. This is AWS MSK; authentication is via IAM. I've followed these docs to no avail (

Login module not specified in JAAS config

..see full error from kafka-setup-job below). We're deploying this with helm/terraform

4 done working
Topic Creation Complete.
Error while executing config command with args '--command-config /tmp/connection.properties --bootstrap-server <http://b-1.xxx:9098,b-2.xxx:9098,b-3.xxx:9098|b-1.xxx:9098,b-2.xxx:9098,b-3.xxx:9098> --entity-type topics --entity-name _schemas --alter --add-config cleanup.policy=compact'
org.apache.kafka.common.KafkaException: Failed to create new KafkaAdminClient
	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:540)
	at org.apache.kafka.clients.admin.Admin.create(Admin.java:134)
	at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:295)
	at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:97)
	at kafka.admin.ConfigCommand.main(ConfigCommand.scala)
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
	at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:66)
	at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:88)
	at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:82)
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:167)
	at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
	at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:513)
	... 4 more

Hey team, I was wondering if anyone had the experience of migrating from self-hosted to Acryl Data? or maybe directly going to BYOC with the enterprise solution?

When I ran

datahub docker quickstart

, it stuck and kept

Pulling docker images...

for hours… Is there any solution?

I am trying to recover from an Elasticsearch outage, it’s okay if we lose historical data, but we are trying to also re-add users and groups. Is there a bootup script we can do that at least creates the initial datahub user along with the admin group?

Hello Please help me with this issue. Issue: On deploying helm chart v0.2.142, even after setting mysqlSetupJob to false deployment is trying to create mysqlSetupJob and failing.

mysqlSetupJob:
  enabled: false

Hello all! Is there anyway we can force an expiration time for the personal access tokens? Whenever a user creates a PA token, he he can select that the TOKEN will never expire. Is there any way to force the token to expire after 1 month for example? Thanks!

Hey Team, I'm facing an issue with the elastic-search setup job and IAM authentication in AWS. https://datahubspace.slack.com/archives/CV2UVAPPG/p1668642655146959?thread_ts=1668620201.836669&cid=CV2UVAPPG Has anyone been able to get around this issue during the setup? i.e. been able to use IAM authentication during the setup job?

Hey team we are eagerly waiting for your new release i.e. 0.9.7 , any ETA please update me. Thanks in advance

Hi Team, Is there anything I need to do if I want to replace my existing Elasticsearch instance which already has the old indices with a new one? I am doing some experimentation, when I replace the existing Elasticsearch instance with a new one and run the ingestion job again I cannot see the ingested metadata on the UI (I am talking about the data which belongs to the older Elasticsearch instance), instead I can see a lot of document missing error in the GMS logs. And when I plug-in the old Elasticsearch instance I am able to see the older data. Any help please? by the way I am using v0.9.1 I was assuming if I re-run all the ingestion jobs again with the NEW Elasticsearch instance I should be able to see the data on the UI regardless, but that is not case, in order to see the older data on the UI, I have to use the older Elasticsearch instance. Do I need handle this thing? I mean cant I just replace the older Elasticsearch instance with a new one, run the ingestion jobs again, and able to see all the data on the UI?

Hi I'm deploy datahub 0.9.6.1 with OIDC via Keycloak. After signin successfully, I'm signout and unable to signin again. Bellow is logging ERROR. Am I missing any configurations?

Hey team any update on releasing 0.9.7 version today as this upgrade is very critical for our project.

hi guys, can I know what changes I need to do in Datahub to get the metrics in these panels up on Grafana? • Search QPS • Search Latency • Browse QPS Thanks! 🙂