I am using a custom user.props file but none of the users have admin privileges

Hi all, I am trying to deploy datahub in the computer and appear the following error: Docker doesn't seem to be running. Did you start it?. I have tried first on my personal computer with docker desktop and later I have created an Azure Virtual Machine, installed Docker, python and run the quickstart and the same error appears. How I can solve it? azureuser@DatahubPrueba:~$ sudo docker info Client: Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.10.2 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.16.0 Path: /usr/libexec/docker/cli-plugins/docker-compose scan: Docker Scan (Docker Inc.) Version: v0.23.0 Path: /usr/libexec/docker/cli-plugins/docker-scan Server: Containers: 1 Running: 0 Paused: 0 Stopped: 1 Images: 1 Server Version: 23.0.1 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: false userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640 runc version: v1.1.4-0-g5fd4c4d init version: de40ad0 Security Options: apparmor seccomp Profile: builtin Kernel Version: 5.15.0-1034-azure Operating System: Ubuntu 20.04.5 LTS OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 7.765GiB Name: DatahubPrueba ID: ecac15f0-d3d0-4eda-a684-dea4224573e7 Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

is datahub can be useful for data mesh?

Hi, I am absolutely new to DataHub and trying to set up the application. Facing some roadblocks in the installation process itself. Anyone who can guide me please?

Hi, I’m planning to build a DataHub GraphQL Client app for CRUD operations on DataHub metadaservice. I would like to use autogenerated Java/Kotlin code from graphql schemas used in datahub project. What is the best way to access those GraphQL schemas?

Hi all, I use Colima and not Docker Desktop, so I couldn't get DataHub up and running. this thread (https://datahubspace.slack.com/archives/C029A3M079U/p1659141725809899?thread_ts=1659102758.255209&cid=C029A3M079U) gave me the answer with this command (after cloning datahub to my machine):

docker-compose -p datahub -f docker/quickstart/docker-compose-without-neo4j-m1.quickstart.yml up -d

I get "Docker doesn't seem to be running. Did you start it?" when I run

datahub docker ingest-sample-data

What command must I use to permanently avoid this 'not running' error?

Hi all, I am having trouble during the installation of Datahub with the quickstart guide, I follow all the steps correctly but when running datahub docker quickstart (py -m datahub docker quickstart, in my case) after a long time I obtain the following errors in red: Unable to run quickstart - the following issues were detected: - datahub-actions is not running - datahub-gms is running by not yet healthy - broker is not running - datahub-upgrade is still running I have attached the logs in case someone could help me. Thank you very much.

Is data-x64.msi broken? - I just installed on Windows via the msi file. On the first step it errors when I enter: data --version "openssl config failed: error02001002system libraryfopenNo such file or directory 0.9.5" - I tried googling this... nothing related to DataHub

Hi All, trying to install DataHub and am getting the following message. Any guidance on the next steps? Pulling docker images...This may take a while depending on your network bandwidth. -time="2023-03-09T180426-08:00" level=warning msg="The \"HOME\" variable is not set. Defaulting to a blank string." @astonishing-answer-96712

Hi Everyone I got error when command code below. Is anyone know how to fix it?

docker-compose -f docker-compose-without-neo4j.quickstart.yml up -d

mysql  | 2023-03-07T02:42:58.629104Z 0 [ERROR] InnoDB: Only one log file found.
mysql  | 2023-03-07T02:42:58.629122Z 0 [ERROR] InnoDB: Plugin initialization aborted with error not found
mysql  | 2023-03-07T02:42:59.229686Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
mysql  | 2023-03-07T02:42:59.229715Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
mysql  | 2023-03-07T02:42:59.229722Z 0 [ERROR] Failed to initialize builtin plugins.
mysql  | 2023-03-07T02:42:59.229725Z 0 [ERROR] Aborting

Thanks in advance :)

Hello I’m just asking, is there any references for datahub deployment using kubernetes helm chart via terraform script?

hi! Is it currently supported to create (or delete) datasets with GraghQL? If not, when can it be expected?

Newbie question: I've had to move my docker to an external SSD D:\ drive (due to storage space on my C:\ drive). Are there any considerations re: running > datahub docker quickstart' (e.g., storing on C:\ vs D:\) ?

Hi community. I am new to the data governance area. I would like to get some advice on design. We are using apache ranger + trino for role access control. Is apache ranger can use the Tag/Owner in Datahub. etc. I can restrict user access with PII tag.

Hi everyone i got an error on datahub GMS when i run with docker please help

the gms can't setup,What should I do. the version: Latest system update version: v0.10.0.6rc1-0 the docker-compose-without-neo4j.quickstart.yml download to https://github.com/datahub-project/datahub/blob/master/docker/quickstart/docker-compose-without-neo4j.quickstart.yml the log: 2023-03-13 105940,680 [ThreadPoolTaskExecutor-1] INFO o.a.k.clients.consumer.KafkaConsumer:1603 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-2, groupId=generic-duhe-consumer-job-client-gms] Seeking to offset 0 for partition DataHubUpgradeHistory_v1-0 2023-03-13 105940,680 [ThreadPoolTaskExecutor-1] INFO o.a.k.clients.consumer.KafkaConsumer:1603 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-2, groupId=generic-duhe-consumer-job-client-gms] Seeking to offset 0 for partition DataHubUpgradeHistory_v1-0 2023-03-13 105940,683 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - generic-duhe-consumer-job-client-gms: partitions assigned: [DataHubUpgradeHistory_v1-0] 2023-03-13 105941,060 [ThreadPoolTaskExecutor-1] INFO c.l.m.b.k.DataHubUpgradeKafkaListener:84 - Latest system update version: v0.10.0.6rc1-0 2023-03-13 105941,061 [ThreadPoolTaskExecutor-1] INFO c.l.m.b.k.DataHubUpgradeKafkaListener:88 - System version is not up to date: v0.10.0-0. Waiting for datahub-upgrade to complete... 2023-03-13 110022,052 [R2 Nio Event Loop-1-1] WARN c.l.r.t.h.c.c.ChannelPoolLifecycle:139 - Failed to create channel, remote=localhost/127.0.0.1:8080 io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080 Caused by: java.net.ConnectException: Connection refused at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:829) 2023-03-13 110024,172 [R2 Nio Event Loop-1-2] WARN c.l.r.t.h.c.c.ChannelPoolLifecycle:139 - Failed to create channel, remote=localhost/127.0.0.1:8080 io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080 Caused by: java.net.ConnectException: Connection refused at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:829) 2023-03-13 110028,384 [R2 Nio Event Loop-1-3] WARN c.l.r.t.h.c.c.ChannelPoolLifecycle:139 - Failed to create channel, remote=localhost/127.0.0.1:8080 io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080 Caused by: java.net.ConnectException: Connection refused at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:829) 2023-03-13 110028,388 [pool-14-thread-1] ERROR c.d.m.ingestion.IngestionScheduler:244 - Failed to retrieve ingestion sources! Skipping updating schedule cache until next refresh. start: 0, count: 30

Hello everyone. I am running a self hosted datahub instance, however the default user does not seem to have admin rights. I require administrator permissions in order to generate invite links. Is tere a way in which this can be fixed? I am aware that you can also configure SSO.

Hi Everyone. I got permission error while running code

sudo docker-compose -f docker-compose-without-neo4j-m1.quickstart.yml up -d

This is the log of quickstart-datahub-actions-1 container.

2023/03/14 03:20:31 Waiting for: <http://datahub-gms:8080/health>
2023/03/14 03:20:31 Received 200 from <http://datahub-gms:8080/health>
ls: cannot access '/etc/datahub/actions/system/conf/': Operation not permitted
No system action configurations found. Not starting system actions.
ls: cannot access '/etc/datahub/actions/conf/': Operation not permitted
No user action configurations found. Not starting user actions.
Usage: datahub-actions actions [OPTIONS] COMMAND [ARGS]...

  Execute one or more Actions Pipelines

Options:
  --help  Show this message and exit.

Commands:
  run*     Execute one or more Actions Pipelines
  version  Print version number and exit.
2023/03/14 03:20:31 Command finished successfully.

Trying to configure mongo datasource with slave member of a replicaset through cross account VPC(AWS Privatelink), because our analytics account and production account are different in AWS accounts, but the ingestion is getting failed as it is trying to resolve the actual hostnames in the replica set though we have configured the connection uri using privatelink dns

Hi, I am trying to log into Datahub using the following link as shared on the instructions page - http://localhost:9002/

I still cannot log in

any suggestions on how to resolve it?

Hello, I have successfully deployed DataHub via docker quickstart on my mac and was able to ingest a sample PostgreSQL database. I was able to add a tag and a description to one of the columns in one of the tables via the UI. However, when I use the GraphQL API to examine the dataset, I don't see the tag or description - they are both null. Thoughts?

Hi, trying to use the GraphQL API in a test app. I was able to get the GraphQL query to work in graphiql, but from the app a bearer token is required. In the UI it says

Token based authentication is currently disabled. Contact your DataHub administrator to enable this feature.

I was able to generate an access token as described here: https://datahubproject.io/docs/api/graphql/token-management/ but that didn't work. Thank you!

I had to reset everything in datahub. Have big query connected and ingesting fine but when i reran the datahub docker it is no longer mounting my files in the directory on the actions hub even though I have the docker-compose file that we edited saved on my desktop. Do you know what commands i need to run to make sure the docker reads that compose file that i edited with my docker manifest and catalog files mounted?

Hi. I'm new to DataHub and am trying to use the searchAcrossEntities query to find any entities with an associated keyword. My results will return a matching glossary term, but will not return the entities that have that term associated. So I'm wondering if there is a way to query for a GlossaryTerm and have the related entities displayed in the results. I was trying to add 'relationships' to the graphQL but I'm not sure if that will return the related entities and if so, what I'm supposed to set for 'types' or 'direction'. I currently have this in my searchAcrossEntities query, which..doesn't work. Any help is appreciated.

... on GlossaryTerm {

            relationships(input: {types: "Contains", direction: INCOMING}) {
              start
              count
            }
          }

Hey all, we are trying to show lineage between different

MlModel

entities (as seen in this townhall: ), and we couldn’t have found a way to link different

MlModel

entities to each other and the lineage tab is greyed out for each

MlModel

How would you go about making a lineage between two models?

Hey all! I am seeking some architecture advice and would appreciate any insights you may have. Our enterprise is looking to deploy a central DataHub instance and connect it to multiple databases, for which we do not have direct access. As a result, the teams responsible for each database will need to push data to the central instance. After researching, I came across the concept of federated instances, and I believe this would be the best approach for our use case, rather than creating custom adapters to push data to kafka/rest. Would anyone happen to have documentation or resources on implementing a federated scenario? Additionally, aside from the central instance, which components should the respective teams deploy? Is it just the metadata service or are there other components that need to be taken into consideration? Thank you in advance for your help!

Hi everyone! I don't know how to fix this error? Can someone help for this?) It was when I create the new ingestion connection to database. The server. which contained DataHub on docker is isolated for the ethernet. But i create and load worked images on docker. The screen in loads

Hi everyone! Is it possible to include a custom transformer in a UI-based metadata ingestion?