Hi everyone! We’re currently evaluating DataHub vs Amundsen and have a few questions: what’s the fastest way to set up a local DataHub instance? Would you be able to provide us with a terraform file to set up the necessary aws infra?

<!here> 📣 Town-hall videos from last Friday are now live on YouTube! 1. Full community video: 2. Performance session that we had to skip due to time: . Thanks @early-lamp-41924 for recording! Please like / share / subscribe 🙂

with JIT user/group provisioning can we force refresh on group membership for users that already exist?

hello i have one question i datahub using ldap then i was write jaas.conf file but got thie error how to change configuration?

WHZ-Authentication {
  com.sun.security.auth.module.LdapLoginModule REQUIRED
  userProvider="ldap://[host]:[port]/ou=ldap/"
  authIdentity="{USERNAME}"
  userFilter="(&(|(samAccountName={USERNAME})(userPrincipalName={USERNAME})(cn={USERNAME}))(objectClass=user))"
  javax.security.auth.login.name=[auth id]
  javax.security.auth.login.password=[auth pw]
  tryFirstPass="true"
  debug="true"
  useSSL="false";
};

error

[LdapLoginModule] authentication-first mode; SSL disabled
                [LdapLoginModule] user provider: ldap://[host]:[port]/ou=ldap/
                [LdapLoginModule] tryFirstPass failed: javax.security.auth.login.FailedLoginException: No password was supplied
                [LdapLoginModule] attempting to authenticate user: datahub
                [LdapLoginModule] authentication failed
                [LdapLoginModule] aborted authentication
23:42:12 [application-akka.actor.default-dispatcher-5] ERROR controllers.AuthenticationController - Authentication error
javax.naming.AuthenticationException: javax.security.auth.login.FailedLoginException: Cannot bind to LDAP server

Dear DataHub Team. I have a question about RBAC/fine-based access control. Demo above ^^^ shows some UI scenarios of denied access and Authorizer component, explained here . The question is: are GMS REST and GraphQL calls also filtered by access policies, or it is only applied to UI part?

Hey all, we're about to start using both DH and Deequ on a project. I noticed that there was mention of quality entities/aspects on the roadmap, but I can't find any reference on github. Is this under development and, if so, can we help out?

Dear DataHub Team. As I see from

DatahubRestEmitter

python class, it supports authentication token for GMS REST calls. Do we an instruction how to setup user/password for REST and GraphQL calls ? At this moment I am not using token with local installation, it works fine (anonymous?)

Recently, I attended two webinars ( & ). Since, DataHub has Data Mesh in the roadmap, these 2 webinars can be the reference for implementation. I personally think that DataHub is in a great position to enable the Mesh Experience Plane to its full potential.

Few Questions on Tags- 1. Do we have a tag library ? 2. When we search via tags, even those that have it dataset name pop up ? Are we able to customize search and filter 3. Are we able to propagate the tags to a downstream system ? eg. When they are looking up datasets

Hi guys, I really like your demo theme (https://demo.datahubproject.io/) How can I use for my own deployment?

Is the demo regularly updated with the new released version?

noticed this line in datahub features list: • Dataset life-cycle management: deprecate/undeprecate, surface removed datasets and tag it with "removed" other than doing a query in mySQL for urns with aspect=Status, is there any other way to find removed datasets?

Not sure if it’s the right place but did not find a better channel 🤷‍♂️ I am on an older Cloud Composer version and want to use the DatahubEmitterOperator only to ingest some metadata. Do you know if its possible to register it as a plugin to use in my DAG or do I need to change my Composer installation? This wouldn’t be my preferred solution because its used by multiple teams and don’t want to mess with the dependencies

image.png

Hi How should I fill in this `access token`when I initialize it

datahub init

/root/.datahubenv already exists. Overwrite? [y/N]: y

Configure which datahub instance to connect to

Enter your DataHub host [<http://localhost:8080>]:

`Enter your DataHub access token (Supports env vars via

{VAR_NAME}

syntax) []:`

hi! so if i started datahub with datahub docker quickstart command how to i upgrade to newer version of datahub when they release? Is there particular command that I can do it? thanks!

Hello Team, am new and exploring datahub.. have a query around it.. i can see the datajob we can have airflow.. but is it somehow possible to record other jobs (like any batch job apart from airflow) as well maybe using rest api.. i cannot find in documentation how to register jobs... maybe if you can please help me direct to some documentation..

HI team, need to help find logs for

mae-consumer-job

. I have run the

mae-consumer-job

in two approaches 1:

docker

2:

java -jar mae-consumer-job.jar

. My goal is to see some logs by log4j either in console, or

mae-consumer-job.log

file. for example.

<https://github.com/linkedin/datahub/blob/master/metadata-jobs/mae-consumer/src/main/java/com/linkedin/metadata/kafka/MetadataAuditEventsProcessor.java#L96>

I changed

log.debug

to

<http://log.info|log.info>

but no matter what I do, I won't see any log printed. Did I do something wrong obviously?

Hi all, is it possible to search for datasets inside a specific path ? asking about graphql methods

Hey, guys, quick question. Could you relate business glossaries to charts or dashboards?

Does Datahub support Active Metadata Management or, Passive?

• Artifact storage: The Pods store two kinds of data: Metadata: Experiments, jobs, pipeline runs, and single scalar metrics. Metric data is aggregated for the purpose of sorting and filtering. Kubeflow Pipelines stores the metadata in a MySQL database.

ICYMI - DataHub’s UI is getting a Makeover! Check out this rundown of the changes you can expect starting with v0.8.12 (and big shoutout to @big-carpet-38439 for his demo at the August Town Hall!)

Hello! Curious how other companies are defining the different roles in the ownership tab, especially differentiating between

producer

and

data owner

. Also what is an example of a

delegate

?

Hi @mammoth-bear-12532 @big-carpet-38439. I was attempting to add query usage for AWS Athena in datahub. AWS Athena does not provide any in-built way to get query history. But we store it ourselves using their API in a postgres table. So I am going to read through these files https://github.com/linkedin/datahub/tree/master/metadata-ingestion/src/datahub/ingestion/source/usage to understand it and then add a source for reading query history from a table to ingest it. Just wanted to know if there are any limitations in the storage model or any gotcha I should be aware of regarding query usage.

And big welcome to @big-ambulance-87245 @eager-answer-71364 @acoustic-vr-82190 @red-egg-59838 @clean-crayon-15379 @broad-crowd-28379 @silly-umbrella-20605 @gorgeous-hairdresser-55031 @plain-state-36866 @most-actor-22367 @careful-controller-71092 @happy-orange-34018 @stocky-noon-61140! We’re excited to see you all here hihi

hi folks, curious if anyone has setup datahub in their org with multi-region (e.g. AWS East/ West AZs) data replication? any considerations to think about?

Hey folks, i was wondering if anyone has successfully deployed datahub using hashicorp nomad?

not sure where to ask, but version

0.8.14

is not published to helm repository.

Curious how close to 'ready for use' the Observability features mentioned in the Roadmap are?