Here's the recording:

Is there a way to hide an aspect from returning when browsing/searching entity? i.e. When get an entity from API, this field doesn't display; only display when getting this aspect specifically from API

Happy Thanksgiving! On popular demand, I just uploaded the last TownHall videos up on our Youtube channel. My Metadata Use-cases At LinkedIn talk is up here:

Hi team, I am trying to create a new Entity in Datahub referring to the document (https://github.com/linkedin/datahub/blob/master/docs/how/entity-onboarding.md), as I see in the last step,

UI for Entity creation is a work in progress

. Is this an essential step for a new Entity to get reflected on UI ?

Hey everyone I am trying to get all the logs for datahub.. following this documentation : https://github.com/grofers/datahub/blob/master/docs/docker/development.md I tried enabling debugging, but I am getting this in the docker desktop dashboard logs :

DefaultTaskDescriptor
log4j:WARN No appenders could be found for logger (com.linkedin.restli.docgen.VelocityTemplatingEngine).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See <http://logging.apache.org/log4j/1.2/faq.html#noconfig> for more info.
Debugger failed to attach: handshake failed - received >GET / HTTP/1.1< - expected >JDWP-Handshake<
Debugger failed to attach: handshake failed - received >GET / HTTP/1.1< - expected >JDWP-Handshake<
Debugger failed to attach: handshake failed - received >GET / HTTP/1.1< - expected >JDWP-Handshake<
Debugger failed to attach: handshake failed - received >GET / HTTP/1.1< - expected >JDWP-Handshake<
Debugger failed to attach: handshake failed - received >GET / HTTP/1.1< - expected >JDWP-Handshake<
Debugger failed to attach: handshake failed - received >GET / HTTP/1.1< - expected >JDWP-Handshake<

Am I missing anything? I want to get all the logs and not just the error logs..

Can someone give me a quick overview of how the login functionality is working in datahub? I looked at the requests it makes and the code as well.. What I understood from there was it makes an

POST

call to

/authenticate

route with username and password which returns a

PLAY_SESSION

cookie no matter what the username and password is being passed ( I am aware of the dummy login which is used right now ). After that it makes a

GET

request to

api/v1/user/me

route with the

PLAY_SESSION

cookie and this route verifies whether the username is actually present or not and sends a response based on that. But I am not able to understand how is the API hit to

api/v1/user/me

logging out a username that is not present. As in, it does not change the cookie or anything. Inside the code as well I am not able to see any logout trigger from

getLoggedInUser

( as per the

datahub-frontend/conf/routes

file, this is the function that is being hit by

api/v1/user/me

route.. )

what's the best way to capture snapshot of list and relationship of available entities, aspects and their properties? our team is basically going through the code to list out PDLs and making relationships manually to create visual map to help us to understand the current landscape and was wondering if there is a better way to do this.

Hi, I have read the codes for past few days, but still confusing that how 's work of datahub if there is table rename(e.g. a hive table), anybody can help me ?

Currently, is there a way for Datahub to allow push data instead of pull data (using crawlers)? What we are looking for is APIs that allow outside system to edit data in Datahub (such as adding new Dataset, editing existing CorpUser etc.). As what I understand, we should not allow other systems to access GMS directly, so we are wondering how people are doing the data push right now. Thanks!

Hi team, Would like to know the API support in DataHub. Could you pls help in sharing the documentation associated to this.

@ambitious-battery-33996 @microscopic-receptionist-23548 Has the team considered introducing fine-grained access control against the Metadata graph? ie. Dictating which identities can read / write to which entities + aspects?

Hi team, is any easy way to search how many columns for a schemametadata asepct of all datasets? for example, if I am interested in the all datsets which have columns count greater than 200, whats the easiest way to do that? Can we use ElasticSearch to do the aggregation for us? I felt we could, also felt it is not easy.

Hello! I am looking to design a technology independent 

#dataCatalog

 for heterogeneous

#datamesh

-es, 

#datalake

-s and `#featureStore`s Found this `GraphQL`/`SHACL` extension as

data schema

language to be very interesting (first tweet below). And would also like to build on top of

DataHub

. In this post https://engineering.linkedin.com/blog/2019/data-hub it says

_... plan to add native support for GraphQL and leverage Pegasus Domain Specific Language (PDL)_

so the

DataHub

project is already considering

GraphQL

🤔 Is there a design channel best suited to discuss how to maybe combine the two ? Please see a series of tweets for better context https://twitter.com/semanticbeeng/status/1338395121344196614 https://twitter.com/semanticbeeng/status/1340659039160680448 https://twitter.com/semanticbeeng/status/1340341801245433858 https://twitter.com/semanticbeeng/status/1340213178735742976 https://twitter.com/semanticbeeng/status/1338165535067017216 /cc @mysterious-waiter-64784 @adorable-processor-445

Hi everyone! DataHub Santa Claus is here and conducting a poll for the community to vote on their wishlist for DataHub in 2021! Please vote for the suggestions I already put or add your own suggestions here! https://poll.ly/#/LnxK8172 Happy Holidays 🎅

Have we had discussion around support for Thrift or Protobuf for RPC?

Welcome @breezy-hospital-33159, @wide-autumn-16831 and @jolly-tent-42863

does anyone know if there is a plan to have a Legal Glossary ? I see that business glossary has recently been added

Hello. Apologies if this is the wrong channel, this is my first time here. I have a question about the GMS component. What is the recommended way to configure the Kafka clients that the service uses? Specifically I'm interested in configuring the encryption and authentication aspects of the connection. I saw traces in the code of Spring boot stuff but I fail at providing the regular application.properties file and getting that to take. /Thanks

Hi, after updating to master I'm having trouble pushing out commits to any branch. The ESLint script that runs fails to find the @dh-tools repository. I tried running through the commands in ESLint section to install it, but that gave me the same error when trying to install the @dh-tools plugin. Any quick fixes for this?

Hi Everyone! Our next DataHub Townhall is Friday, February 19th, 9am-10am PDT. Let us know if you have any topics that you would like added to the agenda.  Also, please sign up here if you haven’t already https://docs.google.com/spreadsheets/d/1hCTFQZnhYHAPa-DeIfyye4MlwmrY7GF4hBds5pTZJYM/edit#gid=1753521814

hey guys, I am getting this issue when trying to build (./gradlew build) on original open source code. Not sure what went wrong, please help

I am trying to build (

./gradlew build

) but getting different error. Can anyone <!here> fix it?

@big-carpet-38439 I think this PR will fix it; test containers was updated in GMA

.30

but datahub wasn't updated to pull that up

Welcome @thousands-dentist-3406 and @elegant-hairdresser-61963!!

Hello, I'm looking into Datahub as a solution for my company's needs (data lineage & discovery). One of the our main requirements is that the underlying Data Model of the metadata system is evolvable. • Is DataHub's dynamic model able to change gradually over time for a given entity? • If DataHub supports this, how does it affect search capabilities? • Is there documentation on this? I've read through the markdown files in Github but couldn't find anything. Any help would be greatly appreciated, thank you for taking the time to read 🙂

Hi Team, have not kept up with the latest Datahub. I am looking at

metatadata-dao-impl

and found out

essearch-dao

are auto-generated classes. Can someone help me point out where to look at so the auto-generated content happens?

Hello, newbie question: think I read somewhere that the hierarchy / organization of datasets could be configured in DataHub, but I can't find how/where that could be configured or updated? Could that be done through MCE ingestion?

Hi team, could you update the roadmap to reflect current priorities? the dates noted on the page is outdated. https://github.com/linkedin/datahub/blob/master/docs/roadmap.md

Hello, when defining a data model for DataHub is it up to the developer to model every relationship between entities in pdl as seen here for the relationships defined in the out-of-the box data model? Is there any tool to help with this generation?

Follow-up question, what functionality does DataHub guarantee if I don't use its pre-packaged DataModel? I assume things like data lineage would (in the UI) would break right?