DataHub #getting-started

prehistoric-grass-6413

09/14/2021, 3:44 PM

From a transactional workload perspective, what attributes of database architectures are most important to you all? Things I'm considering: -node based (scale easily) - Raft protocol (replicating data for resiliency) - compatibility with kubernetes -automatic load rebalancing / eliminate manual sharding - geo-partion data (tie data to a location for low latency and compliance) - consistency

mammoth-bear-12532

09/14/2021, 7:01 PM

<!here> 📣 I’m delighted to announce that our very own @little-megabyte-1074 is joining Acryl Data as DataHub’s Community Product Manager. I’m looking forward to working with her to drive the project forward and grow this amazing community of metadata enthusiasts! 🎉 . Do say hi to her!

🙌 8

🙌🏻 1

🎉 23

blob wave 3

datahub 5

little-megabyte-1074

09/14/2021, 9:20 PM

hihi Welcome to the community@average-bear-318 @gorgeous-refrigerator-79151 @mysterious-controller-90641 @gentle-alligator-23743 @little-address-54150@cool-state-20157 @witty-library-27671 @prehistoric-grass-6413 @billowy-postman-96279 @enough-midnight-97468 @crooked-raincoat-27806! Come on in, make yourselves comfortable, and please don’t hesitate to let me know how I can help you out!

👍 1

little-megabyte-1074

09/15/2021, 5:53 PM

hihi panda Hello, hello @worried-lion-81038 @adorable-crayon-96932 @sparse-planet-56664 @average-oxygen-94446 @millions-soccer-98440 @red-pizza-28006 @refined-flower-25872 @cuddly-postman-70897! Welcome to the DataHub community - we’re so happy to see you here yay

cuddly-postman-70897

09/15/2021, 5:56 PM

@little-megabyte-1074 Will datahub be useful in my recsys project which is made using kubeflow

cuddly-postman-70897

09/15/2021, 5:57 PM

NVIDIA-MERLIN , triton inference server , nvtabular

cuddly-postman-70897

09/15/2021, 6:01 PM

@little-megabyte-1074 In kfp there is already a artifact store which has a metadata service https://www.kubeflow.org/docs/components/pipelines/overview/pipelines-overview/

little-megabyte-1074

09/16/2021, 7:56 PM

Gooood morning/afternoon/evening, DataHub Enthusiasts! We need your help! We found this post on reddit where OP is looking for perspectives on why companies have chosen to integrate an open-source data discovery platform. Please take a few minutes to hop on over and post your DataHubLove - what considerations you made and why DataHub was the right tool for the job ❤️

datahub 7

🔥 4

little-megabyte-1074

09/16/2021, 11:27 PM

hihi koala Warm welcome to @high-manchester-50384 @boundless-room-44377 @hundreds-twilight-96303 @lively-jackal-98096 @billowy-knife-57444 @quiet-kilobyte-82304 @limited-postman-40280!

👋 2

high-manchester-50384

09/16/2021, 11:41 PM

Thank you @little-megabyte-1074!

limited-postman-40280

09/17/2021, 6:38 AM

Thanks @little-megabyte-1074 I'm happy to join this community

little-megabyte-1074

09/20/2021, 3:03 PM

Good morning/afternoon/evening, DataHub! I hope everyone’s week is off to a great start 🙂

little-megabyte-1074

09/21/2021, 10:07 PM

hihi penguin Sending a very warm welcome to @broad-crowd-13788 @green-scooter-59365 @colossal-refrigerator-56735 @clean-oil-40817 @breezy-waitress-94564 @important-airplane-19810 @straight-dentist-7439 @witty-keyboard-20400 @lemon-receptionist-90470 @orange-army-54704 so happy you all have joined us! Please let me know what we can do to get you started teamwork

🙌 1

little-megabyte-1074

09/22/2021, 12:31 AM

has renamed the channel from "general" to "getting-started"

little-megabyte-1074

09/22/2021, 1:53 AM

‼️ This is a big change!! Please check out this post for more details

dry-lizard-54588

09/22/2021, 6:33 AM

1300 !!

🙌 3

✅ 1

little-megabyte-1074

09/27/2021, 9:00 PM

Hello, all! In the coming weeks/months, I am going to be rolling out DataHub Product/Feature walk-through videos and blog posts to help folks get more familiar with the platform. 🗣️ Looking for your input! Thinking about your experience using DataHub… • what features, functionality, and/or workflows are still a bit unclear to you? • what would help new DataHub Community Members get up & running more quickly? • how can we help you onboard/educate your end-users once you have DataHub running in prod? Can’t wait to hear your thoughts/suggestions — add them in the thread! 🧵 thanks ewe

🙌 2

some-cricket-23089

09/30/2021, 6:06 AM

Hi Team , I was doing some changes in UI look and feel of datahub. To verify those changes i was trying to build the docker image of datahub-frontend module on running the below command from datahub home folder

Copy code

sudo docker build -t updated_datahub_frontend_react -f ./docker/datahub-frontend

But this end with the error below

Copy code

---> Running in 55db64e7685d
/bin/sh: ./gradlew: not found
The command '/bin/sh -c cd datahub-src && ./gradlew :datahub-frontend:dist -PenableEmber=${ENABLE_EMBER} -x test -x yarnTest -x yarnLint     && cp datahub-frontend/build/distributions/datahub-frontend.zip ../datahub-frontend.zip     && cd .. && rm -rf datahub-src && unzip datahub-frontend.zip' returned a non-zero code: 127

Could anyone please help me resolve this issue.

witty-keyboard-20400

10/04/2021, 11:49 AM

In the SchemaMetadata.pdl I see the platformSchema as a union

Copy code

/** The native schema in the dataset's platform. */
  platformSchema: union[
    EspressoSchema,
    OracleDDL,
    MySqlDDL,
    PrestoDDL,
    KafkaSchema,
    BinaryJsonSchema,
    OrcSchema,
    Schemaless,
    KeyValueSchema,
    OtherSchema
  ]

If I've to mention for MongoDB, then which choice would be appropriate:

Schemaless

, KeyValueSchema
, or

OtherSchema

agreeable-hamburger-38305

10/06/2021, 5:32 PM

Curious if the preview feature has been implemented!

witty-keyboard-20400

10/12/2021, 12:38 PM

The output I'm getting is not as per https://github.com/linkedin/datahub/tree/master/metadata-service#get-a-dataset Could anyone please help me understand why am I not getting the full details of the DatasetSnapshot?

brief-lock-26227

10/18/2021, 2:52 PM

I've been unable to get the datahub quickstart up and running fully on my M1 Macbook. I get about 8 blocks of output listing the 10 services like the one at the beginning of this excerpt, then it stops:

Copy code

..............
[+] Running 10/10
 ⠿ Container elasticsearch           Running                                                                          0.0s
 ⠿ Container elasticsearch-setup     Started                                                                          0.8s
 ⠿ Container zookeeper               Running                                                                          0.0s
 ⠿ Container broker                  Running                                                                          0.0s
 ⠿ Container schema-registry         Running                                                                          0.0s
 ⠿ Container kafka-setup             Started                                                                          1.7s
 ⠿ Container mysql                   Running                                                                          0.0s
 ⠿ Container mysql-setup             Started                                                                          0.8s
 ⠿ Container datahub-gms             Running                                                                          0.0s
 ⠿ Container datahub-frontend-react  Running                                                                          0.0s
.............
Unable to run quickstart - the following issues were detected:
- kafka-setup is still running
- datahub-gms is still starting
- schema-registry is not running

I had to slightly modify the Docker file executed by the quickstart to get this far (mysql wouldn't start unless I specified the platform; see my post in this channel about 24 hours ago). But these new errors seem to have nothing to do with mysql. Suggestions?

quiet-pilot-28237

10/20/2021, 3:47 AM

hi all: when I want to add tag for my demo dataset, I got this error, I logged in using datahub

quiet-pilot-28237

10/20/2021, 3:48 AM

image.png

quiet-pilot-28237

10/20/2021, 5:32 AM

Ignore it, I add new policy fix it

👍 2

agreeable-hamburger-38305

10/20/2021, 8:28 PM

Hi @big-carpet-38439 Thanks for the reply. A follow-up question: since I will be putting the app behind GCP IAP, so it’s only accessible to users in the company, do you know how I can skip the authentication page altogether?

silly-translator-73123

10/22/2021, 8:45 AM

hello guys , can i ask a question about datahub?

few-pilot-60859

10/28/2021, 11:58 AM

Wow, I joined today and had the same question for ClickHouse 🙂

teamwork 2

mammoth-bear-12532

10/29/2021, 4:02 PM

<!here> Townhall is getting started :D

fierce-action-87313

11/01/2021, 2:54 PM

anyone else facing that issue?