Hello, when defining a data model for DataHub is i...
# getting-startedi
incalculable-ocean-74010
02/03/2021, 10:33 AMHello, when defining a data model for DataHub is it up to the developer to model every relationship between entities in pdl as seen here for the relationships defined in the out-of-the box data model? Is there any tool to help with this generation?
g
green-football-43791
02/03/2021, 5:32 PMHey Pedro 👋 You're right that its up to developers to model any additional relationships between entities. In terms of guidance for helping modeling, you can always refer to the existing examples. Out of curiosity, what kind of tool do you think would make things easier for you?
i
incalculable-ocean-74010
02/03/2021, 5:49 PMHey Gabe, thank you for your reply. The first thing that came to mind was a visual editing tool akin to UML diagram tools.
incalculable-ocean-74010
02/03/2021, 5:50 PMPerhaps a parser for a markdown-based diagram tool like mermaid.js could help.
g
green-football-43791
02/03/2021, 5:50 PMI like that- I think a visual way to configure entities, aspects & their relationships would be very useful to folks
green-football-43791
02/03/2021, 5:51 PMah, so do you feel like understanding the existing relationship graph or extending it is a bigger challenge?
i
incalculable-ocean-74010
02/03/2021, 5:51 PMJust something that will automatically generate the pegasus files would help tremendously and also gives a fast way to understand the data model that someone may be extending/adapting/creating.
g
green-football-43791
02/03/2021, 5:51 PMthose two tools would solve different problems, correct?
green-football-43791
02/03/2021, 5:52 PMI like being able to automatically generate a chart- perhaps we could even put that under version control and have it be re-checked in after each change
i
incalculable-ocean-74010
02/03/2021, 5:53 PMVersion control and auto-generation of the data model in a chart sounds like a great idea
incalculable-ocean-74010
02/03/2021, 5:53 PMI think there are a couple of use-cases that can addressed here.
incalculable-ocean-74010
02/03/2021, 5:53 PM1.) Understand the current data model a given DataHub deployment has.
incalculable-ocean-74010
02/03/2021, 5:54 PM2.) Update/extend an existing model without having to directly edit pegasus files, which can be a significant number making it a bit hard to grok the underlying model.
incalculable-ocean-74010
02/03/2021, 5:55 PM3.) Generate Data Models from scratch (this is a current need for me) and making writing the pegasus records an easier & less prone task.
incalculable-ocean-74010
02/03/2021, 5:58 PMThis Data Model viz/edit tool could even be part of the DataHub UI (admin probably) with import/export capabilities to enable version control (think Grafana dashboards)
g
green-football-43791
02/03/2021, 6:00 PMThese are all great ideas- would you be interested in filing an issue on DH open source to capture some of this? I wouldn't want it to get lost in slack. There's a feature-request tag for this sort of thing
i
incalculable-ocean-74010
02/03/2021, 6:01 PMSure
g
green-football-43791
02/03/2021, 6:01 PMfor #3 - are you unable to make progress without a tool to generate data models?
i
incalculable-ocean-74010
02/03/2021, 6:03 PMI'm still wrapping my head around DataHub generally speaking so it's little soon to say.
incalculable-ocean-74010
02/03/2021, 6:03 PMSo far I haven't tried to create a data model.
g
green-football-43791
02/03/2021, 6:04 PMok 👍 sounds good
i
incalculable-ocean-74010
02/03/2021, 6:32 PMHere is the issue: https://github.com/linkedin/datahub/issues/2080
If there is anything that needs elaboration, feel free to ping me
g
green-football-43791
02/03/2021, 6:35 PMthanks!