Another question, from a newb trying to understand...
# getting-started
h
Another question, from a newb trying to understand how Datahub works. In
datahub/metadata-models/src/main/pegasus/com/linkedin/metadata/snapshot/
we define e.g. ChartSnapshot and MLModelSnapshot. However, in
Snapshot.pdl
we only list MLModelSnapshot and not ChartSnapshot in the union. Why is that? Similary, in
datahub/metadata-models/src/main/pegasus/com/linkedin/metadata/entity/
we define a ChartEntity, but not a MLModelEntity, and the ChartEntity is not listed in the union in
Entity.pdl
. Why is that?
Similar thing in
../search
. ChartDocument is defined, but on listed in
Document.pdl
a
most likely, the complete chart feature is in progress. typically, a PR is sent for data model only. once data model is approved, the related implementation will be started. You probably have seen an approved data model part.
h
okey, makes sense! So just for me to understand, the idea is to eventually list all snapshots/entities in `Snapshot.pdl`/`Entity.pdl`, respectively?
s
Snapshot
model is used to generate MetadataChangeEvent & MetadataAuditEvent
On the other hand, Entity and Document are used in MetadataSearchEvent & MetadataGraphEvent which are not productionized and not used at this point
So that being said: All snapshots should be added to Snapshot.pdl but it doesn't quite matter if we don't add search documents to Document.pdl and entities to Entity.pdl
And yes @acceptable-architect-70237 is correct, chart/dashboard stuff is still WIP and seems like I forgot to add their snapshots to Snapshot union :)
h
Thanks for the explanation! @steep-airplane-62865
b
Btw, sounds like you're building a DataHub POC? Which company are you with if you don't mind sharing?
h
I’ll DM you ☝️