I've been trying out Datahub, and I can't quite un...
# ingestiona
average-city-12965
11/30/2020, 2:26 PMI've been trying out Datahub, and I can't quite understand some parts of the metadata change events. For example, the field version in SchemaMetadata is mandatory in the MCE Avro schema, but at the same time the GMS is also responsible for automatically incrementing the dataset version for new changes. Are these versions related? Or is the version field used for some domain-specific style of versioning that is not related to Datahub?
m
mammoth-bear-12532
11/30/2020, 6:53 PM
@ambitious-battery-33996: do you want to take this one?
a
ambitious-battery-33996
11/30/2020, 7:36 PMSure .. let me do some more digging on the schema story in OSS
ambitious-battery-33996
12/02/2020, 4:10 AMLooks like the internal API model is open sourced as aspect model. Can you log a ticket? We need to change the definition of SchemaMetadata IMO
ambitious-battery-33996
12/02/2020, 4:11 AMversion, platform and audit stamps shouldn't even be part of SchemaMetadata.pdl
a
average-city-12965
12/02/2020, 8:09 AMThank you! This makes sense, as the current implementation sounds very different from the usage the documentation implies. Manually crafting audit stamps and versioning doesn't sound reasonable
a
ambitious-battery-33996
12/02/2020, 4:23 PMhttps://github.com/linkedin/datahub/issues/2026 I created this issue for tracking