is the schema history feature currently disabled i...
# troubleshootc
colossal-furniture-76714
09/23/2021, 3:57 PMis the schema history feature currently disabled in the ui?
g
green-football-43791
09/23/2021, 4:09 PMYes, we disabled this feature a few months ago because the way it was implemented caused confusion for our users.
green-football-43791
09/23/2021, 4:09 PMDid you use in previous versions? I'd be curious to hear if you found it useful / what you would look for in a schema history tool
green-football-43791
09/23/2021, 4:10 PMa few open questions we have:
1. how to group versions of a schema together
2. how to interweave schema history with the history of tags + description updates made from the UI
c
colossal-furniture-76714
09/23/2021, 4:14 PMNo i did not use it in the past, since we still work on a POC. We receive that comes from devices out there in the world which our technology teams can define by themselves. This leads to constant change of this dataset between release cycles. This is store in a column called doc that is in the end a long string which is accessible as a struct field. It would be quite nice to look at the schema for different releases eg ReleaseA, Release B etc.
For this specific use case we would be fine with not allowing updates made from the UI
g
green-football-43791
09/23/2021, 4:15 PMhow do you define releaseA, releaseB, etc?
green-football-43791
09/23/2021, 4:15 PMis this stored in the metadata of the schema somewhere?
c
colossal-furniture-76714
09/23/2021, 4:15 PMits similar to ios13 and ios14
colossal-furniture-76714
09/23/2021, 4:19 PMthe technology teams can change the schema containing the data the devices are supposed to transmit. But I'm not sure whether the release like ios13 is stored in the schema. But I guess that we could change that. I guess this is an edge case but the technology teams like wifi or networking have to define a schema any that the devices collect and send that data to us. This means that no updates in the ui are necessary and in general it's probably not our way to update much in the UI anyway.
colossal-furniture-76714
09/23/2021, 4:21 PMHow would i check out this feature now? Check out an old commit of the repository or is there a feature flag I can set?
g
green-football-43791
09/23/2021, 5:19 PMAh, unfortunately for now it is removed (without feature flag)
green-football-43791
09/23/2021, 5:20 PMthis was the PR that removed it- since then more of the code was removed in refactors: https://github.com/linkedin/datahub/pull/3099
green-football-43791
09/23/2021, 5:20 PMif you checked out the sha before that, you would see it
green-football-43791
09/23/2021, 5:21 PMhowever, this is a bit of an older version of Datahub- our metadata model and APIs have evolved since then
green-football-43791
09/23/2021, 5:21 PMits possible you may see strange behavior if you've ingested data since then, I'm not exactly sure
c
colossal-furniture-76714
09/24/2021, 9:29 AMokay and there is no other way to look a different schema versions of one dataset besides maybe using tags?
g
green-football-43791
09/24/2021, 7:30 PMGot it! Makes sense. For further context, reviving schema version history is in our plans for the next few weeks. It will work by treating each addition/removal/mutation of a column as a new "version" of the schema.
green-football-43791
09/24/2021, 7:30 PMWould that align with your expectations of schema versions?