Hi, I am trying to check the versioning of DataHub...
# troubleshoot
a
Hi, I am trying to check the versioning of DataHub. To test this, I made the table named 'datahub_test' with three columns (account_id, asset_amount, asset_name) and ingested with hive. Therefore, those three columns are written 'Added in v0.0.0'. Next, I added two columns (avg_price, quantity) to the same table and ingested again. What I expected is that UI provides the two versions (v0.0.0, v1.0.0) and can see the schema according to the version. However, when I clicked Schema tab, there no existed the box where version appeared and two columns had no message (expected msg is added in v1.0.0) What's wrong with it? As far as I know, when technical schema change occurs (such as adding new columns), version is incremented by 1 and DataHub recognized it automatically when ingesting.
e
Hi there, are you seeing any errors in GMS logs when you checked this?
a
@echoing-airport-49548 You mean, give command "docker logs datahub-gms"? GMS logs (first figure) show the error message like "failed to send event to mixpanel". Also, the problem is solved after I waited for some time. ("quantity" and "avg_price" is added in v0.1.0 and the box enabling me to choose the version appeared) However, when I added one more column (stock_code), still the problem appeared again. What's strange is that using timeline API it recognized the change (as the third figure shows) BTW, why adding columns aren't regarded as "major change" but "minor change"? According to https://datahubproject.io/docs/dev-guides/timeline/ , "adding columns" is one of "technical schema change" and it should be considered as major change.
e
That is bizarre
@orange-night-91387 no changes were made recently here right?
I’m pretty sure they weren’t as CI would have caught that
Which version of DataHub are you on?
o
Field adds are a minor version change since they are backwards compatible, removes & modifies are both major, but it should still increment the minor version if you have added a field. When you added it, you added it to the underlying SchemaMetadata aspect, correct? If you directly modified the EditableSchemaMetadata aspect via API to add a field and did not also modify the underlying SchemaMetadata you might see visual oddities. It is very odd that the UI is not reflecting it for a period time and then it resolves. Seems like a frontend caching thing? I know there was some caching added at the browser layer .
a
I've solved it just waiting for more time (about several hours). Thanks for your answers!