When I input a description from the UI and then re-ingest it, it overwrites what I inputed from the UI and disappears. Is it possible to prevent it from being overwritten by setting up a recipe or something?

how did u ingest the description from ui?

I edit the description directly from the front-end web UI.

you're doing it on a dataset field description?

Yes. Also the tags will disappear. I use transformers in my recipes.

that is strange, because UI edits go to the editableschemametadata aspect, but backend ingest writes to a schemametadata aspect so they normally do not overwrite each other

If the metadata exists in both, which one is displayed in the web UI?

for field description you should be able to see both when you edit it in UI

Thank you very much. I understand. I will check again in my environment.

np! datasets level tags seems to be overwriteable from both UI and backend (not very certain)

I've checked the table and the globalTags aspect has been overwritten. We will ingest periodically, and sometimes users add tags from the web UI, so we don't would like to lose user-added tags.😧

I recall there used to be globaltags field in datasetProperties and editabledatasetproperties, but seems like there is no longer a separation between UI and backend for dataset tags. Not sure what happened

I agree with the opinion that transformers should be enabled only the first time when a new table is created. However, if the database administrator is not the same as the data catalog administrator, the data catalog administrator will not know when a new table will be created, and will probably use the recipe with transformers enabled repeatedly.

There is still a "tag" field in dataset property aspect, but the transformer writes to globaltag aspect. I think we can solve this probably by changing the aspect used. need to give it a try later. (Not sure why it's marked as deprecated though🤔) Edit: Doesnt seem to show up in UI anymore. 🤨