Hello all, newbie here, I have a MSQL source which I have ingested in to datahub. I also have a Yaml file for each table that has more meta data info about that table in key:value pair. What would be best approach to load those details in the properties page in the data hub UI please?

Hi @flaky-soccer-57765, you could use a transformer to add your custom properties for each dataset. Please take a look at this documentation.

Thanks @helpful-optician-78938.

I think you need to follow some convention for naming your files. Depending on the size of the files, you can either bundle all metadata per database in one file and load it or load individual files. Essentially, you should have an effective way of going from a dataset to its corresponding metadata in a file based on a simple lookup convention.

That makes sense @helpful-optician-78938. I will have the metadata files the same as the name of the dataset. how do I loop through the datasets while adding properties to it? Is there any relevant use case example available where I can take reference from? The property addition class seems to load the meta data to all the datasets available in that ingestion.