Hi there, I have a small question about using dataset properties within Datahub. Can I extract these properties in a transformer to populate, for instance, dataset glossary terms or owner fields? thanks!

Hi sibren! yes that's totally possible! Here are the docs on transformers: https://datahubproject.io/docs/metadata-ingestion/transformers/ and adding glossary terms and owners are actually examples for transformers there in the docs you can follow!

Hi Chris, thanks a lot for your response. What I actually meant is if it is possible to extract table properties (the ones you see in the "properties" tab when you look at a dataset in the UI), and subsequently use those properties to set glossary terms or owner fields? But I reckon I'd need to write a custom transformer to make a call to extract the properties? thanks

ah sorry for the misunderstanding on my part, but yes I believe you are correct in that a custom transformer would be your friend in this instance

Hi @nutritious-vegetable-81282, did you have any luck creating this custom transformer? I'm trying to do something similar except I want to set dataset properties and owner based on elements parsed from the description field. Currently having trouble defining one aspect based on the value of another at ingestion time and thought you might have encountered this as well. If so, was there something you did to solve it? Thanks!

Hi @bland-balloon-48379, so sorry I was on holiday for a month. And we decided to go a different route so unfortunately I don't have a solution. Curious if you found something new though. kind regards.

Hey @nutritious-vegetable-81282 sorry for the delay as well. Talked with Shirshanka briefly on this and it seems a new class would likely needed to be added to datahub that exposes multiple aspects to the transformer at the same time. However, due to time constraints we chose to go in another direction as well for the time being. Here's Shrishanka's comment on the task.