Migration of business glossaries from Confluence to DataHub My client has plenty of glossaries (100+) defined on Confluence pages but would like to connect them somehow to data assets ingested to DataHub (dbt models, Snowflake tables, etc.). A couple of things to consider: • putting all definitions into yml and ingesting them to DH via recipes might be problematic as the glossaries most likely will be maintained by non-technical people • they also complained about some limitations of having all the definitions in yml (formatting etc.) • so maybe we should keep content as-is in Confluence and in DataHub's recipes we should only keep glossary entities and their relationship with each other plus

url

to corresponding Confluence pages? What would be your best bet here (or some good practices based on experience)? Thanks!

What I am doing right now, 1. Fetch the terms from Confluence using their API (sample:

<https://xxxx.atlassian.net/wiki/rest/api/content>

) and need to use BeautifulSoup as well to play around with the html tag etc 2. Push to Excel, Google Sheet 3. Write a python script to fetch from Excel and transform to yaml 4. Ingest them to Datahub

Thanks for the answer @gifted-knife-16120. It looks to me as a lot of custom development. Was the Excel part necessary? Was the whole process one-off one (just migration) or even now confluence is your glossary master? Do you remember how long did it take to build this solution end to end?

• no need Excel if don’t want extra backup 😄 • we moved to Excel now. The terms are stored in Excel because we are using SharePoint, so easier to collaborate with others. also, I add is_push column to indicate the new or existing. • 2 days including the testing. But, my script not cover the .yaml needed by DataHub. Only automate the Confluence + Excel + terms.yaml • So still need to copy

terms.yaml

into

recipe.yaml