Hi team, Could you pls share the detailed description of the physical data model of Datahub. We'd like to create or modify the metadata/lineage from the code level directly. Until now we've read python code and tried to ceate a new lineage, but it's not created correctly. We very appreciate that if you may simply guide us how to create the lineage via coding from the scratch.

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ button ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? ✅ button Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!

Actually we found some similar requests. The main solution recommends to use the API to create the lineage. But we prefer to create lineage flow via coding directly without API.

Hey Stefan, So I am working on something very similar and was curios to know what approach you took when creating the entities especially entities that represent datasets and their attributes.

There are quite a few resources for how to achieve this programmatically in our docs page. e.g. https://datahubproject.io/docs/api/tutorials/modifying-dataset-tags. Please take a look and let us know if this helps you. cc @delightful-ram-75848

@gifted-market-81341 Currently we use API to create entities. You may take this link as reference. https://datahubspace.slack.com/archives/CUMUWQU66/p1675806650114449 But the goal we'd like to achieve is to create entity via coding directly without API, which requires us to completely learn the implementing mechanism of the corresponding APIs. This is a huge challenge facing us.

Hey Stefan, so I managed to get most of the use cases I was interested in using the Python SDK either through the rest emitter or the graph emitter. The challenge we are facing is not being able to fully grasp the Python SDK, our efforts have been based on trial and error and going through the examples, but we can't really find a complete documentation for that exaplains the various classes, functions, etc. Any idea where I can find such documentation?

cc @delightful-ram-75848 @gray-shoe-75895! This is great feedback

Python SDK reference is here: https://datahubproject.io/docs/python-sdk/builder