Hi, I have a custom data source and I need to emit...
# troubleshoot
v
Hi, I have a custom data source and I need to emit the data in said source with the Python SDK. I have emitted datasets and container entities, but they're separate... Currently, I have datasets such as
container_name.dataset_name
and containers with the name
container_name
. They exist independently of each other currently. I'd like to assign a set of datasets to the Container. How would I go about doing so? Thanks!
✅ 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: 1️⃣ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? 2️⃣ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues?
a
Hi could you elaborate what you mean by assign to the container? As in you’d like to deploy the data with the container for ingest?
v
Hi Paul, I'd like to put existing Dataset entities into an existing Container entity using an ingestion script (Python Emitter). I have an existing database that contains metadata and would like to migrate this data into Datahub. I'm currently following this guide to create Datasets: https://datahubproject.io/docs/api/tutorials/creating-datasets/ I've created Datasets and Containers and would now put these Datasets inside said containers. Thanks!
For example, this container on the demo datahub has datasets in it. I'm trying to put datasets into a container. https://demo.datahubproject.io/container/urn:li:container:SCHEMA/Entities?is_lineage_mode=false
Well, if anyone is looking for the answer... I simply had to emit with the ContainerClass in the aspect field of the MetadataProposalWrapper constructor. Simply put, emit using the container aspect of the Container Entity.