Hi Everyone - I had a question about source lineag...
# ingestion
h
Hi Everyone - I had a question about source lineage. In DBT we track sources to our source db in our data warehouse - however ideally we'd like to add lineage prior to this db. We use stitch to move data from various sources into Snowflake. I wanted to ask about documenting sources prior to Snow: • Is there any roadmap to add stitch (or workaround) to monitor this EL workflow? • The lineage is fairly static (once setup we don't really modify - so this would be possible to manually manage in the absence of automated stitch ingestion). Is it possible to setup a manual file? And any example of lineage parsing with a manual file? I looked here which is very helpful....but wasn't 100% sure how the entities in the json file mapped to datasets/fields/lineage in datahub
m
Hi @handsome-airplane-62628, this is a great resource for DIY lineage emission: https://datahubproject.io/docs/metadata-ingestion/#using-as-a-library
h
Fantastic! Thank you - I'll take a look. Much appreciated!
b
No current plans around Stitch, but if you can make something general purpose we'd be happy to take a contribution 🙂