Hi, The architecture of my data platform is such that data moves from Kafka to S3 to Snowflake. On Snowflake, we have written Airflow tasks that convert one table to another. I have ingested metadata from the three and it looks good. Now, I want to be able to view the lineage around this (Kafka -> S3 -> Snowflake table1 -> Airflow -> Snowflake table2 -> Airflow -> Snowflake table3). In this, do I have to manually add upstream and downstream for each of these platforms and draw the lineage or can it be done automatically somehow? PS. Our Snowflake is the Standard Edition and not the Enterprise Edition

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ button ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? ✅ button Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!

@big-carpet-38439 @astonishing-answer-96712 @better-orange-49102 Sorry for tagging you guys directly. But can you help here or align someone

Hey there, thanks for your patience here- you can do this programmatically with our API and SDKs-https://datahubproject.io/docs/api/tutorials/modifying-dataset-lineage

thanks