hey everyone! 👋 In our airflow instance we run subdags within one main dag in order to manage dependencies between workflows. We’re noticing that this results in a broken lineage between parent dag, and the sub-dag. Has anyone noticed this before and found a work around?

Good catch. I think currently there is an assumption in the definition of the DataFlow/Pipeline class that they dont have any upstream/downstream lineage property on their own, but only contains DataJobs /Tasks.

it seems that way. In datahub terms I think we’ve done the equivalent of connecting pipelines together. I’d guess the underlying graph structure would need to be extended to account for this type of hierarchy. Something like Pipeline -> executes -> Pipeline

@chilly-holiday-80781 is looking into fixing this