👋 Hello Team! Has anyone tried to use DataHub 0.8.44 with Airflow 2.4? I'm getting a conflicting dependency with MarkupSafe. Datahub is requesting 2.0.1 but Flask 2.2.2 (which is an Airflow dependency) requires >= 2.1.1 which is the lastest relase of MarkupSafe (from March 2022)...

do those two packages need to be in the same project? for us, we just use the

acryl-datahub-airflow-plugin

with airflow which seems to work

Thanks for your reply @proud-table-38689! I can give it a try to the airflow plugin... I have some code I need to revisit for sure.... We have custom Operators to push some metadata, plus some custom sources for QuickSight and Superset..

cool, maybe you can try the K8 operator and see if that helps? I think it’s pretty cool that with the Airflow K8 operator, tasks can be any docker container which means it can be almost any language/runtime with any package-tree etc.

well, after installing

acryl-datahub-airflow-plugin

...