Hey Team, few noob questions on the airflow integration. I have an amazon managed airflow environment (v 1.10.12). I am planning to try the https://datahubproject.io/docs/metadata-ingestion#emitting-lineage-via-a-separate-operator example. So my questions are, 1. In order to use

datahub

library in my dag, should I install

acryl-datahub

python package or a is there a different one for airflow ? 2. I am trying to add the datahub rest connection through the UI, if I have installed

acryl-datahub

package - should I see an option as

datahub

on the drop down ? or do I need to select a different option ? [screenshot attached]

@calm-addition-66352: 1. Yes

acryl-datahub

python package is the only one you need.

I don't believe you configure this through the UI... @gray-shoe-75895 to confirm

on 2. you won’t see it in the dropdown unfortunately - that’s only supported in Airflow 2.x. For Airflow 1.10.12, you should just leave the “conn type” field blank

Thanks guys, let me give it a try and see how it goes 🙂

@calm-addition-66352: how did this go for you?

Hey @mammoth-bear-12532 - sorry for the delay in replying. Had a pretty hectic week 😅 I did ran into few issues, so we ended up integrating it via the MCE files since it was only a POC and we only wanted to see how the pipelines can be represented in the DataHub side. Might re-look at this again when we come to implementation stage again. At the moment feedback for DataHub has been great 🙂