A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi Everyone, we are trying to switch to MWAA (<https://aws.amazon.com/managed-workflows-for-apache-airflow/> ). We would like to understand how to integrate Datahub with MWAA and if there is any documentation for the same.

what exactly are you trying to do with MWAA?

We don't have any specific documentation for MWAA, but I would think you should still be able to integrate the Airflow instance with our Airflow integrations

There are definitely lots of quirks with MWAA and DataHub that we have discovered. <@UV14447EU> should be able to help.

<@UV14447EU> -  any details on this is really helpful. we are trying to understand the configurations and changes required for the Integration.

Sorry, I need to check with MWAA. I need some more time.

<@UV14447EU> - Thank you for looking into this. Please do let me know once you have the details,.

Which version of Airflow are you using in MWAA?

<@UV14447EU> - We are using 2.0.2 of Airflow.

<@UV14447EU> - Wanted to check if you had some details on this.

<@UV14447EU> - Hope you are doing well ! Wanted to check if you any details on MWAA.

<@UV14447EU> - Do we have any documentation on configuring and using MWAA

Hey,
We are going to release soon a new version of the airflow integration which can work with MWAA.
I will briefly demo it tomorrow on town hall if you can join.


<@UV14447EU> - Thats Awesome. will join tomorrow's Townhall

<@UV14447EU> - Thank you for the details ! Could you also share the documentation for the same

<@UV14447EU> Hi sorry to jump into the conversation. I couldn’t attend the town hall meeting. Is this MWAA integration available now? Is there any documentation on this? (I assume it is different from <https://datahubproject.io/docs/lineage/airflow/|this> .)

I’m just about to release the pypi package today

It is released now:
<https://pypi.org/project/acryl-datahub-airflow-plugin/0.8.35.6/>

Thanks for letting me know! It's awesome!

please, let me know if there is any issue with it as this is the first release, there can be hickups