Hello everybody. I'm having Airflow version 2.4+ i...
# troubleshootg
gentle-camera-33498
10/24/2022, 1:49 PMHello everybody. I'm having Airflow version 2.4+ incompatibility issues with the backend lineage dependencies. Airflow requires SQL ALchemy 1.4+ while DataHub plugins require SQL Alchemy 1.3.24. Is there any task mapped to solving these dependency issues?
m
microscopic-mechanic-13766
10/24/2022, 3:50 PMThe version 0.8.44 of the airflow plugin uses 1.4.9 SQLAlchemy so by fixing the version of the package in the pip install command you might solve that problem
microscopic-mechanic-13766
10/24/2022, 3:51 PMI have a related question here that might be of interest.
g
gentle-camera-33498
10/24/2022, 3:52 PMI forced some things by hand, but some errors kept happening. This issue here reports precisely what I'm going through: https://github.com/datahub-project/datahub/issues/6145
gentle-camera-33498
10/24/2022, 3:53 PMThe SQL Alchemy version is not the only problem
m
microscopic-mechanic-13766
10/24/2022, 3:57 PMIs it compulsory to specify both the outlets and the inlets?? I ignore if they are both required (although it would lack of sense)
g
gentle-camera-33498
10/24/2022, 4:04 PMThe problem is that "_ inlets" and "_outlets" attributes don't exist anymore. Now It's just self.inlets and self.outlets.
gentle-camera-33498
10/24/2022, 4:06 PMAs the current DataHub backend code is looking for "_inlets", this error is generated:
Copy code
AttributeError: 'PythonOperator' object has no attribute '_inlets'gentle-camera-33498
10/24/2022, 4:07 PMgentle-camera-33498
10/24/2022, 4:08 PMIf you remove the SQL Alchemy global dependency for versions lower than 1.4 and adjust the above function to get the attributes correctly, then the plugin will be compatible with Airlfow 2.3+/2.4+
4 Views