A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

datahub_upgrade_log.txt

Hi Everyone, We are trying to upgrade the Packages in Airflow , but it is upgrading the version of Airflow. Any suggestions how to resolve
Commands used-
python3 -m pip install 'acryl-datahub==0.8.32'
python3 -m pip install 'acryl-datahub[airflow,kafka,mysql,athena,glue,hive,kafka-connect,ldap,looker,lookml,okta,postgres,druid,redshift,sagemaker,snowflake,sql-profiles,sqlalchemy,datahub-rest,datahub-kafka,datahub-business-glossary,great-expectations,s3]==0.8.32'

Attaching the logs for reference

<@U027B7R23J4> <@UV14447EU> - Any suggestions ?

Can you specify a constraints file with that you can make sure it does not try to touch any existing package.

here is the official one from Airflow -&gt; <https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#constraints-files>

<@UV14447EU> - It is trying to upgrade Airflow itself

Are we able to give a constraint for airflow ?

In our setup.py we only say this:
```"apache-airflow &gt;= 1.10.2",```
You can specify constraints file to your pip command -&gt;
```pip install -c <https://raw.githubusercontent.com/apache/airflow/constraints-${airflow_version}/constraints-3.7.txt> 'acryl-datahub==0.8.32'```

Thank you <@UV14447EU>.Will try it out and update here.

<@UV14447EU> - Getting this -
The conflict is caused by:
acryl-datahub[airflow,athena,datahub-business-glossary,datahub-kafka,datahub-rest,druid,glue,great-expectations,hive,kafka,kafka-connect,ldap,looker,lookml,mysql,okta,postgres,redshift,s3,sagemaker,snowflake,sql-profiles,sqlalchemy] 0.8.32 depends on typing-extensions&gt;=3.10.0.2
The user requested (constraint) typing-extensions==3.7.4.3

Which version of python and airflow do you have?

<@UV14447EU>
Python 3.7.9
airflow version 2.0.2

Please, can you try with acryl datahub `0.8.34.2` there I fixed this issue with the constraints file of Airflow 2.0.2

<@UV14447EU> - Getting this when we try to install the 0.8.34.2 version

```ERROR: Cannot install acryl-datahub[airflow,athena,datahub-business-glossary,datahub-kafka,datahub-rest,druid,glue,great-expectations,hive,kafka,kafka-connect,ldap,looker,lookml,mysql,okta,postgres,redshift,s3,sagemaker,snowflake,sql-profiles,sqlalchemy]==0.8.34.2 because these package versions have conflicting dependencies.



The conflict is caused by:
acryl-datahub[airflow,athena,datahub-business-glossary,datahub-kafka,datahub-rest,druid,glue,great-expectations,hive,kafka,kafka-connect,ldap,looker,lookml,mysql,okta,postgres,redshift,s3,sagemaker,snowflake,sql-profiles,sqlalchemy] 0.8.34.2 depends on grpcio==1.44.0; extra == "kafka"
The user requested (constraint) grpcio==1.37.0```

<@UV14447EU> - Still facing issues when installing packages  ( trying with the latest 0.8.28 version of datahub )