Hello guys, it looks like with the pip version change, some plugins of pypi package are taking a lot of time to install. Issue is something like this: https://github.com/pypa/pip/issues/9215 New pip resolver takes a lot of time to install all deps everytime it is installed on a fresh cluster. For example, it took more than 30 minutes to install

acryl-datahub[s3]

on a databricks cluster and hence got timed out. Can the devs make a fix to this?

Hi @lemon-terabyte-66903 The latest pip version pip 22.0.4, which seems doesn't have above issue. Could you please execute below command python3 -m pip install --upgrade pip on pip 22.0.4 I am not facing above mentioned issue Even after upgrading to the latest pip you face the issue , then please share the datahub version and databricks cluster version you are using

Unfortunately, I can’t do that as it’s running on a databricks cluster and we do not have control to upgrade pip.

Hi Dilip - the pip resolver often takes a really long time for backtracking, as in the issue you linked. The easiest way to solve this is to get a working setup on a local machine (e.g.

pip install acryl-datahub[s3]

, then dump all of the exact version numbers using

pip freeze

, and pass the results of pip freeze into the databricks cluster so it knows exactly what to install and doesn’t need to do any dependency resolution or backtracking

thanks harshal, that makes. sense