Jenny Brown
06/08/2021, 8:57 PMcheckPython
step, and it’s unclear how to control which python it tries to use. It seems unable to see my conda python at all (despite it being in my path), and when it uses the brew python, virtualenv chokes with the following error.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':airbyte-integrations:bases:base-normalization:checkPython'.
> Python not found: .venv/bin/python. This must be a bug of virtualenv support, please report it (<https://github.com/xvik/gradle-use-python-plugin/issues>). You can disable virtualenv usage with 'python.scope = USER'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
Is there a way I can control what python is used for building? I’m sure I’m not the only one with a messy python situation and a preference to control which one it finds and tries to use. If this is a version issue, I might be able to pin brew’s version to 3.7 or 3.8 instead of 3.9 (hopefully).user
06/08/2021, 9:47 PMbrew install python
, it gives me 3.9 immediately, which turns up the virtualenv crash documented above. So that’s a dead end, even though it is pretty much a fresh current install.
Trying to install with brew install python@3.7
gives me a python install but it isn’t found by the gradle plugin as it’s not the default python in brew. I might be able to forcibly link that one by following brew’s manual instructions.brew install python@3.7
has worked for most of us? I think that's what we've had to do.pyenv
then install python 3.7 does this problem persist?brew install pyenv
pyenv install 3.7.10
# (fresh terminal window in case of path adjustments)
CORE_ONLY=1 ./gradlew build -x :airbyte-webapp:test
still results in python3 not being found at all by the gradle plugin.java.io.IOException: Cannot run program "python3" (in directory "/Users/jennybrown/dev/airbyte/airbyte/airbyte-integrations/bases/base-normalization"): error=2, No such file or directory
pyenv init
stuff. With that fixed, it’s found. My build appears to be progressing, now.minPythonVersion = '3.7'
$ python --version
pyenv: version `3.7.9' is not installed (set by /Users/jennybrown/dev/airbyte/airbyte/.python-version)
I never mentioned 3.7.9 even once in my manual installs. I either typed 3.7
or I typed 3.7.10
but never 9.--scan
flag to upload the build report?--scan
is useful in case anything goes sidewayspythonBinary = 'python'
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':airbyte-cdk:python:checkPython'.
> Python not found: .venv/bin/python. This must be a bug of virtualenv support, please report it (<https://github.com/xvik/gradle-use-python-plugin/issues>). You can disable virtualenv usage with 'python.scope = USER'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See <https://docs.gradle.org/6.7.1/userguide/command_line_interface.html#sec:command_line_warnings>
airbyte-cdk/.venv/
exist?airbyte-cdk/python/.venv/
does exist. It contains a .venv/lib/python3.9/
folder in it, which means we’re mixing python versions and picking up the broken 3.9 stuff.IMPORTANT: virtualenv creates local python copy (by default in). Copy is created from global python and later used instead of global python. If you want to change used python version in the environment, then manually remove.gradle/python
so it could be created again (from global python)..gradle/python
./gradlew clean
remove .gradle
?airbyte-cdk/python/.venv/
manually and then retrying will help.rm .gradle
in the root of the project.(base) Jennys-MacBook-Pro:airbyte jennybrown$ rm -rf airbyte-integrations/bases/base-python/.venv/
(base) Jennys-MacBook-Pro:airbyte jennybrown$ rm -rf airbyte-integrations/bases/airbyte-protocol/.venv/
(base) Jennys-MacBook-Pro:airbyte jennybrown$ rm -rf airbyte-integrations/bases/base-normalization/.venv/
(base) Jennys-MacBook-Pro:airbyte jennybrown$ rm -rf airbyte-cdk/python/.venv/
$ CORE_ONLY=1 ./gradlew build -x :airbyte-webapp:test --scan
Python (/System/Library/Frameworks/Python.framework/Versions/2.7) verion 2.7.16 does not match minimal required version: 3.7
So now it’s finding the system python but not the pyenv one.rm -rf ./**/.venv
rm -rf .gradle
in the root of the project too?./gradlew cleanPythonVenvs
to nuke all of them