Guys what is the best way to manage python versions Default

Question

Guys what is the best way to manage python versions Default version for `python` command in macos is 2 7 however according the docs ```We assume `python` points to python gt =3 7 ``` as a result build is failing however it was always built fine before What is a proper way to make it to be built with python3 just create alias in zshrc pyenv some other tool

user · Answer

the simplest for me was to uninstall all the default python binaries and use pyenv

user · Answer

it just started failing today

user · Answer

well probably I did not rebuild it for a quite long period slightly smiling face

user · Answer

i think i am the reason

Artem Astapenko · Answer

user · Answer

this assumption started recently

user · Answer

lmk if you re still having issues artem

user · Answer

sherif what about this commit should have caused this

Artem Astapenko · Answer

is it because we added `pythonBinary = python `

user · Answer

or some other line in there

user · Answer

ah i m getting a python version issue now too

user · Answer

``` What went wrong Execution failed for task airbyte integrations bases base normalization checkPython gt Python System Library Frameworks Python framework Versions 2 7 verion 2 7 16 does not match minimal required version 3 7```

user · Answer

pythonBinary

user · Answer

removing

Artem Astapenko · Answer

well yeah it s not working for me slightly smiling face I tried to alias it via zshrc and `python version` prints `Python 3 9 0` but when I run it it still displays issue with version

user · Answer

user · Answer

well my guess is that `python` on my computer is python 2 whatever but python3 is python 3 7 so adding the pyton binary points it to 2 by accident

Artem Astapenko · Answer

Its the default for all macos python is 2 and it is in `usr bin python` and the one installed via brew is in `usr local bin` but I do not think that removing the built in one is a good idea

user · Answer

i m seeing a new error now

user · Answer

``` gt Task airbyte integrations bases base normalization unitTest FAILED python venv bin python m pytest s unit tests ============================= test session starts ============================== platform darwin Python 3 7 9 pytest 6 1 2 py 1 10 0 pluggy 0 13 1 rootdir Users charles code airbyte airbyte integrations bases base normalization collected 0 items 4 errors ==================================== ERRORS ==================================== ERROR collecting unit tests test destination name transformer py ImportError while importing test module Users charles code airbyte airbyte integrations bases base normalization unit tests test destination name transformer py Hint make sure your test modules packages have valid Python names Traceback venv lib python3 7 importlib init py 151 in import module return bootstrap gcd import name level package level unit tests test destination name transformer py 29 in lt module gt from normalization destination type import DestinationType normalization init py 2 in lt module gt from normalization transform catalog transform import TransformCatalog normalization transform catalog init py 1 in lt module gt from normalization transform catalog transform import TransformCatalog normalization transform catalog transform py 30 in lt module gt import yaml E ModuleNotFoundError No module named yaml ERROR collecting unit tests test stream processor py ImportError while importing test module Users charles code airbyte airbyte integrations bases base normalization unit tests test stream processor py Hint make sure your test modules packages have valid Python names Traceback venv lib python3 7 importlib init py 151 in import module return bootstrap gcd import name level package level unit tests test stream processor py 30 in lt module gt from airbyte protocol models airbyte protocol import DestinationSyncMode SyncMode E ModuleNotFoundError No module named airbyte protocol ERROR collecting unit tests test table name registry py ImportError while importing test module Users charles code airbyte airbyte integrations bases base normalization unit tests test table name registry py Hint make sure your test modules packages have valid Python names Traceback venv lib python3 7 importlib init py 151 in import module return bootstrap gcd import name level package level unit tests test table name registry py 32 in lt module gt from normalization transform catalog catalog processor import CatalogProcessor normalization init py 2 in lt module gt from normalization transform catalog transform import TransformCatalog normalization transform catalog init py 1 in lt module gt from normalization transform catalog transform import TransformCatalog normalization transform catalog transform py 30 in lt module gt import yaml E ModuleNotFoundError No module named yaml ERROR collecting unit tests test transform config py ImportError while importing test module Users charles code airbyte airbyte integrations bases base normalization unit tests test transform config py Hint make sure your test modules packages have valid Python names Traceback venv lib python3 7 importlib init py 151 in import module return bootstrap gcd import name level package level unit tests test transform config py 30 in lt module gt from normalization transform catalog transform import extract schema normalization init py 2 in lt module gt from normalization transform catalog transform import TransformCatalog normalization transform catalog init py 1 in lt module gt from normalization transform catalog transform import TransformCatalog normalization transform catalog transform py 30 in lt module gt import yaml E ModuleNotFoundError No module named yaml =========================== short test summary info ============================ ERROR unit tests test destination name transformer py ERROR unit tests test stream processor py ERROR unit tests test table name registry py ERROR unit tests test transform config py Interrupted 4 errors during collection ============================== 4 errors in 0 19s =============================== FAILURE Build failed with an exception ```

user · Answer

i think i ve diagnosed it

Artem Astapenko · Answer

``` gradlew airbyte integrations bases base normalization build gradlew airbyte integrations bases base normalization cleanPythonVenv gradlew airbyte integrations bases base normalization build``` this sequence of commands fails on the second build

user · Answer

this sequence is okay ``` gradlew airbyte integrations bases base normalization build gradlew airbyte integrations bases base normalization cleanPythonVenv gradlew airbyte integrations bases base normalization clean gradlew airbyte integrations bases base normalization build```

user · Answer

my guess is that running `cleanPythonVenv` by itself is not really a safe operation and it always has to be paired with a clean < s> do you think we should codify that in the gradle setup or should we just be documenting it that they have to be used together

user · Answer

I d prefer that if they re strictly required to happen together codify it Don t rely on human variance in accurate steps if automation can make it right

Artem Astapenko · Answer

I think it s because the clean venv task has to also get rid of other secondary python artifacts like the eggs and what not

user · Answer

clean is very sane to do here

user · Answer

Fix worked < s> thank you for it

user · Answer

now I have another error ``` gt Task airbyte integrations connectors source amazon seller partner unitTest FAILED unit tests test source py type LOG log level INFO message Checking access to Amazon SP API =================================== FAILURES =================================== test get cursor state def test get cursor state cursor value = 2021 05 02 end date = 2021 06 02 base client = get base client SP CREDENTIALS gt assert 2021 05 03 == base client get cursor state cursor value=cursor value end date=end date E AssertionError assert 2021 05 03 == 2021 06 03 ``` Am I the only 1 who has this error

Ping · Answer

even though I am running gradlew clean build x test

user · Answer

this seems like a legitimate error

user · Answer

can you build with `CORE ONLY=1`

user · Answer

this is a problem with a connector which I don t think you re trying to build

user · Answer

I tried to rebuild it without core only because airbyte temporal was not starting with 60sec timeout exception so I decided that total rebuilt could help

user · Answer

building with core only should still rebuild temporal

Ping · Answer

but this is the error I get with temporal ```ERROR for airbyte temporal UnixHTTPConnectionPool host= localhost port=None Read timed out read timeout=60 ERROR for airbyte temporal UnixHTTPConnectionPool host= localhost port=None Read timed out read timeout=60 ERROR An HTTP request took too long to complete Retry with verbose to obtain debug information If you encounter this issue regularly because of slow network conditions consider setting COMPOSE HTTP TIMEOUT to a higher value current value 60 ```

user · Answer

the command I am using is `CORE ONLY=true gradlew no daemon build scan x test`

user · Answer

worked now Strange

user · Answer

is it possible that it was not starting due to some non up to date info stored in volumes