Artem Astapenko
06/09/2021, 2:36 PMpython
command in macos is 2.7, however according the docs:
We assume `python` points to python >=3.7.
as a result build is failing ( however it was always built fine before ). What is a proper way to make it to be built with python3?
• just create alias in .zshrc
• pyenv
• some other tool?user
06/09/2021, 2:53 PMuser
06/09/2021, 2:53 PMuser
06/09/2021, 2:54 PMuser
06/09/2021, 3:06 PMArtem Astapenko
06/09/2021, 3:06 PMuser
06/09/2021, 3:07 PMuser
06/09/2021, 3:07 PMuser
06/09/2021, 3:42 PMArtem Astapenko
06/09/2021, 3:42 PMpythonBinary = 'python'
?user
06/09/2021, 3:43 PMuser
06/09/2021, 3:45 PMuser
06/09/2021, 3:45 PM* What went wrong:
Execution failed for task ':airbyte-integrations:bases:base-normalization:checkPython'.
> Python (/System/Library/Frameworks/Python.framework/Versions/2.7) verion 2.7.16 does not match minimal required version: 3.7
user
06/09/2021, 3:45 PMuser
06/09/2021, 3:45 PMArtem Astapenko
06/09/2021, 3:46 PMpython --version
prints Python 3.9.0
but when I run it - it still displays issue with versionuser
06/09/2021, 3:46 PMuser
06/09/2021, 3:47 PMpython
on my computer is python 2.whatever but python3 is python 3.7 so adding the pyton binary points it to 2 by accident?Artem Astapenko
06/09/2021, 3:48 PMusr/bin/python
and the one installed via brew is in usr/local/bin
but I do not think that removing the built-in one is a good ideauser
06/09/2021, 3:49 PMuser
06/09/2021, 3:49 PM> Task :airbyte-integrations:bases:base-normalization:unitTest FAILED
[python] .venv/bin/python -m pytest -s unit_tests
============================= test session starts ==============================
platform darwin -- Python 3.7.9, pytest-6.1.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/charles/code/airbyte/airbyte-integrations/bases/base-normalization
collected 0 items / 4 errors
==================================== ERRORS ====================================
_______ ERROR collecting unit_tests/test_destination_name_transformer.py _______
ImportError while importing test module '/Users/charles/code/airbyte/airbyte-integrations/bases/base-normalization/unit_tests/test_destination_name_transformer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.venv/lib/python3.7/importlib/__init__.py:151: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
unit_tests/test_destination_name_transformer.py:29: in <module>
from normalization.destination_type import DestinationType
normalization/__init__.py:2: in <module>
from normalization.transform_catalog.transform import TransformCatalog
normalization/transform_catalog/__init__.py:1: in <module>
from normalization.transform_catalog.transform import TransformCatalog
normalization/transform_catalog/transform.py:30: in <module>
import yaml
E ModuleNotFoundError: No module named 'yaml'
_____________ ERROR collecting unit_tests/test_stream_processor.py _____________
ImportError while importing test module '/Users/charles/code/airbyte/airbyte-integrations/bases/base-normalization/unit_tests/test_stream_processor.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.venv/lib/python3.7/importlib/__init__.py:151: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
unit_tests/test_stream_processor.py:30: in <module>
from airbyte_protocol.models.airbyte_protocol import DestinationSyncMode, SyncMode
E ModuleNotFoundError: No module named 'airbyte_protocol'
___________ ERROR collecting unit_tests/test_table_name_registry.py ____________
ImportError while importing test module '/Users/charles/code/airbyte/airbyte-integrations/bases/base-normalization/unit_tests/test_table_name_registry.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.venv/lib/python3.7/importlib/__init__.py:151: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
unit_tests/test_table_name_registry.py:32: in <module>
from normalization.transform_catalog.catalog_processor import CatalogProcessor
normalization/__init__.py:2: in <module>
from normalization.transform_catalog.transform import TransformCatalog
normalization/transform_catalog/__init__.py:1: in <module>
from normalization.transform_catalog.transform import TransformCatalog
normalization/transform_catalog/transform.py:30: in <module>
import yaml
E ModuleNotFoundError: No module named 'yaml'
_____________ ERROR collecting unit_tests/test_transform_config.py _____________
ImportError while importing test module '/Users/charles/code/airbyte/airbyte-integrations/bases/base-normalization/unit_tests/test_transform_config.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.venv/lib/python3.7/importlib/__init__.py:151: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
unit_tests/test_transform_config.py:30: in <module>
from normalization.transform_catalog.transform import extract_schema
normalization/__init__.py:2: in <module>
from normalization.transform_catalog.transform import TransformCatalog
normalization/transform_catalog/__init__.py:1: in <module>
from normalization.transform_catalog.transform import TransformCatalog
normalization/transform_catalog/transform.py:30: in <module>
import yaml
E ModuleNotFoundError: No module named 'yaml'
=========================== short test summary info ============================
ERROR unit_tests/test_destination_name_transformer.py
ERROR unit_tests/test_stream_processor.py
ERROR unit_tests/test_table_name_registry.py
ERROR unit_tests/test_transform_config.py
!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 4 errors in 0.19s ===============================
FAILURE: Build failed with an exception.
user
06/09/2021, 3:51 PMArtem Astapenko
06/09/2021, 3:52 PM./gradlew :airbyte-integrations:bases:base-normalization:build
./gradlew :airbyte-integrations:bases:base-normalization:cleanPythonVenv
./gradlew :airbyte-integrations:bases:base-normalization:build
this sequence of commands fails on the second build.user
06/09/2021, 3:52 PM./gradlew :airbyte-integrations:bases:base-normalization:build
./gradlew :airbyte-integrations:bases:base-normalization:cleanPythonVenv
./gradlew :airbyte-integrations:bases:base-normalization:clean
./gradlew :airbyte-integrations:bases:base-normalization:build
user
06/09/2021, 3:53 PMcleanPythonVenv
by itself is not really a safe operation and it always has to be paired with a clean. @s do you think we should codify that in the gradle setup? or should we just be documenting it that they have to be used together.user
06/09/2021, 3:56 PMArtem Astapenko
06/09/2021, 3:58 PMuser
06/09/2021, 3:58 PMuser
06/09/2021, 4:23 PMuser
06/09/2021, 5:55 PM> Task :airbyte-integrations:connectors:source-amazon-seller-partner:unitTest FAILED
unit_tests/test_source.py {"type": "LOG", "log": {"level": "INFO", "message": "Checking access to Amazon SP-API"}}
.
=================================== FAILURES ===================================
____________________________ test_get_cursor_state _____________________________
def test_get_cursor_state():
cursor_value = "2021-05-02"
end_date = "2021-06-02"
base_client = get_base_client(SP_CREDENTIALS)
> assert "2021-05-03" == base_client._get_cursor_state(cursor_value=cursor_value, end_date=end_date)
E AssertionError: assert '2021-05-03' == '2021-06-03'
Am I the only 1 who has this errorPing
06/09/2021, 5:55 PMuser
06/09/2021, 5:55 PMuser
06/09/2021, 5:56 PMCORE_ONLY=1
user
06/09/2021, 5:56 PMuser
06/09/2021, 5:57 PMuser
06/09/2021, 5:57 PMPing
06/09/2021, 5:58 PMERROR: for airbyte-temporal UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: for airbyte-temporal UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
user
06/09/2021, 5:59 PMCORE_ONLY=true ./gradlew --no-daemon build --scan -x test
user
06/09/2021, 6:06 PMuser
06/09/2021, 6:07 PM