Channels
c
charles
11/20/2020, 5:49 AM
does anyone else get these errors when they clean build? it's like a dependency isn't getting respected or something. curious if i'm the only one.
Copy code
> Task :airbyte-integrations:connectors:source-facebook-marketing-api-singer:unitTest FAILED
[python] .venv/bin/python -m pytest unit_tests
============================= test session starts ==============================
platform darwin -- Python 3.7.8, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/charles/code/airbyte/airbyte-integrations/connectors/source-facebook-marketing-api-singer
collected 0 items / 1 error
==================================== ERRORS ====================================
___________________ ERROR collecting unit_tests/unit_test.py ___________________
ImportError while importing test module '/Users/charles/code/airbyte/airbyte-integrations/connectors/source-facebook-marketing-api-singer/unit_tests/unit_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.venv/lib/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
unit_tests/unit_test.py:25: in <module>
from base_python import AirbyteLogger
base_python/__init__.py:25: in <module>
from .integration import AirbyteSpec, ConfigContainer, Destination, Integration, Source
base_python/integration.py:30: in <module>
from airbyte_protocol import AirbyteCatalog, AirbyteConnectionStatus, AirbyteMessage, ConnectorSpecification
airbyte_protocol/__init__.py:25: in <module>
from .models import (
airbyte_protocol/models/__init__.py:2: in <module>
from .airbyte_protocol import *
E ModuleNotFoundError: No module named 'airbyte_protocol.models.airbyte_protocol'
=========================== short test summary info ============================
ERROR unit_tests/unit_test.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.51s ===============================u
user
11/20/2020, 6:02 AMnot just for fb, it's for a bunch of the python integrations.
u
user
11/20/2020, 6:03 AMgetting the same error for mkto
u
user
11/20/2020, 6:03 AMyeah.
u
user
11/20/2020, 6:03 AMit feels like it's an order of operations thing.
u
user
11/20/2020, 6:04 AMI’m gonna go on a limb and say this is related to the recent build changes?
u
user
11/20/2020, 6:05 AMseems like a rasonable guess.
u
user
11/20/2020, 6:05 AMi have seen this exact same issue before.
u
user
11/20/2020, 6:05 AMbut i don't remember if it was before we got rid of python caching last time. or after.
u
user
11/20/2020, 6:05 AMah. here
u
user
11/20/2020, 6:05 AMu
user
11/20/2020, 6:05 AMsimilar but not quite the same.
u
user
11/20/2020, 6:05 AMi kinda think i saw this before the most recent build changes.
u
user
11/20/2020, 6:05 AMbut not 100% sure.
u
user
11/20/2020, 6:15 AMI think I see the problem
u
user
11/20/2020, 6:16 AMthere’s a race condition with generating python protocol class files
u
user
11/20/2020, 6:16 AMI think
u
user
11/20/2020, 6:17 AMintegrations should only run after the generation process has run, but I’m seeing the two steps run in parallel. generation cleans up output dir before putting files there so that would explain why they can’t find that module, it’s the generated stuff
u
user
11/20/2020, 6:20 AMu
user
11/20/2020, 6:23 AMdope.
u
user
11/20/2020, 6:24 AMdid it work?
u
user
11/20/2020, 6:24 AMlike adding a dependson or somethign?
u
user
11/20/2020, 6:24 AMincedentally, i
rm -rf .gradleu
user
11/20/2020, 6:24 AM🤔
u
user
11/20/2020, 6:24 AMhuh
u
user
11/20/2020, 6:24 AM Copy code
rm -rf .gradle
./gradlew clean
./gradlew buildu
user
11/20/2020, 6:24 AM^ this is what worked for me.
u
user
11/20/2020, 6:25 AMmaybe it needed to get reset since all of the caching changes?
u
user
11/20/2020, 6:25 AMlmk if it works for you?
u
user
11/20/2020, 6:25 AMgiving it a shot
u
user
11/20/2020, 6:33 AMstill getting same problem
u
user
11/20/2020, 7:31 AMit works for me to do
./gradlew :airbyte-integrations:connectors:source-facebook-marketing-api-singer:unitTestu
user
11/20/2020, 7:32 AMI'm wondering if we need a
implementation files(project(':airbyte-integrations:bases:airbyte-protocol').generateProtocolClassFiles.outputs)base-pythonu
user
11/20/2020, 7:33 AMbut I guess airbyteDocker should depend on build anyways
u
user
11/20/2020, 4:52 PMRunning the unit test tasks individually always seems fine. But when just hitting build it all doesn't work.
u
user
11/20/2020, 4:58 PMshould we install main reqs and test reqs before unit tests?
u
user
11/20/2020, 5:00 PMit seems that way.
u
user
11/20/2020, 5:02 PMwell is it that or is it that it is depending on a module that hasn't installed its own deps.
u
user
11/20/2020, 5:03 PMhaven't full dug into the stack trace.
3 Views