https://linen.dev logo
c

charles

11/20/2020, 5:49 AM
does anyone else get these errors when they clean build? it's like a dependency isn't getting respected or something. curious if i'm the only one.
Copy code
> Task :airbyte-integrations:connectors:source-facebook-marketing-api-singer:unitTest FAILED
[python] .venv/bin/python -m pytest unit_tests
         ============================= test session starts ==============================
         platform darwin -- Python 3.7.8, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
         rootdir: /Users/charles/code/airbyte/airbyte-integrations/connectors/source-facebook-marketing-api-singer
         collected 0 items / 1 error

         ==================================== ERRORS ====================================
         ___________________ ERROR collecting unit_tests/unit_test.py ___________________
         ImportError while importing test module '/Users/charles/code/airbyte/airbyte-integrations/connectors/source-facebook-marketing-api-singer/unit_tests/unit_test.py'.
         Hint: make sure your test modules/packages have valid Python names.
         Traceback:
         .venv/lib/python3.7/importlib/__init__.py:127: in import_module
             return _bootstrap._gcd_import(name[level:], package, level)
         unit_tests/unit_test.py:25: in <module>
             from base_python import AirbyteLogger
         base_python/__init__.py:25: in <module>
             from .integration import AirbyteSpec, ConfigContainer, Destination, Integration, Source
         base_python/integration.py:30: in <module>
             from airbyte_protocol import AirbyteCatalog, AirbyteConnectionStatus, AirbyteMessage, ConnectorSpecification
         airbyte_protocol/__init__.py:25: in <module>
             from .models import (
         airbyte_protocol/models/__init__.py:2: in <module>
             from .airbyte_protocol import *
         E   ModuleNotFoundError: No module named 'airbyte_protocol.models.airbyte_protocol'
         =========================== short test summary info ============================
         ERROR unit_tests/unit_test.py
         !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
         =============================== 1 error in 0.51s ===============================
u

user

11/20/2020, 6:02 AM
not just for fb, it's for a bunch of the python integrations.
u

user

11/20/2020, 6:03 AM
getting the same error for mkto
u

user

11/20/2020, 6:03 AM
yeah.
u

user

11/20/2020, 6:03 AM
it feels like it's an order of operations thing.
u

user

11/20/2020, 6:04 AM
I’m gonna go on a limb and say this is related to the recent build changes?
u

user

11/20/2020, 6:05 AM
seems like a rasonable guess.
u

user

11/20/2020, 6:05 AM
i have seen this exact same issue before.
u

user

11/20/2020, 6:05 AM
but i don't remember if it was before we got rid of python caching last time. or after.
u

user

11/20/2020, 6:05 AM
ah. here
u

user

11/20/2020, 6:05 AM
similar but not quite the same.
u

user

11/20/2020, 6:05 AM
i kinda think i saw this before the most recent build changes.
u

user

11/20/2020, 6:05 AM
but not 100% sure.
u

user

11/20/2020, 6:15 AM
I think I see the problem
u

user

11/20/2020, 6:16 AM
there’s a race condition with generating python protocol class files
u

user

11/20/2020, 6:16 AM
I think
u

user

11/20/2020, 6:17 AM
integrations should only run after the generation process has run, but I’m seeing the two steps run in parallel. generation cleans up output dir before putting files there so that would explain why they can’t find that module, it’s the generated stuff
u

user

11/20/2020, 6:23 AM
dope.
u

user

11/20/2020, 6:24 AM
did it work?
u

user

11/20/2020, 6:24 AM
like adding a dependson or somethign?
u

user

11/20/2020, 6:24 AM
incedentally, i
rm -rf .gradle
and then rebuilt and it all worked fine.
u

user

11/20/2020, 6:24 AM
🤔
u

user

11/20/2020, 6:24 AM
huh
u

user

11/20/2020, 6:24 AM
Copy code
rm -rf .gradle
./gradlew clean
./gradlew build
u

user

11/20/2020, 6:24 AM
^ this is what worked for me.
u

user

11/20/2020, 6:25 AM
maybe it needed to get reset since all of the caching changes?
u

user

11/20/2020, 6:25 AM
lmk if it works for you?
u

user

11/20/2020, 6:25 AM
giving it a shot
u

user

11/20/2020, 6:33 AM
still getting same problem
u

user

11/20/2020, 7:31 AM
it works for me to do
./gradlew :airbyte-integrations:connectors:source-facebook-marketing-api-singer:unitTest
on a clean repo
u

user

11/20/2020, 7:32 AM
I'm wondering if we need a
implementation files(project(':airbyte-integrations:bases:airbyte-protocol').generateProtocolClassFiles.outputs)
in
base-python
u

user

11/20/2020, 7:33 AM
but I guess airbyteDocker should depend on build anyways
u

user

11/20/2020, 4:52 PM
Running the unit test tasks individually always seems fine. But when just hitting build it all doesn't work.
u

user

11/20/2020, 4:58 PM
should we install main reqs and test reqs before unit tests?
u

user

11/20/2020, 5:00 PM
it seems that way.
u

user

11/20/2020, 5:02 PM
well is it that or is it that it is depending on a module that hasn't installed its own deps.
u

user

11/20/2020, 5:03 PM
haven't full dug into the stack trace.
3 Views