https://linen.dev logo
d

Davin Chia (Airbyte)

07/06/2021, 5:55 AM
Heads up Airbyte team, I've figured out a short term fix for the build failures we've been seeing. TLDR: • The test/publish flows will also run on ec2 instances on AMI images that contain all the dependencies required to build Airbyte. See https://github.com/airbytehq/airbyte/pull/4558/files for more context. Lmk if anyone sees issues/has questions! • The frontend builds will also run on the ec2 runner • In short we are fully on the ec2 runners now - should also see a mild boost in speeds!
u

user

07/06/2021, 6:47 AM
@Davin Chia (Airbyte) Is
virtualenv
the only thing that is missing ? I tried installing it but exception remains. I don't build it inside the docker though. I run gradle build from my host system which is Ubuntu 21.04. Any suggestions ? Here is partial call-stack:
Copy code
FAILURE: Build failed with an exception.

* Where:
Build file '/home/expert/work/sideprojects/airbyte/airbyte-cdk/python/build.gradle' line: 2

* What went wrong:
An exception occurred applying plugin request [id: 'airbyte-python']
> Failed to apply plugin 'airbyte-python'.
   > java.lang.ClassNotFoundException: precompiled_AirbytePython

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.plugins.InvalidPluginException: An exception occurred applying plugin request [id: 'airbyte-python']
        at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.exceptionOccurred(DefaultPluginRequestApplicator.java:207)
        at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugin(DefaultPluginRequestApplicator.java:189)
        at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.lambda$applyPlugins$1(DefaultPluginRequestApplicator.java:127)
        at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugins(DefaultPluginRequestApplicator.java:127)
        at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugins(DefaultPluginRequestApplicator.java:123)
        at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:117)
        at org.gradle.configuration.BuildOperationScriptPlugin$1.run(BuildOperationScriptPlugin.java:65)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:56)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.lambda$run$1(DefaultBuildOperationExecutor.java:74)
u

user

07/06/2021, 6:49 AM
hmm that's a different stacktrace than the one I was seeing though it seems to be related to the same plugin
u

user

07/06/2021, 6:50 AM
yes can you try and manually install
virtualenv
before running
./gradlew build
once more? something like
python3 -m pip install virtualenv==16.7.9 --user
u

user

07/06/2021, 6:58 AM
Yes, I did exactly that (from your PR) but it didn't change the error. Too bad stacktrace doesn't give much information. What is
airbyte-python
resposible for? Here is the most inner exception. Does it give you any ideas?
Copy code
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: precompiled_AirbytePython
        at AirbytePythonPlugin.apply(AirbytePythonPlugin.java:23)
        at AirbytePythonPlugin.apply(AirbytePythonPlugin.java:10)
        at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:43)
        at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:51)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:187)
        at org.gradle.api.internal.plugins.DefaultPluginManager.access$100(DefaultPluginManager.java:52)
        at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:282)
u

user

07/06/2021, 6:59 AM
No it doesn't. This makes me think there is some Gradle corruption or Java runtime mismatch.
u

user

07/06/2021, 6:59 AM
what Java and Python versions are you on?
d

Davin Chia (Airbyte)

07/06/2021, 7:00 AM
Java 16, Gradle 7.1 (I fixed few problems in your script to make it work) and Python 3.9.5
u

user

07/06/2021, 7:01 AM
that might be it. we are on Java 14, Gradle 6 and Python 3.7
u

user

07/06/2021, 7:02 AM
in the spirit of cyclomatic complexity I would minimise the change set and try again 😛
2 Views