https://datahubproject.io logo
Join Slack
Powered by
Good morening, I'm trying to run `./gradlew build`...
# troubleshoot
b
Good morening, I'm trying to run 
./gradlew build
locally on an M1 mac and I'm running into this error
Copy code
Bens-MacBook-Pro:df-datahub benmarte$ ./gradlew build
Downloading <https://services.gradle.org/distributions/gradle-6.9.2-bin.zip>
......................................................................................................

Welcome to Gradle 6.9.2!

Here are the highlights of this release:
 - This is a small backport release.
 - Java 16 can be used to compile when used with Java toolchains
 - Dynamic versions can be used within plugin declarations
 - Native support for Apple Silicon processors

For more details see <https://docs.gradle.org/6.9.2/release-notes.html>

To honour the JVM settings for this build a single-use Daemon process will be forked. See <https://docs.gradle.org/6.9.2/userguide/gradle_daemon.html#sec:disabling_the_daemon>.
Daemon will be stopped at the end of the build 
Configuration on demand is an incubating feature.

> Configure project :metadata-integration:java:datahub-client
In else section

> Configure project :metadata-integration:java:spark-lineage
In else section
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See <http://www.slf4j.org/codes.html#StaticLoggerBinder> for further details.

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':metadata-integration:java:datahub-protobuf:compileJava'.
> Could not resolve all dependencies for configuration ':metadata-integration:java:datahub-protobuf:annotationProcessor'.
   > Failed to calculate the value of task ':metadata-integration:java:datahub-protobuf:compileJava' property 'javaCompiler'.
      > Unable to download toolchain matching these requirements: {languageVersion=11, vendor=any, implementation=vendor-specific}
         > Unable to download toolchain. This might indicate that the combination (version, architecture, release/early access, ...) for the requested JDK is not available.
            > Could not read '<https://api.adoptopenjdk.net/v3/binary/latest/11/ga/mac/aarch64/jdk/hotspot/normal/adoptopenjdk>' as it does not exist.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See <https://docs.gradle.org/6.9.2/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD FAILED in 1m 47s
Bens-MacBook-Pro:df-datahub benmarte$ ./gradlew :metadata-integration:java:datahub-protobuf:compileJava
To honour the JVM settings for this build a single-use Daemon process will be forked. See <https://docs.gradle.org/6.9.2/userguide/gradle_daemon.html#sec:disabling_the_daemon>.
Daemon will be stopped at the end of the build 
Configuration on demand is an incubating feature.
d
you should install java 11 next to java 8 on your machine and this should be fine. Gradle tries to install automatically if you don’t have but the java version it uses does not have m1 compatible jdk
b
d
i think yes
s
Everything is tested with openjdk. So I would say prefer that
b
Got a link @square-activity-64562?
s
yes should work
b
Yeah I dont see a mac version
s
I usually use https://sdkman.io/ to install jdk. I can see there is zulu present in sdkman. Not sure where it downloads from. zulu is openjdk AFAIK
b
I installed the dmg I found in the first link building now will report if I run into any other issues thanks guys
ok build is crashing now on this task, any ideas?
Copy code
INFO:
      ########### EXT COMPILER OPTIMIZATION ###########
      INFO: Platform      :
        Architecture: aarch64
        Compiler    : clang
      
      CPU baseline  :
        Requested   : 'min'
        Enabled     : none
        Flags       : none
        Extra checks: none
      
      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : none
        Generated   : none
      INFO: CCompilerOpt.cache_flush[817] : write cache to path -> /private/var/folders/ly/hvy7j1y530sgvwgxlvnylfp80000gn/T/pip-install-ae74p7rk/numpy_f9ecc4b34f774a748c0db7092c8c9980/build/temp.macosx-10.14-arm64-3.8/ccompiler_opt_cache_ext.py
      INFO:
      ########### CLIB COMPILER OPTIMIZATION ###########
      INFO: Platform      :
        Architecture: aarch64
        Compiler    : clang
      
      CPU baseline  :
        Requested   : 'min'
        Enabled     : none
        Flags       : none
        Extra checks: none
      
      CPU dispatch  :
        Requested   : 'max -xop -fma4'
        Enabled     : none
        Generated   : none
      INFO: CCompilerOpt.cache_flush[817] : write cache to path -> /private/var/folders/ly/hvy7j1y530sgvwgxlvnylfp80000gn/T/pip-install-ae74p7rk/numpy_f9ecc4b34f774a748c0db7092c8c9980/build/temp.macosx-10.14-arm64-3.8/ccompiler_opt_cache_clib.py
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> numpy

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

> Task :metadata-ingestion:runPreFlightScript FAILED

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':metadata-service:restli-api:compileMainGeneratedRestJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':metadata-ingestion:runPreFlightScript'.
> Process 'command 'scripts/datahub_preflight.sh'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at <https://help.gradle.org>

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See <https://docs.gradle.org/5.6.4/userguide/command_line_interface.html#sec:command_line_warnings>

BUILD FAILED in 58s
184 actionable tasks: 67 executed, 117 up-to-date
I get this error when I try to run 
./dev-without-neo4j.sh
Copy code
Creating elasticsearch   ... done
Creating datahub_neo4j_1     ... done
Creating zookeeper           ... done
Creating mysql               ... done
Creating elasticsearch-setup ... done
Creating mysql-setup         ... done
Creating broker              ... done
Creating schema-registry     ... done
Creating kafka-setup         ... done
Creating datahub-gms         ... done
Creating datahub_datahub-actions_1 ... done
Creating datahub-frontend-react    ... error

ERROR: for datahub-frontend-react  Cannot start service datahub-frontend-react: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "datahub-frontend/bin/datahub-frontend": stat datahub-frontend/bin/datahub-frontend: no such file or directory: unknown

ERROR: for datahub-frontend-react  Cannot start service datahub-frontend-react: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "datahub-frontend/bin/datahub-frontend": stat datahub-frontend/bin/datahub-frontend: no such file or directory: unknown
ERROR: Encountered errors while bringing up the project.
b
thats relevant to me too. i also tried to install openjdk8 and openjdk11 on the same machine, (which solved the jdk11 issue) but kept encountering some annotation issue. (I don't have access to internet)
Copy code
Task :datahub-graphql-core:compileJava 
/datahub/datahub-graphql-core/src/mainGeneratedGraphQL/java/com/linkedin/datahub/graphql/generated/VisualConfiguration.java:7: error: cannot find symbol @javax.annotation.processing.Generated(
symbol: class Generated
location: package javax.annotation.processing

<followed by all the other files in the same folder giving the same annotation error msg>
We was wondering if openjdk11 was not an appropriate JDK or something
s
Can you please try running these commands one by one from project root and let me know if it works for you?
Copy code
./gradlew :metadata-service:war:build
./gradlew :datahub-web-react:yarnLintFix
./gradlew :datahub-frontend:build -x yarnTest -x yarnLint
./docker/dev-without-neo4j.sh -d
I use these on M1 MAC and these work for me. Have not used them after a clean gradle for a while so not sure if something has changed since then or not
b
Will do and report back thanks 🙏
@square-activity-64562 yeah no luck still getting this error
Copy code
Building datahub-frontend-react
[+] Building 0.2s (8/8) FINISHED                                                                                                                                                                                                                                                                    
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                           0.0s
 => => transferring dockerfile: 32B                                                                                                                                                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                                0.0s
 => [internal] load metadata for <http://docker.io/library/alpine:3.14|docker.io/library/alpine:3.14>                                                                                                                                                                                                                                 0.2s
 => [base 1/3] FROM <http://docker.io/library/alpine:3.14@sha256:06b5d462c92fc39303e6363c65e074559f8d6b1363250027ed5053557e3398c5|docker.io/library/alpine:3.14@sha256:06b5d462c92fc39303e6363c65e074559f8d6b1363250027ed5053557e3398c5>                                                                                                                                                                      0.0s
 => CACHED [base 2/3] RUN addgroup -S datahub && adduser -S datahub -G datahub                                                                                                                                                                                                                 0.0s
 => CACHED [base 3/3] RUN apk --no-cache --update-cache --available upgrade     && apk --no-cache add curl openjdk8-jre                                                                                                                                                                        0.0s
 => CACHED [final 1/1] RUN echo 9002                                                                                                                                                                                                                                                           0.0s
 => exporting to image                                                                                                                                                                                                                                                                         0.0s
 => => exporting layers                                                                                                                                                                                                                                                                        0.0s
 => => writing image sha256:bb4bbdaf2ded7f7fd15f087740cf314182fa88279c4159d1a867d3b93485de24                                                                                                                                                                                                   0.0s
 => => naming to <http://docker.io/linkedin/datahub-frontend-react:debug|docker.io/linkedin/datahub-frontend-react:debug>                                                                                                                                                                                                                               0.0s
zookeeper is up-to-date
elasticsearch is up-to-date
mysql is up-to-date
broker is up-to-date
Starting elasticsearch-setup ... 
Starting elasticsearch-setup ... done
Starting mysql-setup         ... done
Starting kafka-setup         ... done
datahub-gms is up-to-date
datahub_datahub-actions_1 is up-to-date
Recreating 5c1b2dd6136d_datahub-frontend-react ... error

ERROR: for 5c1b2dd6136d_datahub-frontend-react  Cannot start service datahub-frontend-react: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "datahub-frontend/bin/playBinary": stat datahub-frontend/bin/playBinary: no such file or directory: unknown

ERROR: for datahub-frontend-react  Cannot start service datahub-frontend-react: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "datahub-frontend/bin/playBinary": stat datahub-frontend/bin/playBinary: no such file or directory: unknown
ERROR: Encountered errors while bringing up the project.
o
Instead of running:
Copy code
./gradlew :datahub-frontend:build -x yarnTest -x yarnLint
it should be:
Copy code
./gradlew :datahub-frontend:dist -x yarnTest -x yarnLint
You may also need to run:
Copy code
./gradlew :datahub-web-react:build -x yarnTest -x yarnLint
first as well
h
I might be late to the party, but I was able to build on a M1 using a JDK from here: https://www.azul.com/downloads/?version=java-8-lts&amp;os=macos&amp;architecture=arm-64-bit&amp;package=jdk
8 Views
Open in Slack
PreviousNext
Powered by