Hello Team, I'm trying to build custom frontend im...
# troubleshoot
m
Hello Team, I'm trying to build custom frontend image. However the build returns a below error Can someone help me ?
Copy code
=> ERROR [prod-install 1/5] COPY ./datahub-frontend.zip /                                                 0.0s
------
 > [prod-install 1/5] COPY ./datahub-frontend.zip /:
------
failed to compute cache key: "/datahub-frontend.zip" not found: not found
d
Hi @modern-answer-65441, have you compiled DataHub before building the image? Maybe this thread should help: https://datahubspace.slack.com/archives/C029A3M079U/p1673287083961339?thread_ts=1673287078.914179&cid=C029A3M079U
m
Thanks for pointing in the direction @delightful-ram-75848 But my gradle buildfails for the below reason can you help me command:
Copy code
sudo ./gradlew :datahub-frontend:dist -x test -x yarnTest -x yarnLint
Error:
Copy code
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':li-utils:compileMainGeneratedDataTemplateJava'.
> Could not find tools.jar. Please check that /usr/lib/jvm/java-8-openjdk-amd64 contains a valid JDK installation.
I'm wondering why it is using java8 when I actually installed java 11 and set my java home to
Copy code
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
Copy code
#24 933.7 > Task :metadata-service:auth-config:compileJava
#24 933.7 /datahub-src/metadata-service/auth-config/src/main/java/com/datahub/authorization/AuthorizationConfiguration.java:3: error: package com.datahub.plugins.auth.authorization does not exist
#24 933.7 import com.datahub.plugins.auth.authorization.Authorizer;
#24 933.7                                              ^
#24 933.7 /datahub-src/metadata-service/auth-config/src/main/java/com/datahub/authorization/AuthorizerConfiguration.java:3: error: package com.datahub.plugins.auth.authorization does not exist
#24 933.7 import com.datahub.plugins.auth.authorization.Authorizer;
#24 933.7                                              ^
#24 933.7 2 errors
#24 933.9
#24 933.9 > Task :metadata-service:auth-config:compileJava FAILED
I don't know why I get import errors I didn't make any change
@brainy-tent-14503 do you know why this error shows up ?
I'm building a custom datahub frontend image by overwriting the Dockerfile with the below one
Copy code
# Defining environment
ARG APP_ENV=prod

FROM alpine:3.14 AS base

RUN addgroup -S datahub && adduser -S datahub -G datahub

# Upgrade Alpine and base packages
RUN apk --no-cache --update-cache --available upgrade \
    && apk --no-cache add curl \
    && apk --no-cache add openjdk11-jre --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community

FROM --platform=$BUILDPLATFORM node:16.13.0-alpine3.14 AS prod-build

# Upgrade Alpine and base packages
RUN apk --no-cache --update-cache --available upgrade \
    && apk --no-cache add perl openjdk8 openjdk11

ARG USE_SYSTEM_NODE="true"
ENV CI=true
ENV GRADLE_OPTS="-Xms256m -Xmx512m"

# copy only what's realy needed for the build
# to get the benefits of caching as much as possible
COPY ./datahub-frontend ./datahub-src/datahub-frontend
COPY ./entity-registry ./datahub-src/entity-registry
COPY ./buildSrc ./datahub-src/buildSrc
COPY ./datahub-web-react ./datahub-src/datahub-web-react
COPY ./li-utils ./datahub-src/li-utils
COPY ./metadata-models ./datahub-src/metadata-models
COPY ./metadata-models-validator ./datahub-src/metadata-models-validator
COPY ./metadata-utils ./datahub-src/metadata-utils
COPY ./metadata-service ./datahub-src/metadata-service
COPY ./metadata-io ./datahub-src/metadata-io
COPY ./datahub-graphql-core ./datahub-src/datahub-graphql-core
COPY ./gradle ./datahub-src/gradle
COPY repositories.gradle gradle.properties gradlew settings.gradle build.gradle ./datahub-src/

RUN cd datahub-src \
    && ./gradlew :datahub-web-react:build -x test -x yarnTest -x yarnLint \
    && ./gradlew :datahub-frontend:dist -PuseSystemNode=${USE_SYSTEM_NODE} -x test -x yarnTest -x yarnLint \
    && cp datahub-frontend/build/distributions/datahub-frontend.zip ../datahub-frontend.zip \
    && cd .. && rm -rf datahub-src && unzip datahub-frontend.zip

FROM base as prod-install
COPY --from=prod-build /datahub-frontend /datahub-frontend/
COPY ./docker/monitoring/client-prometheus-config.yaml /datahub-frontend/
RUN wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.4.1/opentelemetry-javaagent-all.jar -O opentelemetry-javaagent-all.jar \
    && wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.16.1/jmx_prometheus_javaagent-0.16.1.jar -O jmx_prometheus_javaagent.jar
RUN chown -R datahub:datahub /datahub-frontend && chmod 755 /datahub-frontend

FROM base as dev-install
# Dummy stage for development. Assumes code is built on your machine and mounted to this image.
# See this excellent thread https://github.com/docker/cli/issues/1134
VOLUME [ "/datahub-frontend" ]

FROM ${APP_ENV}-install as final
USER datahub

ARG SERVER_PORT=9002
ENV SERVER_PORT=$SERVER_PORT
RUN echo $SERVER_PORT
EXPOSE $SERVER_PORT

HEALTHCHECK --start-period=2m --retries=4 CMD curl --fail http://localhost:$SERVER_PORT/admin || exit 1
COPY ./docker/datahub-frontend/start.sh /
CMD ./start.sh
the version I'm using is v0.9.6.1
b
There are parts of the code which have to be compiled with jdk8 for Spark compatibility. Certain modules thus will request the jdk8 toolkit. The artifacts are compatible with the other modules using jdk11 because of backwards compatibility. To fully build the project both jdks are required, especially if gradle cannot detect and fetch the required toolchain at build time automatically. For m1 mac, I manually install a build for apple arm for both 8 and 11. The primary jdk is preferred to be jdk11 with jdk8 being installed somewhere that gradle can find it. There is a blog here which describes toolchains in gradle.