gray-agency-10420
03/30/2022, 10:25 AMacryl-datahub[airflow,great-expectations]==0.8.31.5
package?gray-agency-10420
03/30/2022, 11:26 AM- name: datahub_action
action:
module_name: datahub.integrations.great_expectations.action
class_name: DataHubValidationAction
server_url: <http://datahub-gms:8080>
Do you have any idea what can be the problem?brief-businessperson-12356
03/30/2022, 12:54 PMextraEnvs:
- name: AUTH_OIDC_ENABLED
value: "true"
- name: AUTH_OIDC_CLIENT_ID
value: datahub-frontend
- name: AUTH_OIDC_CLIENT_SECRET
value: <secret>
- name: AUTH_OIDC_DISCOVERY_URI
value: <https://keycloak.192.168.49.2.nip.io/realms/datahub/.well-known/openid-configuration>
- name: AUTH_OIDC_BASE_URL
value: <http://192.168.49.1:9002>
Then ran :
helm install datahub datahub/datahub --values values.yaml
However, when accessing http://192.168.49.1:9002 I get the message "Oops an error occurred" on the front end. The datahub-frontend pod logs show the following:
Unexpected exception[CompletionException: org.pac4j.core.exception.TechnicalException: javax.net.ssl.SSLHandshakeException: PKIX path building failed:sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
I note that @handsome-football-66174 had an similar error which was due to a problem with the java server not being able to trust the SSL certificate. I have tried to following the suggestions from @big-carpet-38439 and@orange-night-91387 on the thread here https://datahubspace.slack.com/archives/C029A3M079U/p1644017149655469?thread_ts=1643841330.686689&cid=C029A3M079U.
To create the certificate I used mkcert and ran the following commands
1) mkcert -install to create a local CA certificate and add it to the Java truststore
2) mkcert -key-file key.pem -cert-file cert.pem keycloak.192.168.49.2.nip.io *.keycloak.192.168.49.2.nip.io to create a locally trusted certificate
I then created a ConfigMap and copied the contents of the truststore (cacerts file) and added the JAVA_OPTS, extraVolumes and extraVolumeMounts similar to here: https://datahubspace.slack.com/archives/C029A3M079U/p1644017149655469?thread_ts=1643841330.686689&cid=C029A3M079U
Any help would be much appreciated! Thanks.bland-balloon-48379
03/30/2022, 1:59 PMdatahub-frontend
image:
DOCKER_BUILDKIT=1 docker build -t "$IMAGE_TAG" -f ./docker/datahub-frontend/Dockerfile
However, doing so eventually gives me the following error:
* What went wrong:
Execution failed for task ':li-utils:generateDataTemplate'.
> Process 'command '/usr/lib/jvm/java-1.8-openjdk/bin/java'' finished with non-zero exit value 1
As well as this further down in the stack trace:
> Task :datahub-frontend:compilePlayBinaryPlayRoutes
I haven't been able to diagnose this issue, at first glace it seems like java issue, perhaps something with the version. But I am confused as to why an error would occur in the first place as we have not made an substantive changes to the code from the source repo and the image should be built in a blank container.
Now I originally dealt with a series of errors about gradle being unable to download plugins and dependencies, so I had to make a few changes to allow these to be pulled in our network environment. I will list those below:
• Updated the distributionUrl
in gradle-wrapper.properties
to point to our Nexus repo
• Added a pluginManagement.repositories.maven
object with our nexus url and credentials to settings.gradle
• Added a new repository object for our nexus repo to repositories.gradle
and commented out all other repos (as they would be blocked when accessed directly and are configured to be proxied by our nexus repo)
• Added a few lines to the datahub-frontend Dockerfile
to accept nexus credentials as ARGs, set them as ENV variables, and used them to set -Dgradle.wrapperUser
& -Dgradle.wrapperPassword
in the GRADLE_OPTS
ENV and set new command line arguments -PsonatypeNexusUsername
& -PsonatypeNexusPassword
to the ./gradlew :datahub-frontend:dist
command.
Has anyone encountered a build issue like this before or is able to assist in the trouble shooting? I appreciate it and have a great week!breezy-portugal-43538
03/30/2022, 5:07 PMbulky-intern-2942
03/30/2022, 6:09 PMrich-policeman-92383
03/30/2022, 8:11 PMgreat_expectations --v3-api checkpoint run my_checkpoint
Using v3 (Batch Request) API
No module named `datahub.integrations.great_expectations.action` could be found in your plugins directory.
- Please verify your plugins directory is configured correctly.
- Please verify you have a module named `datahub.integrations.great_expectations.action` in your plugins directory.
adorable-flower-19656
03/31/2022, 5:40 AMred-window-75368
03/31/2022, 9:47 AMnumerous-morning-88512
03/31/2022, 9:55 AMable-rain-74449
03/31/2022, 1:40 PMicy-piano-35127
03/31/2022, 1:45 PMdatahub delete --urn urn:li:container:37c7bb069d2b23708574c0bdf835dea8 --entity_type container --hard
The information in the website after the command is in the print bellow.red-window-75368
03/31/2022, 2:15 PMdatahub docker check
says "datahub-gms is running but not healthy"
Any thoughts?flaky-airplane-82352
03/31/2022, 2:30 PMicy-piano-35127
03/31/2022, 6:55 PMclever-postman-6589
04/01/2022, 2:23 AMmammoth-fall-12031
04/01/2022, 4:53 AMbetter-orange-49102
04/01/2022, 5:54 AMbusy-ghost-21488
04/01/2022, 6:15 AMsticky-dawn-95000
04/01/2022, 7:25 AMbreezy-portugal-43538
04/01/2022, 9:21 AMmicroscopic-elephant-47912
04/01/2022, 11:34 AMblue-boots-43993
04/01/2022, 4:52 PMUnable to run quickstart - the following issues were detected:
- datahub-gms is running but not healthy
strong-spoon-1980
04/01/2022, 4:56 PMCaused by: com.typesafe.config.ConfigException$WrongType: env variables: auth.oidc.enabled has type STRING rather than BOOLEAN
This is happening for any variation of the OAUTH_OIDC_ENABLED env var. The YAML currently looks like this:
extraEnvs:
- name: "AUTH_OIDC_ENABLED"
value: "true" (have also tried just plain true)
- name: "AUTH_OIDC_CLIENT_ID"
value: "SNIP"
- name: "AUTH_OIDC_DISCOVERY_URI"
value: "<https://accounts.google.com/.well-known/openid-configuration>"
- name: "AUTH_OIDC_BASE_URL"
value: "SNIP"
- name: "AUTH_OIDC_SCOPE"
value: "openid profile email"
- name: "AUTH_OIDC_USER_NAME_CLAIM"
value: "email"
- name: "AUTH_OIDC_USER_NAME_CLAIM_REGEX"
value: "([^@]+)"
- name: "AUTH_OIDC_CLIENT_SECRET"
value: "SNIP"
blue-boots-43993
04/01/2022, 7:33 PMswift-breakfast-25077
04/02/2022, 2:16 PMdatahub docker quickstart (without source code),
I want to create a new user to login (with username and password). How can I do it ? where can i find user.props file ?shy-parrot-64120
04/02/2022, 9:29 PMdatahub-upgrade-job
***************************
APPLICATION FAILED TO START
***************************
Description:
The bean 'kafkaProducerFactory', defined in class path resource [org/springframework/boot/autoconfigure/kafka/KafkaAutoConfiguration.class], could not be registered. A bean with that name has already been defined in URL [jar:file:/datahub/datahub-u
pgrade/bin/datahub-upgrade.jar!/BOOT-INF/lib/factories.jar!/com/linkedin/gms/factory/kafka/KafkaProducerFactory.class] and overriding is disabled.
Action:
Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true
numerous-morning-88512
04/04/2022, 9:12 AMTypeError: 21:30:00 is of type timedelta which cannot be serialized
does anyone has experience how to solve it ?breezy-portugal-43538
04/04/2022, 10:38 AMbored-analyst-98685
04/04/2022, 11:34 AMdatahub
) we get an empty page and the following error:
react-dom.production.min.js:216 Error: Could not fetch logged in user from cache. + Exception while fetching data (/corpUser) : java.lang.RuntimeException: Failed to retrieve entities of type CorpUser
useGetAuthenticatedUser.tsx Uncaught (in promise) Error: Could not fetch logged in user from cache. + Exception while fetching data (/corpUser) : java.lang.RuntimeException: Failed to retrieve entities of type CorpUser.
Also this in the pods logs:
Caused by: com.linkedin.r2.message.rest.RestException: Received error 404 from server for URI <http://datahub-datahub-gms:8080/dataJobs>
As we are following the recommended procedure, would like to know if you have seen these issues before.