nice-yak-49999
07/13/2021, 8:12 AMmysql-setup-job
container only run this sql?
-- create datahub database
CREATE DATABASE IF NOT EXISTS DATAHUB_DB_NAME;
USE DATAHUB_DB_NAME;
-- create metadata aspect table
create table if not exists metadata_aspect_v2 (
urn varchar(500) not null,
aspect varchar(200) not null,
version bigint(20) not null,
metadata longtext not null,
systemmetadata longtext,
createdon datetime(6) not null,
createdby varchar(255) not null,
createdfor varchar(255),
constraint pk_metadata_aspect_v2 primary key (urn,aspect,version)
);
-- create default records for datahub user if not exists
CREATE TABLE temp_metadata_aspect_v2 LIKE metadata_aspect_v2;
INSERT INTO temp_metadata_aspect_v2 (urn, aspect, version, metadata, createdon, createdby) VALUES(
'urn:li:corpuser:datahub',
'corpUserInfo',
0,
'{"displayName":"Data Hub","active":true,"fullName":"Data Hub","email":"<mailto:datahub@linkedin.com|datahub@linkedin.com>"}',
now(),
'urn:li:principal:datahub'
), (
'urn:li:corpuser:datahub',
'corpUserEditableInfo',
0,
'{"skills":[],"teams":[],"pictureLink":"<https://raw.githubusercontent.com/linkedin/datahub/master/datahub-web/packages/data-portal/public/assets/images/default_avatar.png>"}',
now(),
'urn:li:principal:datahub'
);
-- only add default records if metadata_aspect is empty
INSERT INTO metadata_aspect_v2
SELECT * FROM temp_metadata_aspect_v2
WHERE NOT EXISTS (SELECT * from metadata_aspect_v2);
DROP TABLE temp_metadata_aspect_v2;
-- create metadata index table
CREATE TABLE IF NOT EXISTS metadata_index (
`id` BIGINT NOT NULL AUTO_INCREMENT,
`urn` VARCHAR(200) NOT NULL,
`aspect` VARCHAR(150) NOT NULL,
`path` VARCHAR(150) NOT NULL,
`longVal` BIGINT,
`stringVal` VARCHAR(200),
`doubleVal` DOUBLE,
CONSTRAINT id_pk PRIMARY KEY (id),
INDEX longIndex (`urn`,`aspect`,`path`,`longVal`),
INDEX stringIndex (`urn`,`aspect`,`path`,`stringVal`),
INDEX doubleIndex (`urn`,`aspect`,`path`,`doubleVal`)
);
fancy-helmet-32669
07/13/2021, 1:01 PMnice-yak-49999
07/14/2021, 8:01 AMblue-holiday-20644
07/14/2021, 2:33 PMrapid-sundown-8805
07/15/2021, 1:02 PMsquare-activity-64562
07/16/2021, 7:23 AMdocker run -it --rm --entrypoint=bash --tty linkedin/datahub-ingestion:v0.8.6
datahub@2523f447d4db:/$ datahub version
DataHub CLI version: unavailable (installed editable via git)
Python version: 3.8.11 (default, Jun 29 2021, 19:54:56)
[GCC 8.3.0]
datahub@2523f447d4db:/$
crooked-leather-44416
07/19/2021, 3:22 PM./gradlew build
that has all necessary dependencies (Gradle, Python, etc.)?square-activity-64562
07/21/2021, 7:22 PMrapid-sundown-8805
07/22/2021, 3:08 PM:9200
(the port) to the elastic URI, although the AWS ElasticSeararch doesn't work that way (it's reached just <https://elastic-host.domain.com>
without the port). This causes the elasticearch init-job to fail,
2021/07/22 143855 Waiting for: https://myuser:mypassword@search-mydomain.eu-central-1.es.amazonaws.com:9200I got past this step by setting it to yaml null, `~`:
global:
elasticsearch:
port: ~
However, gms does not heed the port variable, and continues to use :9200
(same log as above). Are there some env vars I can set on GMS to make it not append a port?hallowed-london-5429
07/27/2021, 9:54 AMcool-iron-6335
08/04/2021, 9:53 AMsquare-activity-64562
08/04/2021, 10:36 AMwith .Values.labels
https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-frontend/templates/deployment.yaml#L6
I am not 100% sure so wanted to ask. Similar includes are present in most of the files. And I don't think any of them are working. Neither labels
nor selectorLabels
are being applied to deploymentsrapid-sundown-8805
08/18/2021, 11:28 AMorg.springframework.beans.factory.BeanCreationException: Error creating bean with name 'configEntityRegistry' defined in com.linkedin.gms.factory.entityregistry.ConfigEntityRegistryFactory: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.linkedin.metadata.models.registry.ConfigEntityRegistry]: Factory method 'getInstance' threw exception; nested exception is java.io.FileNotFoundException: ../../metadata-models/src/main/resources/entity-registry.yml (No such file or directory)
I'm wondering if any of you have faced the same? Seems there is some yaml file that it is missing (or cannot find at that path).blue-holiday-20644
08/26/2021, 1:03 PMhandsome-football-66174
09/02/2021, 5:38 PMcool-iron-6335
09/06/2021, 8:49 AMmillions-jelly-76272
09/13/2021, 5:33 AMkubectl get pods -n datahub
NAME READY STATUS RESTARTS AGE
elasticsearch-master-0 0/1 Pending 0 16m
elasticsearch-master-1 0/1 Pending 0 16m
elasticsearch-master-2 0/1 Pending 0 16m
prerequisites-cp-schema-registry-cf79bfccf-zxp2d 1/2 Error 7 16m
prerequisites-kafka-0 0/1 Pending 0 16m
prerequisites-zookeeper-0 0/1 Pending 0 16m
Error coming from cp-schema-registry pod:
kubectl logs -n datahub -f pod/prerequisites-cp-schema-registry-cf79bfccf-zxp2d
error: a container name must be specified for pod prerequisites-cp-schema-registry-cf79bfccf-zxp2d, choose one of: [prometheus-jmx-exporter cp-schema-registry-server]
Has anyone seen this before?handsome-football-66174
09/13/2021, 7:35 PMbetter-orange-49102
09/14/2021, 9:13 AMprehistoric-grass-6413
09/14/2021, 3:37 PMlittle-address-54150
09/15/2021, 2:25 PMcareful-artist-3840
09/16/2021, 2:02 AMhandsome-football-66174
09/21/2021, 2:44 PMclean-furniture-99495
09/22/2021, 8:35 AMhandsome-football-66174
09/22/2021, 5:24 PMbrief-insurance-68141
09/23/2021, 12:18 AMchilly-barista-6524
09/27/2021, 11:45 AMdatahub-gms
and datahubUpgrade
job.
datahub-gms
throws following error:
javax.persistence.PersistenceException: Query threw SQLException:Table 'datahub.metadata_aspect_v2' doesn't exist
and
datahubUpgrade
throws following error:
ERROR: Cannot connect to GMSat host test-datahub-datahub-gms port 8080. Make sure GMS is on the latest version and is running at that host before starting the migration.
Now both the errors seems to be dependent on each other to me. I was wondering if we are missing any step in between and does the metadata_aspect_v2
table needs to be created manually?some-cricket-23089
09/30/2021, 9:23 AMsudo docker build -t updated_datahub_frontend_react -f ./docker/datahub-frontend
But this end with the error below
---> Running in 55db64e7685d
/bin/sh: ./gradlew: not found
The command '/bin/sh -c cd datahub-src && ./gradlew :datahub-frontend:dist -PenableEmber=${ENABLE_EMBER} -x test -x yarnTest -x yarnLint && cp datahub-frontend/build/distributions/datahub-frontend.zip ../datahub-frontend.zip && cd .. && rm -rf datahub-src && unzip datahub-frontend.zip' returned a non-zero code: 127
Could anyone please help me resolve this issue.some-cricket-23089
10/01/2021, 1:15 PMsome-cricket-23089
10/04/2021, 5:43 AM