Getting this error in `datahub-gms`:

Error creating bean with name 'ebeanServer' defined in com.linkedin.gms.factory.entity.EbeanServerFactory

I am running MySQL and

mysql-setup

ran without issues so seems connection is fine from that operation. Using same credentials for

datahub-gms

. Anyone have ideas?

Hey @bright-motherboard-35257 when does this error show up?

Docker container. Compose file here:

version: '3'

networks:

default:

name: datahub_network

services:

datahub-actions:

depends_on:

- datahub-gms

environment:

- DATAHUB_GMS_HOST=datahub-gms

- DATAHUB_GMS_PORT=8080

- KAFKA_BOOTSTRAP_SERVER=172.30.231.2:9092

- SCHEMA_REGISTRY_URL=http://172.30.231.2:8081

- METADATA_AUDIT_EVENT_NAME=MetadataAuditEvent_v4

- METADATA_CHANGE_LOG_VERSIONED_TOPIC_NAME=MetadataChangeLog_Versioned_v1

- DATAHUB_SYSTEM_CLIENT_ID=__datahub_system

- DATAHUB_SYSTEM_CLIENT_SECRET=JohnSnowKnowsNothing

- KAFKA_PROPERTIES_SECURITY_PROTOCOL=PLAINTEXT

hostname: actions

image: acryldata/datahub-actions:${ACTIONS_VERSION:-head}

restart: on-failure:5

datahub-frontend-react:

container_name: datahub-frontend-react

depends_on:

- datahub-gms

environment:

- DATAHUB_GMS_HOST=datahub-gms

- DATAHUB_GMS_PORT=8080

- DATAHUB_SECRET=YouKnowNothing

- DATAHUB_APP_VERSION=1.0

- DATAHUB_PLAY_MEM_BUFFER_SIZE=10MB

- JAVA_OPTS=-Xms512m -Xmx512m -Dhttp.port=9002 -Dconfig.file=datahub-frontend/conf/application.conf

-Djava.security.auth.login.config=datahub-frontend/conf/jaas.conf -Dlogback.configurationFile=datahub-frontend/conf/logback.xml

-Dlogback.debug=false -Dpidfile.path=/dev/null

- KAFKA_BOOTSTRAP_SERVER=172.30.231.2:9092

- DATAHUB_TRACKING_TOPIC=DataHubUsageEvent_v1

- ELASTIC_CLIENT_HOST=172.30.231.2

- ELASTIC_CLIENT_PORT=9200

hostname: datahub-frontend-react

image: ${DATAHUB_FRONTEND_IMAGE linkedin/datahub frontend react}${DATAHUB_VERSION:-head}

ports:

- ${DATAHUB_MAPPED_FRONTEND_PORT 9002}9002

volumes:

- ${HOME}/.datahub/plugins:/etc/datahub/plugins

datahub-gms:

container_name: datahub-gms

environment:

- DATAHUB_SERVER_TYPE=${DATAHUB_SERVER_TYPE:-quickstart}

- DATAHUB_TELEMETRY_ENABLED=${DATAHUB_TELEMETRY_ENABLED:-true}

- DATASET_ENABLE_SCSI=false

- EBEAN_DATASOURCE_USERNAME=datahub

- EBEAN_DATASOURCE_PASSWORD=Datahub@2022

- EBEAN_DATASOURCE_HOST=172.30.231.2:3306

- EBEAN_DATASOURCE_URL=jdbc:mysql://172.30.231.2:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8&enabledTLSProtocols=TLSv1.2

- EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver

- KAFKA_BOOTSTRAP_SERVER=172.30.231.2:9092

- KAFKA_SCHEMAREGISTRY_URL=http://172.30.231.2:8081

- ELASTICSEARCH_HOST=172.30.231.2

- ELASTICSEARCH_PORT=9200

- NEO4J_HOST=http://neo4j:7474

- NEO4J_URI=bolt://neo4j

- NEO4J_USERNAME=neo4j

- NEO4J_PASSWORD=datahub

- JAVA_OPTS=-Xms1g -Xmx1g

- GRAPH_SERVICE_IMPL=neo4j

- ENTITY_REGISTRY_CONFIG_PATH=/datahub/datahub-gms/resources/entity-registry.yml

- ENTITY_SERVICE_ENABLE_RETENTION=true

- MAE_CONSUMER_ENABLED=true

- MCE_CONSUMER_ENABLED=true

- PE_CONSUMER_ENABLED=true

- UI_INGESTION_ENABLED=true

hostname: datahub-gms

image: ${DATAHUB_GMS_IMAGE linkedin/datahub gms}${DATAHUB_VERSION:-head}

ports:

- ${DATAHUB_MAPPED_GMS_PORT 8080}8080

volumes:

- ${HOME}/.datahub/plugins/:/etc/datahub/plugins

- ${HOME}/.datahub/plugins/auth/resources/:/etc/datahub/plugins/auth/resources

elasticsearch-setup:

container_name: elasticsearch-setup

depends_on:

- elasticsearch-rpa

environment:

- ELASTICSEARCH_HOST=172.30.231.2

- ELASTICSEARCH_PORT=9200

- ELASTICSEARCH_PROTOCOL=http

hostname: elasticsearch-setup

image: ${DATAHUB_ELASTIC_SETUP_IMAGE linkedin/datahub elasticsearch setup}${DATAHUB_VERSION:-head}

kafka-setup:

container_name: kafka-setup

depends_on:

- broker

- schema-registry

environment:

- KAFKA_ZOOKEEPER_CONNECT=172.30.231.2:2181

- KAFKA_BOOTSTRAP_SERVER=172.30.231.2:9092

hostname: kafka-setup

image: ${DATAHUB_KAFKA_SETUP_IMAGE linkedin/datahub kafka setup}${DATAHUB_VERSION:-head}

mysql-setup:

container_name: mysql-setup

environment:

- MYSQL_HOST=172.30.231.2

- MYSQL_PORT=3306

- MYSQL_USERNAME=datahub

- MYSQL_PASSWORD=Datahub@2022

- DATAHUB_DB_NAME=datahub

hostname: mysql-setup

image: acryldata/datahub-mysql-setup:${DATAHUB_VERSION:-head}

neo4j:

container_name: neo4j

environment:

- NEO4J_AUTH=neo4j/datahub

- NEO4J_dbms_default__database=graph.db

- NEO4J_dbms_allow__upgrade=true

hostname: neo4j

image: neo4j:4.4.9-community

ports:

- ${DATAHUB_MAPPED_NEO4J_HTTP_PORT 7474}7474

- ${DATAHUB_MAPPED_NEO4J_BOLT_PORT 7687}7687

volumes:

- neo4jdata:/data

volumes:

neo4jdata: null

so this is just on startup?

And do you see messages like the following in the gms start-up??

[main] INFO  o.a.k.c.consumer.ConsumerConfig:347 - ConsumerConfig values:

INFO  o.a.kafka.common.utils.AppInfoParser:117 - Kafka version:

If you don't see them it is probably cause some problem with Kafka, either that it is not able to connect properly or Kafka is down, etc.

@microscopic-mechanic-13766 I’ll check. When the

kafka-setup

container runs it connects and I see the message and topic activity it performs in Kafka. My Kafka instance is actively running for other applications (not just Datahub).

In my experience deploying Datahub, I always get that error when there is some problem with the configuration of the dependencies (usually Kafka as, if I am not mistaken, it is the service used to create such beans). But as you pointed out that you are able to see messages of the Datahub actions in Kafka, I don't think the connection would be a problem...

@microscopic-mechanic-13766 I believe you are correct regarding an issue connecting to Kafka. What I find odd is the docker compose file has another container on same

datahub_network

network (e.g.-

kafka-setup

) that connects to the same Kafka address and creates the topics successfully; but

datahub-gms

container cannot. 🤔

Yeah, that seems to be even worse. I think if you would want to connect to it by specifying the host name instead of the IP address you should use the following port instead of 9092:

broker:29092

, as it is the one specified in the property

KAFKA_ADVERTISED_LISTENERS

@microscopic-mechanic-13766 Elastic is fine. Setup completes successfully. Same with connection to MySQL. Where is the property

KAFKA_ADVERTISED_LISTENERS

to be set? Under the

datahub-gms

environment variables? I do not have that in my docker compose at all and wonder if this is the issue.

But is any index created?? I am afraid that it might not be able to create them

Yes, here are the log results showing the various setup containers success (last line of each)...

elasticsearch-setup

```2022/10/13 170604 Waiting for: http://172.30.231.2:9200

2022/10/13 170604 Received 200 from http://172.30.231.2:9200

going to use protocol: http

going to use default elastic headers

not using any prefix

datahub_analytics_enabled: true

>>> GET _ilm/policy/datahub_usage_event_policy response code is 200

>>> _ilm/policy/datahub_usage_event_policy already exists ✓

>>> GET _index_template/datahub_usage_event_index_template response code is 200

>>> _index_template/datahub_usage_event_index_template already exists ✓

2022/10/13 170604 Command finished successfully. ```

mysql-setup

```-- only add default records if metadata_aspect is empty

INSERT INTO metadata_aspect_v2

SELECT * FROM temp_metadata_aspect_v2

WHERE NOT EXISTS (SELECT * from metadata_aspect_v2);

DROP TABLE temp_metadata_aspect_v2;

-- create metadata index table

CREATE TABLE IF NOT EXISTS metadata_index (

id

BIGINT NOT NULL AUTO_INCREMENT,

urn

VARCHAR(200) NOT NULL,

aspect

VARCHAR(150) NOT NULL,

path

VARCHAR(150) NOT NULL,

longVal

BIGINT,

stringVal

VARCHAR(200),

doubleVal

DOUBLE,

CONSTRAINT id_pk PRIMARY KEY (id),

INDEX longIndex (

urn

,

aspect

,

path

,

longVal

),

INDEX stringIndex (

urn

,

aspect

,

path

,

stringVal

),

INDEX doubleIndex (

urn

,

aspect

,

path

,

doubleVal

)

);

2022/10/13 170608 Command finished successfully. ```

kafka-setup

```[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 6.1.4-ccs

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: c9124241a6ff43bc

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1665680782951

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

Completed updating config for topic _schemas. ```

That is quite strange, could you show the output of http//<elastic host>9200/_cat/indices?v

My understanding is all indices show yellow in Elastic if you are only using 1 server because the cluster has only one node so the primary shard is presumably allocated to that one node. It is yellow because a replica shard is unable to be allocated.

The indices that need to be created are the ones shown in this page if I am not mistaken. If it is not this I don't know what exactly is the problem, but would recommend to try to redeploy from scratch (trying to erase all the data created in elastic and postgres)

Hi @microscopic-mechanic-13766 I installed on an entirely different server with nothing else running on it to get it to work,