I am having trouble running GMS with error a NPE. ...
# getting-starteds
some-crayon-90964
09/25/2020, 8:36 PMI am having trouble running GMS with error a NPE. I have explored channels and found a solved case. It mentioned that its a SQL config issue, but I dont see anything wrong with my SQL config. I am running mysql with:
docker run --name mysql --hostname mysql -e "MYSQL_DATABASE=datahub" -e "MYSQL_USER=datahub" -e "MYSQL_PASSWORD=datahub" -e "MYSQL_ROOT_PASSWORD=datahub" -v mysql:/docker-entrypoint-initdb.d -v mysqldata:/var/lib/mysql -v mysql:/var/run/mysqld:rw --publish 3306:3306 --tmpfs /tmp:rw --tmpfs /run:rw --tmpfs /var/run:rw --network geotab_docker_bridge --read-only --security-opt=no-new-privileges -d <http://gcr.io/data-infrastructure-test-env/myql:5.7|gcr.io/data-infrastructure-test-env/myql:5.7> --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_cia
acceptable-architect-70237
09/25/2020, 8:52 PMI think gms is looking for the setup of KAFKA_BOOTSTRAP_SERVER which is an issue I raised today: https://github.com/linkedin/datahub/issues/1892
b
bumpy-keyboard-50565
09/26/2020, 12:34 PMThanks to @acceptable-architect-70237's PR we now have good default value for these. Please try again from HEAD to see if it's still throwing NPE.
s
some-crayon-90964
09/28/2020, 1:35 PMA lot thanks to @acceptable-architect-70237 who got to the bottom of this problem and figure out and @bumpy-keyboard-50565 who helped resolving this issue. I will try to run again with latest change. Just curious, is datahub looking for
KAFKA_BOOTSTRAP_SERVERb
bumpy-keyboard-50565
09/28/2020, 1:39 PMYes, and you can set them easily if you're using Docker or k8s.
s
some-crayon-90964
09/28/2020, 6:24 PMUnfortunately this fix doesnt work for me. I run the broker with env var
docker run -d --name broker --hostname broker --publish 29092:29092 --publish 9092:9092 -e "KAFKA_BROKER_ID=1" -e "KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181" -e "KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT" -e "KAFKA_ADVERTISED_LISTENERS=<PLAINTEXT://broker:29092>,PLAINTEXT_<HOST://localhost:9092>" -e "KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1" -e "KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=0" --tmpfs /tmp:rw --tmpfs /run:rw --tmpfs /var/run:rw --tmpfs /etc/kafka --network geotab_docker_bridge --read-only --security-opt=no-new-privileges <http://gcr.io/data-infrastructure-test-env/confluentinc/cp-kafka:5.4.0|gcr.io/data-infrastructure-test-env/confluentinc/cp-kafka:5.4.0>some-crayon-90964
09/28/2020, 6:25 PMI run the GMS with
docker run -d --name datahub-gms --hostname datahub-gms --publish 8080:8080 -e "EBEAN_DATASOURCE_USERNAME=datahub" -e "EBEAN_DATASOURCE_PASSWORD=datahub" -e "EBEAN_DATASOURCE_HOST=mysql:3306" -e "EBEAN_DATASOURCE_URL=jdbc:<mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8>" -e "EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver" -e "KAFKA_BOOTSTRAP_SERVER=broker:29092" -e "KAFKA_SCHEMAREGISTRY_URL=<http://schema-registry:8081>" -e "ELASTICSEARCH_HOST=elasticsearch" -e "ELASTICSEARCH_PORT=9200" -e "NEO4J_HOST=neo4j:7474" -e "NEO4J_URI=<bolt://neo4j>" -e "NEO4J_USERNAME=neo4j" -e "NEO4J_PASSWORD=datahub" --network geotab_docker_bridge --security-opt=no-new-privileges <http://gcr.io/data-infrastructure-test-env/linkedin/datahub-gms:latest|gcr.io/data-infrastructure-test-env/linkedin/datahub-gms:latest>some-crayon-90964
09/28/2020, 6:26 PMThe kafka bootstrap server should match, but it still gives the NPE
b
bumpy-keyboard-50565
09/28/2020, 6:28 PMDo you mind to create an issue on GH with details on how to reproduce it?
s
some-crayon-90964
09/29/2020, 1:54 PMI have added issue https://github.com/linkedin/datahub/issues/1902
some-crayon-90964
09/29/2020, 1:54 PMif there is any more details needed, please let me know
b
bumpy-keyboard-50565
09/29/2020, 2:01 PM👍
a
acceptable-architect-70237
09/30/2020, 2:39 PMSorry my suggestion didn't help you. I saw the *similar` NPE error and similar bean instantiation error so I thought it was the same cause.