https://datahubproject.io logo
Join Slack
Powered by
hi all, I'm trying to get an instance running via ...
# troubleshoot
b
hi all, I'm trying to get an instance running via linux on a Chromebook, I'm getting the following error:
Copy code
(base) scottlam@penguin:~$ datahub docker quickstart
Unable to run quickstart:
- Docker doesn't seem to be running. Did you start it?
(base) scottlam@penguin:~$
l
Hi @broad-thailand-41358! Quickstart requires Docker & Python - do you both installed?
b
I do!
Copy code
(base) scottlam@penguin:~$ sudo docker -v
Docker version 20.10.6, build 370c289
(base) scottlam@penguin:~$ 

/home/scottlam/anaconda3/bin/python
(base) scottlam@penguin:~$ python
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
I also installed docker-compose and jq
l
Gotcha - ok, let me loop in the team!
i
Hello Scott, Is your docker only accessible to the sudo group? Could you run the 
datahub docker quickstart
command with sudo?
plus1 1
b
thanks @incalculable-ocean-74010, I was able to change add permissions to my user so I can run it without sudo now... that seemed to fix the issue. However, now I get this. I have a socks proxy on 8080:
Copy code
Error starting userland proxy: listen tcp4 0.0.0.0:8080: bind: address already in use
i
datahub docker quickstart
launches a docker-compose instance based on your machine’s characteristics. You can see here which one: https://github.com/acryldata/datahub-fork/blob/90b7b9ddddcb5f3778089692c62c3eb52af195c8/metadata-ingestion/src/datahub/cli/docker.py#L202, or alternatively check your terminal logs when you run that command. For instance on my machine it says:
Copy code
(datahub-env) ➜  ~ datahub docker quickstart
Detected M1 machine
Datahub Neo4j volume found, starting with neo4j as graph service.
If you want to run using elastic, run `datahub docker nuke` and re-ingest your data.

Running with neo4j on M1 is not currently supported, will be using elasticsearch as graph
Fetching docker-compose file <https://raw.githubusercontent.com/linkedin/datahub/master/docker/quickstart/docker-compose-without-neo4j-m1.quickstart.yml> from GitHub
Meaning that the cli tool runs this file where port 8080 is for DataHub’s backend called GMS. You can copy the file your cli tool refers to your local machine, modify the 8080 port to something else and then run: 
datahub docker --quickstart-compose-file <path to your local docker-compose file>
b
Thank you! I have it up and running now, but when I try and login with u/pw datahub/datahub, I get this error: 
Failed to log in! SyntaxError: Unexpected token < in JSON at position 0
datahub docker check says No issues detected
i
Can you run 
datahub docker check
to confirm whether all services are up and running?
b
yeah, I did that and it says no issues detected
any thoughts @incalculable-ocean-74010?
i
Nothing out of the box, what version of the cli are you using?
g
sounds like your frontend is not able to connect to your GMS container
Can you share the logs of datahub-frontend-react
and datahub-gms?
b
DataHub CLI version: 0.8.26.3
g
looks like your gms instance is having trouble starting up
Hmm- those logs make it seem like gms is in the process of starting up
are the logs continuing to print things? or have they quieted down now?
b
@green-football-43791 They quiet down... here are the last few lines:
Copy code
17:28:33 [qtp544724190-13] INFO  c.l.m.r.entity.EntityResource - GET LIST RESULTS for dataHubIngestionSource with filter {or=[{and=[]}]}
17:28:33 [qtp544724190-13] INFO  c.l.parseq.TaskDescriptorFactory - No provider found for TaskDescriptor, falling back to DefaultTaskDescriptor
17:28:33 [pool-9-thread-1] INFO  c.l.m.filter.RestliLoggingFilter - POST /entities?action=list - list - 200 - 288ms
17:28:33 [pool-9-thread-1] INFO  c.l.m.filter.RestliLoggingFilter - GET /entitiesV2?aspects=List(dataHubIngestionSourceInfo)&ids=List() - batchGet - 200 - 21ms
17:28:33 [pool-8-thread-1] INFO  c.d.m.ingestion.IngestionScheduler - Successfully fetched 0 ingestion sources.
@green-football-43791 @incalculable-ocean-74010 sorry to bug, but any thoughts / ideas?
i
Do you have any WARN or ERROR logs in GMS?
b
Copy code
17:39:51 [main] WARN  o.a.k.c.producer.ProducerConfig - The configuration 'schema.registry.security.protocol' was supplied but isn't a known config.
17:39:51 [main] WARN  o.a.k.c.producer.ProducerConfig - The configuration 'schema.registry.ssl.keystore.location' was supplied but isn't a known config.
17:39:51 [main] WARN  o.a.k.c.producer.ProducerConfig - The configuration 'schema.registry.ssl.keystore.password' was supplied but isn't a known config.
17:39:51 [main] WARN  o.a.k.c.producer.ProducerConfig - The configuration 'schema.registry.ssl.truststore.password' was supplied but isn't a known config.
17:39:51 [main] WARN  o.a.k.c.producer.ProducerConfig - The configuration 'schema.registry.ssl.truststore.location' was supplied but isn't a known config.
Copy code
17:41:46 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No scheduled executor is provided to HttpClientFactory, using it's own scheduled executor.
17:41:46 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No callback executor is provided to HttpClientFactory, using it's own call back executor.
17:41:46 [main] WARN  c.l.r.t.h.client.HttpClientFactory - No Compression executor is provided to HttpClientFactory, using it's own compression executor.
Copy code
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.security.protocol' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.keystore.password' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.truststore.password' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.truststore.location' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.keystore.location' was supplied but isn't a known config.
Copy code
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.security.protocol' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.keystore.password' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.truststore.password' was supplied but isn't a known config.
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.truststore.location' was supplied but isn't a known config.
Copy code
17:41:48 [main] WARN  o.a.k.c.consumer.ConsumerConfig - The configuration 'schema.registry.ssl.keystore.location' was supplied but isn't a known config.
No ERROR though
e
Sorry about the late reply. Picking up from here
This is veryw eird. Above error is usually when gms is not running.
Can you post result of the following? so it includes the health status as well as ports, etc.
Copy code
docker container ls
also once gms is in a stable state, do you see any other msgs on either gms or frontend?>
b
Based on these logs, GMS looks good. I'm surprised that the error about JSON < is still coming when you navigate to the browser. It could be a docker network / dns resolution error. We can try to triage further once we have the output of that command^^
b
thank you @early-lamp-41924 / @big-carpet-38439 ! Here's the output:
Copy code
CONTAINER ID   IMAGE                                               COMMAND                  CREATED              STATUS                        PORTS                                                                                      NAMES
3a1dcaed7b2a   public.ecr.aws/datahub/acryl-datahub-actions:head   "/bin/sh -c 'dockeri…"   About a minute ago   Up About a minute                                                                                                        datahub_datahub-actions_1
cd63b9cd5eee   linkedin/datahub-frontend-react:head                "datahub-frontend/bi…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:9002->9002/tcp, :::9002->9002/tcp                                                  datahub-frontend-react
3c26bb606ad1   linkedin/datahub-gms:head                           "/bin/sh -c /datahub…"   About a minute ago   Up About a minute (healthy)   8080/tcp, 0.0.0.0:9090->9090/tcp, :::9090->9090/tcp                                        datahub-gms
555be60eb89f   mysql:5.7                                           "docker-entrypoint.s…"   9 hours ago          Up About a minute             0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp                                       mysql
c90ed022bb9b   confluentinc/cp-schema-registry:5.4.0               "/etc/confluent/dock…"   5 days ago           Up 4 minutes                  0.0.0.0:8081->8081/tcp, :::8081->8081/tcp                                                  schema-registry
bdce0f1a5795   confluentinc/cp-kafka:5.4.0                         "/etc/confluent/dock…"   5 days ago           Up 4 minutes                  0.0.0.0:9092->9092/tcp, :::9092->9092/tcp, 0.0.0.0:29092->29092/tcp, :::29092->29092/tcp   broker
67b719b990a4   elasticsearch:7.9.3                                 "/tini -- /usr/local…"   5 days ago           Up 4 minutes (healthy)        0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp                                        elasticsearch
8126295e8328   confluentinc/cp-zookeeper:5.4.0                     "/etc/confluent/dock…"   5 days ago           Up 4 minutes                  2888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp, 3888/tcp                              zookeeper
e
I see 0.0.0.0:9090->9090/tcp, :::9090->9090/tcp on datahub-gms
did you change docker compose to make it use 9090 instead of 8080?
b
sorry... I'm running a socks proxy for my company on 8080... so I downloaded a copy of the 
docker-compose-without-neo4j.quickstart.yml
per @incalculable-ocean-74010’s reply above... I changed all references to 8080 in the yml to 9090 and used the following command to start: 
datahub docker quickstart --quickstart-compose-file docker-compose-without-neo4j.quickstart.yml
I'm not sure if that was the correct way to do it, but I wanted to set it to something other than 8080
e
ah
so you shouldn’t change the internal container port
you just need to map it to another port
so it should be 9090:8080 on the gms port
plus1 1
so inside the datahub_network (docker network) it still uses 8080 port but externally you would query localhost:9090
b
ah thank you
I'm in! Thank you so much @early-lamp-41924!!!!
e
Awesome!!! Let us know if you run into any other issue!
teamwork 1
b
Woohoooo!!
thank you 1
c
d
18 Views
Open in Slack
PreviousNext
Powered by