icy-easter-2378
04/06/2021, 6:29 PMpi@raspberrypi:~/workspace/datahub $ ./docker/quickstart.sh
Pulling zookeeper ... done
Pulling neo4j ... pulling from library/neo4j
Pulling mysql ... pulling from library/mysql
Pulling broker ... done
Pulling schema-registry ... done
Pulling kafka-setup ... done
Pulling elasticsearch ... pulling from library/elasticsearch
Pulling kibana ... pulling from library/kibana
Pulling elasticsearch-setup ... done
Pulling datahub-gms ... done
Pulling datahub-mce-consumer ... done
Pulling datahub-frontend-react ... done
Pulling datahub-mae-consumer ... done
Pulling kafka-rest-proxy ... done
Pulling kafka-topics-ui ... done
Pulling schema-registry-ui ... done
ERROR: for elasticsearch no matching manifest for unknown in the manifest list entries
ERROR: for mysql no matching manifest for unknown in the manifest list entries
ERROR: for neo4j no matching manifest for unknown in the manifest list entries
ERROR: for kibana no matching manifest for unknown in the manifest list entries
ERROR: no matching manifest for unknown in the manifest list entries
green-football-43791
04/06/2021, 6:32 PMgreen-football-43791
04/06/2021, 6:32 PMgreen-football-43791
04/06/2021, 6:32 PMgreen-football-43791
04/06/2021, 6:33 PMicy-easter-2378
04/06/2021, 6:35 PMgreen-football-43791
04/06/2021, 6:36 PMgreen-football-43791
04/06/2021, 6:37 PMicy-easter-2378
04/06/2021, 6:38 PMgreen-football-43791
04/06/2021, 6:40 PMgreen-football-43791
04/06/2021, 6:40 PMbig-carpet-38439
04/06/2021, 6:41 PMdocker-compose.yml
big-carpet-38439
04/06/2021, 6:41 PMicy-easter-2378
04/06/2021, 6:41 PMicy-easter-2378
04/06/2021, 6:50 PMgreen-football-43791
04/06/2021, 7:01 PMgreen-football-43791
04/06/2021, 7:02 PMicy-easter-2378
04/06/2021, 7:04 PMmammoth-bear-12532
icy-easter-2378
04/06/2021, 7:07 PMicy-easter-2378
04/06/2021, 7:07 PMicy-easter-2378
04/06/2021, 7:07 PM(base) blars-book-pro:datahub bryanlarson$ ./docker/quickstart.sh
Traceback (most recent call last):
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/packages/six.py", line 734, in reraise
raise value.with_traceback(tb)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/python@3.9/3.9.2_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/transport/unixconn.py", line 43, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
return self.version(api_version=False)["ApiVersion"]
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/api/daemon.py", line 181, in version
return self._result(self._get(url), json=True)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/api/client.py", line 237, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/docker-compose", line 33, in <module>
sys.exit(load_entry_point('docker-compose==1.28.6', 'console_scripts', 'docker-compose')())
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/compose/cli/main.py", line 81, in main
command_func()
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/compose/cli/main.py", line 198, in perform_command
project = project_from_options('.', options)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/compose/cli/command.py", line 60, in project_from_options
return get_project(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/compose/cli/command.py", line 152, in get_project
client = get_client(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/compose/cli/docker_client.py", line 41, in get_client
client = docker_client(
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/compose/cli/docker_client.py", line 170, in docker_client
client = APIClient(use_ssh_client=not use_paramiko_ssh, **kwargs)
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/api/client.py", line 197, in __init__
self._version = self._retrieve_server_version()
File "/usr/local/Cellar/docker-compose/1.28.6/libexec/lib/python3.9/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
mammoth-bear-12532
icy-easter-2378
04/06/2021, 7:28 PMicy-easter-2378
04/06/2021, 7:29 PMicy-easter-2378
04/06/2021, 7:29 PMCaused by: com.linkedin.r2.RemoteInvocationException: com.linkedin.r2.RemoteInvocationException: Failed to get response from server for URI <http://datahub-gms:8080/charts>
datahub-frontend-react | at com.linkedin.restli.internal.client.ExceptionUtil.wrapThrowable(ExceptionUtil.java:135)
datahub-frontend-react | at com.linkedin.restli.internal.client.ResponseFutureImpl.getResponseImpl(ResponseFutureImpl.java:130)
datahub-frontend-react | at com.linkedin.restli.internal.client.ResponseFutureImpl.getResponse(ResponseFutureImpl.java:94)
datahub-frontend-react | at com.linkedin.chart.client.Charts.search(Charts.java:108)
datahub-frontend-react | at com.linkedin.datahub.graphql.types.chart.ChartType.search(ChartType.java:97)
datahub-frontend-react | at com.linkedin.datahub.graphql.resolvers.search.SearchResolver.lambda$get$1(SearchResolver.java:53)
datahub-frontend-react | ... 6 common frames omitted
icy-easter-2378
04/06/2021, 7:29 PMicy-easter-2378
04/06/2021, 7:33 PMicy-easter-2378
04/06/2021, 7:34 PMearly-lamp-41924
04/06/2021, 7:45 PMCOMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -p datahub -f docker-compose.yml -f docker-compose.override.yml up --build -d elasticsearch-setup
early-lamp-41924
04/06/2021, 7:45 PMicy-easter-2378
04/06/2021, 7:48 PMearly-lamp-41924
04/06/2021, 7:49 PMearly-lamp-41924
04/06/2021, 7:49 PMearly-lamp-41924
04/06/2021, 7:49 PMicy-easter-2378
04/06/2021, 7:50 PMicy-easter-2378
04/06/2021, 7:51 PMearly-lamp-41924
04/06/2021, 7:51 PMicy-easter-2378
04/06/2021, 7:53 PMicy-easter-2378
04/06/2021, 7:53 PMicy-easter-2378
04/06/2021, 7:53 PMjava.net.UnknownHostException: broker: Name or service not known
schema-registry | at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
schema-registry | at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
schema-registry | at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
schema-registry | at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
schema-registry | at java.net.InetAddress.getAllByName(InetAddress.java:1193)
schema-registry | at java.net.InetAddress.getAllByName(InetAddress.java:1127)
schema-registry | at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:104)
schema-registry | at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
schema-registry | at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
schema-registry | at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
schema-registry | at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:955)
schema-registry | at org.apache.kafka.clients.NetworkClient.access$600(NetworkClient.java:69)
schema-registry | at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1126)
schema-registry | at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:1017)
schema-registry | at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:538)
schema-registry | at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
schema-registry | at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
schema-registry | at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
schema-registry | at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:161)
schema-registry | at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:484)
schema-registry | at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1267)
schema-registry | at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
schema-registry | at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
schema-registry | at io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread.doWork(KafkaStoreReaderThread.java:152)
schema-registry | at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
early-lamp-41924
04/06/2021, 7:56 PMdocker container ls
?icy-easter-2378
04/06/2021, 7:56 PMicy-easter-2378
04/06/2021, 7:56 PMorg.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
datahub-mce-consumer | at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185)
datahub-mce-consumer | at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53)
datahub-mce-consumer | at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360)
datahub-mce-consumer | at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158)
datahub-mce-consumer | at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122)
datahub-mce-consumer | at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:894)
datahub-mce-consumer | at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.finishRefresh(ServletWebServerApplicationContext.java:163)
datahub-mce-consumer | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:553)
datahub-mce-consumer | at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142)
datahub-mce-consumer | at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775)
datahub-mce-consumer | at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
datahub-mce-consumer | at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
datahub-mce-consumer | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
datahub-mce-consumer | at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
datahub-mce-consumer | at com.linkedin.metadata.kafka.MceConsumerApplication.main(MceConsumerApplication.java:12)
datahub-mce-consumer | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
datahub-mce-consumer | at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
datahub-mce-consumer | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
datahub-mce-consumer | at java.lang.reflect.Method.invoke(Method.java:498)
datahub-mce-consumer | at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
datahub-mce-consumer | at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
datahub-mce-consumer | at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
datahub-mce-consumer | at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
datahub-mce-consumer | Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
datahub-mce-consumer | at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:825)
datahub-mce-consumer | at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:631)
datahub-mce-consumer | at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:164)
datahub-mce-consumer | at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumerWithAdjustedProperties(DefaultKafkaConsumerFactory.java:160)
datahub-mce-consumer | at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:135)
datahub-mce-consumer | at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumer(DefaultKafkaConsumerFactory.java:109)
datahub-mce-consumer | at org.springframework.kafka.listener.AbstractMessageListenerContainer.checkTopics(AbstractMessageListenerContainer.java:305)
datahub-mce-consumer | at org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerC
icy-easter-2378
04/06/2021, 7:57 PMearly-lamp-41924
04/06/2021, 7:57 PMearly-lamp-41924
04/06/2021, 7:57 PMicy-easter-2378
04/06/2021, 7:58 PMicy-easter-2378
04/06/2021, 8:01 PMicy-easter-2378
04/06/2021, 8:01 PMdatahub-mae-consumer | 2021/04/06 19:59:38 Problem with dial: dial tcp 172.19.0.8:29092: connect: connection refused. Sleeping 1s
datahub-mae-consumer | 2021/04/06 19:59:38 Problem with request: Get <http://neo4j:7474>: dial tcp 172.19.0.3:7474: connect: connection refused. Sleeping 1s
elasticsearch-setup | 2021/04/06 19:59:39 Problem with request: Get <http://elasticsearch:9200>: dial tcp 172.19.0.2:9200: getsockopt: connection refused. Sleeping 1s
datahub-gms | 2021/04/06 19:59:39 Problem with dial: dial tcp 172.19.0.8:29092: connect: connection refused. Sleeping 1s
datahub-mae-consumer | 2021/04/06 19:59:39 Problem with request: Get <http://elasticsearch:9200>: dial tcp 172.19.0.2:9200: connect: connection refused. Sleeping 1s
datahub-gms | 2021/04/06 19:59:39 Problem with request: Get <http://neo4j:7474>: dial tcp 172.19.0.3:7474: connect: connection refused. Sleeping 1s
datahub-gms | 2021/04/06 19:59:40 Problem with request: Get <http://elasticsearch:9200>: dial tcp 172.19.0.2:9200: connect: connection refused. Sleeping 1s
kafka-rest-proxy | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/172.19.0.8:29092) could not be established. Broker may not be available.
kafka-setup | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/172.19.0.8:29092) could not be established. Broker may not be available.
icy-easter-2378
04/06/2021, 8:02 PMearly-lamp-41924
04/06/2021, 8:03 PMicy-easter-2378
04/06/2021, 8:03 PMicy-easter-2378
04/06/2021, 8:03 PM20:02:51.724 [mce-consumer-job-client-0-C-1] WARN o.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Error connecting to node broker:29092 (id: 1 rack: null)
datahub-mce-consumer | java.net.UnknownHostException: broker
datahub-mce-consumer | at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
datahub-mce-consumer | at java.net.InetAddress.getAllByName(InetAddress.java:1193)
datahub-mce-consumer | at java.net.InetAddress.getAllByName(InetAddress.java:1127)
datahub-mce-consumer | at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
icy-easter-2378
04/06/2021, 8:04 PMmammoth-bear-12532
early-lamp-41924
04/06/2021, 11:44 PM