https://datahubproject.io logo
Join Slack
Powered by
Hi, I am trying to install datahub on my Mac M1 la...
# troubleshoot
m
Hi, I am trying to install datahub on my Mac M1 laptop but some of the containers fail to start:
Copy code
datahub version
DataHub CLI version: 0.9.4

datahub docker quickstart --arch m1

Getting the errors as below:
Unable to run quickstart - the following issues were detected:
- kafka-setup is still running
- datahub-gms is still starting
- zookeeper is not running
✅ 1
zookeeper container shows error as below:
Copy code
Command [/usr/local/bin/dub path /var/lib/zookeeper/data writable] FAILED !
When I added this:
Copy code
zookeeper:
h
Hey Deepti! can you try this? 1. Stop and Remove all datahub containers 
datahub docker nuke
2. Start datahub quickstart again. Pull all containers before starting. 
datahub docker quickstart --pull-images
Let me know if you continue to face the issue !
t
Hi, @millions-hydrogen-95879 keep trying couple of times , that's how it worked for me... So far 0.9.3 is the most successful one for me as well
b
There are known issues on M1 starting around v0.9.4, this should have been addressed in more recent builds of head. The nuke command suggested should remove the problem images and re-pull newer ones. Alternatively, you can target a specific version in the quickstart command 
datahub docker quickstart --version v0.9.3
m
Hi, I tried multiple times with all above suggestions but it's still having the same issue.
Copy code
Unable to run quickstart - the following issues were detected:
- kafka-setup is still running
- datahub-gms is running but not healthy
b
Please run the nuke command 
datahub docker nuke
and then please share the output of the following command. 
docker image ls | grep datahub
m
Copy code
acryldata/datahub-actions                                 head                                                                         342bb779dd1e   8 hours ago     5.9GB
linkedin/datahub-frontend-react                           v0.9.3                                                                       9afbb6126831   3 weeks ago     568MB
linkedin/datahub-gms                                      v0.9.3                                                                       d08dde4d8aad   3 weeks ago     654MB
linkedin/datahub-kafka-setup                              v0.9.3                                                                       606843410867   3 weeks ago     673MB
acryldata/datahub-mysql-setup                             v0.9.3                                                                       64d7b35323d8   3 weeks ago     55.2MB
linkedin/datahub-elasticsearch-setup                      v0.9.3                                                                       3aa51338a55d   3 weeks ago     23.1MB
This is what I see
Copy code
Unable to run quickstart - the following issues were detected:
- kafka-setup is still running
- datahub-gms is running but not healthy
b
That looks fine, using the 
--version v0.9.3
argument. Let’s take a look at the logs. Look at the running containers 
docker container ls
and find the container id for 
kafka-setup
, then run 
docker logs <container-id>
and lets see what might be happening from logs there. Certainly gms is expected to be unhealthy if the 
kafka-setup
process fails to create the required topics and configuration. Let’s consider the fact that running M1 is not the problem.
m
kafka-setup logs:
Copy code
[kafka-admin-client-thread | adminclient-1] INFO org.apache.kafka.clients.admin.internals.AdminMetadataManager - [AdminClient clientId=adminclient-1] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1672174422043, tries=1, nextAllowedTryMs=1672174422144) timed out at 1672174422044 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: fetchMetadata
[main] ERROR io.confluent.admin.utils.ClusterStatus - Error while getting broker list.
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1672174422108, tries=1, nextAllowedTryMs=1672174422250) timed out at 1672174422150 after 1 attempt(s)
	at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
	at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
	at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
	at io.confluent.admin.utils.ClusterStatus.isKafkaReady(ClusterStatus.java:149)
	at io.confluent.admin.utils.cli.KafkaReadyCommand.main(KafkaReadyCommand.java:150)
Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes, deadlineMs=1672174422108, tries=1, nextAllowedTryMs=1672174422250) timed out at 1672174422150 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: listNodes
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/172.25.0.8:29092) could not be established. Broker may not be available.
b
Ok, let’s take a look at the broker container’s logs. It might be a port conflict which is confusing docker compose, meaning some other service is listening on the same port(s) that the broker uses.
m
Thanks @brainy-tent-14503, below are the logs from the broker container: (1), MetadataChangeLog_Timeseries_v1-0 -> Vector(1), MetadataChangeProposal_v1-0 -> Vector(1), DataHubUsageEvent_v1-0 -> Vector(1), FailedMetadataChangeEvent_v4-0 -> Vector(1), MetadataAuditEvent_v4-0 -> Vector(1), FailedMetadataChangeProposal_v1-0 -> Vector(1), MetadataChangeLog_Versioned_v1-0 -> Vector(1))) (kafka.controller.KafkaController) [2022-12-29 151125,122] DEBUG [Controller id=1] Topics not in preferred replica for broker 1 Map() (kafka.controller.KafkaController) [2022-12-29 151125,124] TRACE [Controller id=1] Leader imbalance ratio for broker 1 is 0.0 (kafka.controller.KafkaController) [2022-12-29 151625,133] INFO [Controller id=1] Processing automatic preferred replica leader election (kafka.controller.KafkaController) [2022-12-29 151625,136] TRACE [Controller id=1] Checking need to trigger auto leader balancing (kafka.controller.KafkaController) [2022-12-29 151625,168] DEBUG [Controller id=1] Preferred replicas by broker Map(1 -> Map(PlatformEvent_v1-0 -> Vector(1), _schemas-0 -> Vector(1), MetadataChangeEvent_v4-0 -> Vector(1), MetadataChangeLog_Timeseries_v1-0 -> Vector(1), MetadataChangeProposal_v1-0 -> Vector(1), DataHubUsageEvent_v1-0 -> Vector(1), FailedMetadataChangeEvent_v4-0 -> Vector(1), MetadataAuditEvent_v4-0 -> Vector(1), FailedMetadataChangeProposal_v1-0 -> Vector(1), MetadataChangeLog_Versioned_v1-0 -> Vector(1))) (kafka.controller.KafkaController) [2022-12-29 151625,169] DEBUG [Controller id=1] Topics not in preferred replica for broker 1 Map() (kafka.controller.KafkaController) [2022-12-29 151625,190] TRACE [Controller id=1] Leader imbalance ratio for broker 1 is 0.0 (kafka.controller.KafkaController) [2022-12-29 151848,234] INFO [GroupMetadataManager brokerId=1] Removed 0 expired offsets in 5 milliseconds. (kafka.coordinator.group.GroupMetadataManager) [2022-12-29 152217,586] INFO [Controller id=1] Processing automatic preferred replica leader election (kafka.controller.KafkaController) [2022-12-29 152217,588] TRACE [Controller id=1] Checking need to trigger auto leader balancing (kafka.controller.KafkaController)
Copy code
[2022-12-29 06:37:22,688] TRACE [Controller id=1 epoch=2] Received response {error_code=0,partition_errors=[{topic_name=MetadataChangeLog_Versioned_v1,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=FailedMetadataChangeProposal_v1,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=FailedMetadataChangeEvent_v4,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=_schemas,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=MetadataChangeEvent_v4,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=PlatformEvent_v1,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=DataHubUsageEvent_v1,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=MetadataAuditEvent_v4,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=MetadataChangeProposal_v1,partition_index=0,error_code=11,_tagged_fields={}},{topic_name=MetadataChangeLog_Timeseries_v1,partition_index=0,error_code=11,_tagged_fields={}}],_tagged_fields={}} for request LEADER_AND_ISR with correlation id 1 sent to broker broker:29092 (id: 1 rack: null) (state.change.logger)
[2022-12-29 06:37:22,702] TRACE [Controller id=1 epoch=2] Received response {error_code=0,_tagged_fields={}} for request UPDATE_METADATA with correlation id 2 sent to broker broker:29092 (id: 1 rack: null) (state.change.logger)
i
@millions-hydrogen-95879 what docker version are you using?
m
Docker version 20.10.21
i
And compose? 
docker compose version
m
Docker Compose version v2.13.0
i
From docker, what is this value for you? 
Server: Docker Desktop 4.15.0 (93002)
m
Docker Desktop 4.15.0 (93002
i
Ok, I believe you are hitting upon https://github.com/datahub-project/datahub/issues/6872 A PR to fix this will come up shortly.
m
Sure Thanks Pedro
i
Note, if my suspicion is correct it means that you will not be able to launch v0.9.5 in quickstart. You would need a more up-to-date tag.
m
I will wait for the fix to be available
b
is this issue resolved now? i am trying to execute 
datahub docker quickstart --version v0.9.3
on mac m1 and still facing issues.
Copy code
Unable to run quickstart - the following issues were detected:
- kafka-setup is still running
- datahub-gms is still starting
- datahub-upgrade is still running
15 Views
Open in Slack
PreviousNext
Powered by