https://datahubproject.io logo
Join Slack
Powered by
Hi all, am trying follow Quickstart doc. setting u...
# getting-started
l
Hi all, am trying follow Quickstart doc. setting up on my Mac local system. but never successful. I even updated the docker-compose with 
platform: linux/amd64
for all services, still the mysql-setup and elastic search-setup keeps exiting and setup never completes. anyone face similar issues?
s
Please share the output of
Copy code
python -c "import platform; print(platform.platform())"
l
macOS-10.16-x86_64-i386-64bit. Thank you for responding
s
Gentle reminder to please post large blocks of code/stack trace in Slack message threads - it’s a HUGE help for the Core Team to keep track of which questions are still unaddressed across our various support channels!
Please use a single thread for a single problem and remove the other messages.
Please share the logs of 
elasticsearch
docker container
l
error: Unable to run quickstart - the following issues were detected: - kafka-setup is still running - datahub-gms is still starting - elasticsearch-setup is still running - elasticsearch is running but not healthy datahub-gms | 2022/07/25 151937 Problem with request: Get "http://elasticsearch:9200": dial tcp 172.25.0.29200 connect: connection refused. Sleeping 1s datahub-frontend-react | Caused by: org.apache.http.conn.HttpHostConnectException: Connect to datahub-gms:8080 [datahub-gms/172.25.0.8] failed: Connection refused (Connection refused) datahub-frontend-react | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:156)
can you please share command for elasticsearh log @square-activity-64562
s
Copy code
docker logs elasticsearch
Can you share
Copy code
docker logs elasticsearch-setup
l
Copy code
2022/07/25 15:21:12 Problem with request: Get <http://elasticsearch:9200>: dial tcp 172.25.0.2:9200: connect: connection refused. Sleeping 1s
2022/07/25 15:21:13 Problem with request: Get <http://elasticsearch:9200>: dial tcp 172.25.0.2:9200: connect: connection refused. Sleeping 1s
elastic search-setup
s
You can run and share the whole log as a single file
Copy code
docker logs elasticsearch > elasticsearch.log
Can you please try without the custom quickstart file and share with us output of
Copy code
docker ps -a
and any other errors you might see.
l
ok will do now
I get errors while image pulling like this , without custom compose
Copy code
⠹ broker Pulling                                                                                           2.2s
  ⠴ f523e7d4eadc Waiting                                                                                       0.5s
  ⠴ b8671a2e7f21 Waiting                                                                                       0.5s
  ⠴ 0c6643d34eb4 Waiting                                                                                       0.5s
no matching manifest for linux/arm64/v8 in the manifest list entries
Error while pulling images. Going to attempt to move on to docker-compose up assuming the images have been built locally
the setup doesn't complete
Copy code
no matching manifest for linux/arm64/v8 in the manifest list entries
s
Please wait and send all the logs. There must be something showing what all images have problem. Also isn't 
macOS-10.16
outdated? Not sure how many people are using this
l
I can upgrade and try again
setup exited with
Copy code
no matching manifest for linux/arm64/v8 in the manifest list entries
..............
Unable to run quickstart - the following issues were detected:
- quickstart.sh or dev.sh is not running
meanwhile can you please let me know if we can use docker images for production? or if there is a way to deploy on kubernetes without helm or cli access
s
the container images are used in production. we do not recommend using quickstart as production. People who are familiar with kubernetes AND datahub might be able to deploy it to kubernetes without helm but we don't recommend it as it will take a lot of effort and time.
l
What can i find the container images? Any guide on how to use them?
s
container images = docker images
l
But here it refers to helm charts correct? What if i want to use images and deploy on the cluster without kubernetes
s
What is this cluster that you refer to? EC2 machines? on-premise machine?
Please note due to kubernetes being portable that is what we recommend. You should be able to run quickstart to run DataHub if you want. But we don't recommend that for production. You can see how all images are being created as the code is open-source https://github.com/datahub-project/datahub/tree/master/docker and ask your engineering team to modify it per your requirements
l
Aws cluster. May be ec2, emr
Yes i plan not to use quickstart for prod. Just doing to test on local. I intend to use images, and design a solution to deply as a service
Is there any video i can refer to regarding installing via images?
If i were to use the docker images is there any order i need to follow?
s
Please ask one of your engineers to look at the docker-compose file. That has all required details
l
Yes i did check, but i guess they were for single host. Also even for prod do we need to follow python install of acryl-datahub
s
There is no guide for multi-host using docker compose. We do not recommend it. It will be highly dependent on your environment.
b
This is correct ^ The operational complexity of splitting the docker compose to a multi node environment typically is not worth the cost. For that reason, we'd strongly recommend using K8s and helm directly!
l
Thank you John. Currently i cant use helm in my setup. I can use k8 but by depolying images on it. So was wondering the options
b
K8s without helm is basically writing your own deployment yamls. Can use Kompose to get started I guess
4 Views
Open in Slack
PreviousNext
Powered by