Hello everyone, I want to start using DataHub in o...
# getting-started
f
Hello everyone, I want to start using DataHub in our company, I need to come up with some answers before I start it out, can you help me? 1. I want the bare minimum instance of DataHub to start showing its value and grow as needed, does the helm setup (on EKS) considered the bare minimum? is there an EC2 setup/guide? (AWS) 2. With that said, is there any predictability on the resource costs? (if EKS, how much is needed, if EC2, how much is needed? - obviously, id depends on the answer to question 1) 3. Does all of the requirements for DataHub on EKS/EC2 containerized? if I start the cluster/EC2 instance and then delete it - nothing changed in my AWS subscriptions? none of the services is managed? Thank you!
i
1.) Yes, defaults in our helm charts should be enough. Alternatively you could simply have a single EC2 instance with 4 CPU and 16GM of RAM with docker and docker-compose installed and run
datahub docker quickstart
from there. This should not be considered a prod environment but it will suffice for experimental purposes initially. 2.) 1 EC2 instance (4 cores, 16GB) if deploying in K8s then 3 node cluster (2 core, 8GB) should be enough
3.) I don’t understand
You can run the stateful dependencies of DataHub (ElasticSearch, Kafka & a SQL database) as managed AWS resources. And that is what we recommend for production scenarios or if managing state is something you are not accustomed to.
f
@incalculable-ocean-74010 Thank you so much for your help, yeah if and when we will run the datahub on K8S we will need state managing.
Why is it recommended to work with K8S for DataHub? Why isn’t an EC2 instance enough?
i
Because DataHub is a multi-component, distributed system
And k8s is the common denominator that the team found for cloud agnostic deployments that had orchestration capabilities