A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hello,
We want to test Datahub using a deployment of docker on ec2 in AWS , I could see the tested &amp; confirmed config: 2 CPUs, 8GB RAM, 2GB Swap area , is there a specific instance type that is recommended \ tested before ?
And also - can we use spot group or it is preferred to use a single on-demand ?

Hi, I think this might work. It worked for previous version of Datahub
CentOS machine (<https://aws.amazon.com/ec2/instance-types/t3/|t3.xlarge> instance, 4 CPUs, 16GB )

<@U030ZARM4JC> thanks ! we will try t3.xlarge on demand

<@U030ZARM4JC> I wanted to ask an additional question - if we are using the quick-start image to deploy datahub using docker do we need to install all dependencies as well ?
• Kafka
• Local DB (MySQL, Postgres, MariaDB)
• Search Index (Elasticsearch)
• Graph Index (Supports either Neo4j or Elasticsearch)
And is it recommended to keep using the quick-start deployment as our production environment ?  are we going to be able to ingest data from our company to the quick start deployment?

you don't need to manually install the dependencies as they are downloaded as separate docker containers. You can see <https://github.com/datahub-project/datahub/tree/master/docker/quickstart|here> what gets executed and downloaded/installed. Especially take a look at docker-compose file(s)

the recommended production deployment is via Kubernetes and not quickstart

and you will be able to ingest all of the sources if network allows you to access the sources and if you have sufficient privileges in the source systems (which I assume you do)

and please read this as well <https://github.com/datahub-project/datahub/tree/master/docker#docker-images>

note that you need a new 2.x version of docker compose in order for 0.8.45 version to be deployed using quickstart