A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi everyone, I’m trying to get an idea of the minimum production hardware requirements (disk, memory, cpu, etc.) for the four core components of DataHub (<https://datahubproject.io/docs/gms|GMS>, <https://datahubproject.io/docs/metadata-jobs/mae-consumer-job|MAE Consumer>, <https://datahubproject.io/docs/metadata-jobs/mce-consumer-job|MCE Consumer>, and <https://datahubproject.io/docs/datahub-frontend|Frontend>).  Does anyone have any insights/recommendations (or maybe point me to the doc where that’s outlined)?

Generally we can run all of that comfortably with 2cpus, 4gb ram, and a couple GBs of disk, and you’d likely get away with less than that.

All four of those containers are largely stateless, which is why the disk requirements are fairly low. The CPU usage depends on the actual load, so it’s hard to really answer in a general sense, but 2 cpus should be able to easily handle tens/hundreds of thousands of requests per day

perfect! thanks for the answer <@U01GZEETMEZ>.