Hi datahub community, for a POC, we have Datahub installed in Docker on a Linux VM. Currently we get often "out-of-memory" problems without running regular data ingestion. Now I would be interested to know what your experiences are for running Datahub in a stable production environment? Is anyone also using Docker on a VM? Is there any experience for running datahub in Azure AKS?

Hello Susann, DataHub is recommended to be run in a Kubernetes environment. It is a multi-component project that is poorly suited to be run in production in a single machine. I have personally run DataHub in AKS in the past (over a year ago) without issue. Unfortunately DataHub is a bit scarse on Azure docs although some community members have done it. I believe @rapid-book-98432 is one such person. Is there any particular question you would like to ask?

Hello @incalculable-ocean-74010, thanks for your reply. We ask ourselves which environment would be the best for our production system. Therefore, I would be interested to know in which environment most customers run datahub and if datahub runs smoothly there. My company provides a big RedHat OpenShift cluster for most of our application. But as far as I understood, this is not a recommended environment. That's why we proof using a virtual server on premises or AKS.

Susann, Correct me if I’m wrong but doesn’t RedHat OpenShift include Kubernetes? If it does then DataHub should run there just fine. One thing to consider though is that DataHub does not like Centos7-based clusters. That is the one thing I can recall that might cause issues.

Pedro, OpenShift does not include Kubernetes. And nobody in the communitiy seems to use OpenShift.

I thought OpenShift was deployed , is that not the case? If not, it is in theory possible to deploy DataHub as a bare bones deployment. Unfortunately we do not have a guide on it but would welcome it! All DataHub components are stateless beyond the pre-requisite software it needs (Kafka, a relational DB & ElasticSearch), which should make deployments simpler.

Hello Pedro, sorry for the missunderstanding. OpenShift is based on kubernates, but it is not a kubernetes cluster. https://www.simplilearn.com/kubernetes-vs-openshift-article At the start of our POC we tried installing datahub on Openshift, but we didn't got it working. Yes, we have a managed Kafka, relational DB and ElasticSearch. But we do not have the experience to find out how to deploy every component as a "bare bone deployment". That's why I asked for experiences of other users/customers.

Hi @steep-pizza-13640 / @incalculable-ocean-74010 ! In fact, trying to deploy on Azure AKS. On deployment is finaly fine into one namespace, but when i try a second install on another namespace, Elastic containers are not starting 😕 + I wan to configure some ingress controler to route to one NS or another one. No log on the ES container , still "Pending" i don't get it