Hi, it looks like GMS needs kafka, elasticsearch, neo4j and mysql whereas Amundsen requires only elasticsearch and neo4j. Is kafka really required for Datahub or can it function without kafka service on the backend?

Thanks @better-orange-49102. Does it mean kafka and other dependency can run on containers rather than using managed services like MSK?

you can tap on your own existing kafka, mysql, ES instances instead of using spinning up new instances, or use the datahub helm chart

We don’t use kafka in our project right now. So kafka needs to be either spun up entirely new. I am not familiar with helm chart. Does it allow running kafka inside a pod in k8s like other services?

yes

Thanks, that helps

One small thing

neo4j

is optional. Most deployments don't use

neo4j

You can see this guide for deployment on kubernetes which contains step by step guide using helm https://datahubproject.io/docs/deploy/kubernetes