Hello everyone. Am trying to deploy datahub on aws kubernetes . I am new to kubernetes as well. Is there a way we can deploy datahub as a service? Rather than using kubectl to run helm?

Hi @quick-animal-47381! IIRC you have a similar setup running - can you provide some direction here?

Hi @lemon-engine-23512, we have a setup running in dev using AWS ECS, and will be making some tweaks as we go into production. Some key takeaways so far: 1. There are some transient setup containers, be sure to mark them as non-essential 2. Had to run this init script manually 3. Had to play around with the network mode to get the containers connecting to each other 4. Default zookeeper webserver port conflicts with GMS on port 8080 (might be relevant based on your network mode) 5. Passed credentials using AWS Secrets

@quick-animal-47381 and @little-megabyte-1074 thank you for this. :)I will go through the same and get back if i have any clarification??

Sorry was OOO. We pushed the required images to ECR, and setup the env variables, ports, container dependencies, in the ECS console. We plan to use Terraform as we move the deployment into production.

@quick-animal-47381 did you use https://github.com/datahub-project/datahub/blob/master/docker/docker-compose.yml for your dockers? And whwre is init.sql being used?

Hi @lemon-engine-23512, we used this one (I believe this depends on your setup), and because we used AWS OpenSearch, we didn’t need the elasticsearch container. The init.sql we ran manually on our AWS RDS cluster.

thank you @quick-animal-47381 did you face any issue with AWS managed services? like AWs open search, RDS, MSK

We didn’t use MSK, but no we didn’t face issues with OpenSearch or RDS