hello, I deployed datahub to AWS EKS using the official helm charts. I'm trying to set up datahub actions. How do I get the

bootstrap-server endpoint

to use in the actions yaml file. I used the values specified in my

values.yaml

file, but those couldn't be resolved. any help will be appreciated. Thank you!

Hoping to enlist my friends @incalculable-ocean-74010 or @brainy-tent-14503 to help out here. If you are not using Managed Kafka (confluent or MSK), then you'll need those DNS addresses. Can you confirm that Kafka bootstrap servers are actually spun up?? You should be able to check using

kubectl get pods -n <your-namespace>

The names used within EKS are not accessible outside of EKS. The names referenced in the yaml are k8 services, so I’d actually recommend looking for the service instead of the pods. It is basically the same command as above however

svc

instead of

pods

Assuming the service is present in the namespace and you want to debug externally it is possible to setup port forwards for each of those 3 services to localhost ports. The actions framework could then be configured to point to the localhost ports using

localhost

rather than the k8 service names. This is not really useful for more than testing and ultimately the actions framework process should run in the same namespace.

Thanks a lot @brainy-tent-14503. I actually hadn't deployed the services yet. I understand, from another thread, this (https://github.com/bitnami/charts/tree/master/bitnami/kafka#accessing-kafka-brokers-from-outside-the-cluster) is what I will need to deploy the LoadBalancer. I'm guessing I will need to update the pre-requisite values and update helm install. Am I reasoning about this correctly?

For EKS, it is more common to expose services via AWS load balancers, either alb or nlb. There is an AWS load balancer controller to be deployed (which is external to any of the datahub charts). Once that is in place you’d have to create a k8 ingress tied to it using annotations to configure that AWS load balancer. This ingress is also outside the scope of the datahub/bitnami helm charts. There are security considerations for exposing services outside of the cluster and you’ll have to reason about how the AWS load balancer is deployed (to which vpc subnets, etc).

This is very helpful! Thank you!