Hey community 👋 We are using KDA to consume topic from MSK, and KDA and MSK are deployed in different AWS accounts, KDA is in account A, while MSK in B. There is vpc and vpc peering set up between these 2 accounts. And corresponding security group is also configured from both sides, aka, MSK allows ingress connection from the security group where KDA is associated. KDA is set up to access the resources in VPC with security group, and private subnets. We could connect to MSK in account B using the private subnet from an instance in account A. But when launching KDA, we get the error:

Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=describeTopics, deadlineMs=1685455751878, tries=1, nextAllowedTryMs=1685455751979) timed out at 1685455751879 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: describeTopics

From this post, it seems this is due to misconfiguration between client application and msk cluster. But seems we have no connectivity issue, and we allow all traffic for security group. Anyone encounters similar issue, and can provide any hints on where could go wrong? Thanks very much! 🙇

I do think that this question is better suited for the AWS support channels then it is for Apache Flink itself

This article may be helpful, @Kaiqi Dong https://docs.aws.amazon.com/kinesisanalytics/latest/java/example-msk.html

alright, will do, thanks!