Hey, I am Data Engineer working at a startup: UNA Brands. We have been using Airbyte for last 8 months on EC2, and now we are doing the load testing on aws EKS. This is my first post on airbyte slack.
One of issue I am facing while testing out the loads on kubernetes aws EKS is that when running 25/30+ connections in parallel, the loads are not getting successful exit status after finishing the load(but I do see kubernetes pods getting completed for both source & destination) while webapp keep on saying it's running the loads. You can see in below screenshot and logs that connection ran for about 4hrs approx, but UI can't fetch the exit code sooner and says 9hrs are taken to complete
start_time: 2022-10-13 192934 UTC
end_time: 2022-10-13 234107 UTC
Please let me know if you want to know more about the configuration settings we are using, happy to share across