Hello, Any Airflow users here? I've a question abo...
# general
a
Hello, Any Airflow users here? I've a question about stress/load testing airflow on EKS. I'd like to know what approaches one would take to determine what changes on airflow/DAGs will help one understand the load EKS Cluster can support? I use following backends for airflow on EKS. • RDS DB instance -
db.m5.large
• REDIS cluster -
cache.t2.medium
(3 nodes cluster) The airflow is setup in a namespace and has toleration to run on EKS node-pool of
m4.4xlarge
instances Please let me know if you need any more details.
i
Try the airflow slack community here: https://airflow.apache.org/community/
s
I think the biggest factor is going to be the DAGs your team runs. The core airflow (non-dag) pods haven't required much at all in my environments & seem to be super stable
👍 1