Hello Team
I have deployed Apache Pinot on Amazon EKS and configured Horizontal Pod Autoscaling (HPA) for the Pinot Broker using both CPU and memory metrics.
However, I am observing that the Broker pods always scale up even when CPU usage is low and no queries are being executed. The memory consumption keeps increasing slowly over time, which continuously triggers the memory-based HPA threshold.
Could you please confirm the recommended autoscaling strategy for Pinot Broker?
Should we scale only based on CPU, or should memory also be included?
Below is the HPA configuration I am currently using:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: hpa-pinot-broker
namespace: pinot-system # <-- Add the namespace here
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: pinot-broker
minReplicas: 2
maxReplicas: 8
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 75
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75