@Rocky Can I use KEDA for autoscaling my starrocks in eks using helm chart .? does the starrocks helm chart provide the KEDA autoscaling .?

Yes, you can use KEDA (Kubernetes Event-Driven Autoscaling) to autoscale StarRocks on Amazon EKS, but it is not natively built into the official StarRocks Helm chart templates yet. 1. Does the StarRocks Helm chart provide KEDA? No. The official StarRocks Helm chart (

starrocks/kube-starrocks

) currently provides an

autoScalingPolicy

field in the

values.yaml

that creates a standard Kubernetes Horizontal Pod Autoscaler (HPA). This HPA is limited to basic CPU and memory metrics. To use KEDA, you must: * Install KEDA in your EKS cluster separately. * Manually create a KEDA

ScaledObject

that targets your

StarRocksCluster

resource. 2. Can I use KEDA for autoscaling? Yes, and it is a recommended approach for advanced EKS deployments. A common pattern used by the StarRocks community and Amazon's own FinTech teams involves scaling stateless Compute Nodes (CN) using KEDA combined with Karpenter for just-in-time node provisioning. KEDA is particularly effective for StarRocks because: * Metric-Driven Scaling: You can scale based on StarRocks-specific metrics (e.g., query queue depth, pending queries) via Prometheus, rather than just raw CPU/RAM. * Stateless CNs: StarRocks CNs carry no data, allowing them to scale in and out instantly without data rebalancing. * Scale-to-Zero: KEDA can scale your compute tier to zero during idle periods to save costs on EKS. 3. Where can I find Helm configurations for KEDA? While the official chart doesn't include KEDA templates, the Data on EKS (DoEKS) project provides "production-ready" blueprints and Helm configurations specifically for running StarRocks with KEDA and Karpenter on AWS. Key Implementation Tips: * Target the CNs: Focus your KEDA

ScaledObject

on the

cn

component of your

StarRocksCluster

. * Use Prometheus Scaler: Most production setups use the KEDA Prometheus scaler to monitor StarRocks' internal metrics. * Disable Built-in HPA: If you use KEDA, ensure you disable the built-in

autoScalingPolicy

in your StarRocks Helm

values.yaml

to prevent scaling conflicts. For a reference implementation, you can check the Data on EKS GitHub repository, which recently added specific patterns for StarRocks KEDA autoscaling. References • deployment/helm.md • quick_start/helm.md • deployment/sr_operator.md

@Rocky can you point me to the helm chart that would enable the HPA for starrocks based on cpu or memory ?

The kube-starrocks Helm chart (which installs the StarRocks Operator and the cluster) allows you to enable HPA via the

autoScalingPolicy

field in your

values.yaml

. In StarRocks, autoscaling is primarily designed for Compute Nodes (CN) because they are stateless. While you can technically scale BEs, it is not recommended for HPA because BEs are stateful and require data rebalancing when scaling. Configuration for CPU/Memory HPA (CN only) To enable HPA for your CNs, you should add the

autoScalingPolicy

block under the

starrocksCnSpec

in your `values.yaml`:

yaml
starrocks:
  starrocksCnSpec:
    enabled: true
    # When using HPA, do not set a fixed replicas count
    # replicas: 3 
    
    # Define resource requests/limits (required for HPA)
    resources:
      requests:
        cpu: "4"
        memory: "16Gi"
      limits:
        cpu: "8"
        memory: "32Gi"

    autoScalingPolicy:
      minReplicas: 2
      maxReplicas: 10
      hpaPolicy:
        metrics:
          - type: Resource
            resource:
              name: cpu
              target:
                type: Utilization
                averageUtilization: 60 # Scale when CPU > 60%
          - type: Resource
            resource:
              name: memory
              target:
                type: Utilization
                averageUtilization: 80 # Scale when RAM > 80%

Important Details: 1. Remove `replicas`: If you enable

autoScalingPolicy

, you must remove or comment out the

replicas

field in the

starrocksCnSpec

. If both are present, the Operator and HPA may conflict. 2. Metrics Server: Ensure your EKS cluster has the Kubernetes Metrics Server installed, as standard HPA relies on it to read CPU/Memory usage. 3. Stateless Advantage: Because CNs do not store data, they can be added or removed by HPA instantly to handle spikes in query load without the overhead of moving data shards. Where to get the chart: You can find the official chart and its default

values.yaml

on GitHub: * Repo: StarRocks/starrocks-kubernetes-operator * ArtifactHub: kube-starrocks References • deployment/sr_operator.md • deployment/helm.md • quick_start/helm.md • quick_start/helm.md • deployment/sr_operator.md • deployment/helm.md • quick_start/helm.md References • deployment/sr_operator.md • quick_start/helm.md • deployment/helm.md