http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> can I add a spill disk to backend n...
# questions-and-troubleshooting
a
@Rocky can I add a spill disk to backend node via helm chart after initial deploy?
r
Yes, you can add a dedicated spill disk to your Backend (BE) nodes after the initial deployment by updating your Helm chart's 
values.yaml
and performing a 
helm upgrade
. To do this, you need to configure two main things: 1. Mount a new volume for the BE nodes. 2. Point the BE configuration to that new mount path. Steps to add a spill disk 1. Update 
values.yaml
 In your Helm configuration, you should use the 
storageVolumes
field within the 
starrocksBeSpec
to define a new volume and its mount path. Then, specify the 
spill_local_storage_dir
in the 
config
section.
Copy code
yaml
starrocks:
  starrocksBeSpec:
    # 1. Define the spill volume
    storageVolumes:
      - name: be-spill
        storageClassName: "your-storage-class" # e.g., gp3, standard, etc.
        storageSize: "100Gi"
        mountPath: "/opt/starrocks/be/spill"

    # 2. Update the BE configuration to use the mount path
    config: |
      spill_local_storage_dir=/opt/starrocks/be/spill
2. Apply the changes Run the Helm upgrade command to apply the new configuration:
Copy code
bash
helm upgrade <release-name> <chart-path> -f values.yaml
Note: This change will trigger a rolling restart of your BE pods as the StatefulSet specification is updated to include the new volume mount. 3. Enable Spilling via SQL After the BEs have restarted with the new disk, you must enable the spilling feature in StarRocks (it is 
false
by default):
Copy code
sql
-- Enable spilling globally
SET GLOBAL enable_spill = true;

-- Optional: Set the mode to 'auto' so it triggers based on memory pressure
SET GLOBAL spill_mode = 'auto';
Key Considerations * Performance: In production, it is highly recommended to use a separate disk for spilling to avoid I/O contention with the primary data storage. * Object Storage Alternative: If you are on StarRocks v3.3 or later, you can also consider spilling to object storage (S3, GCS, etc.) using the 
spill_storage_volume
variable. This allows you to scale spill capacity without adding local disks to your Kubernetes nodes. * Capacity: Ensure the 
storageSize
for the spill disk is sufficient for your largest memory-intensive queries (large joins or aggregations), as StarRocks will fail a query if the spill disk runs out of space. References • quick_start/helm.md • administration/management/BE_configuration.md • administration/management/resource_management/spill_to_disk.md • faq/Others.md
Yes, you can add a spill disk to the Backend (BE) nodes after the initial deployment using the Helm chart. To do this, you need to perform two main actions in your 
values.yaml
file: mount a new volume and configure the BE to use it for spilling. Since the 
volumeClaimTemplates
in a Kubernetes StatefulSet are immutable after creation, the recommended way to add a new spill disk via Helm without recreating the entire cluster is to use the 
extraVolumes
and 
extraVolumeMounts
fields. Steps to add a spill disk 1. Update your 
values.yaml
: Add the 
spill_local_storage_dir
configuration to the BE config and define the volume mount.
Copy code
yaml
    starrocks:
      starrocksBeSpec:
        # 1. Configure the BE to use the new path for spilling
        config: |
          spill_local_storage_dir=/opt/starrocks/be/spill

        # 2. Mount the volume to the pod
        extraVolumeMounts:
          • name: spill-disk
            mountPath: /opt/starrocks/be/spill

        # 3. Define the volume (e.g., using a PersistentVolumeClaim or hostPath)
        extraVolumes:
          • name: spill-disk
            persistentVolumeClaim:
              claimName: your-pre-created-pvc-name # Or use other volume types
Note: If you want each BE pod to have its own unique PVC, using 
extraVolumes
with a single claim name may not work for multiple replicas. In that case, you might need to use the Operator's 
storageVolumes
feature if your Operator version supports adding them post-deployment, though this often requires a manual update to the StatefulSet. 2. Apply the changes: Run the Helm upgrade command:
Copy code
bash
    helm upgrade <release_name> starrocks/kube-starrocks -f values.yaml
3. Enable Spilling in StarRocks: Once the BE nodes have restarted and are running with the new mount, you must enable the spilling feature via SQL:
Copy code
sql
    -- Enable spilling globally or for the session
    SET GLOBAL enable_spill = true;
Important Considerations: * Performance: It is strongly recommended to use a different physical disk for spilling than the one used for data storage (
storage_root_path
) to avoid I/O contention during memory-intensive queries. * Immutable Fields: If you attempt to add the disk via 
storageSpec.storageVolumes
(which uses 
volumeClaimTemplates
), Kubernetes will block the update because 
volumeClaimTemplates
cannot be modified after creation. Using 
extraVolumes
is the standard workaround. * Cleanup: StarRocks automatically clears spilled data after a query completes. If a BE crashes, it will clean up the 
spill
directory upon restart. References • quick_start/helm.md • administration/management/resource_management/spill_to_disk.md • administration/management/BE_configuration.md • administration/management/FE_configuration.md • faq/Others.md
Open in Slack
PreviousNext
Powered by