@Rocky How do I modify the StarRocks helm chart for a star rocks cluster to be in shareddata mode and have an s3 backend utlizing different assumed IAM roles based on s3 location?

To deploy a StarRocks cluster in shared-data mode using the Helm chart (typically the

kube-starrocks

chart which uses the StarRocks Operator) and configure it for S3 with multiple IAM roles, you need to follow these steps: 1. Configure the Helm Chart for Shared-Data Mode In your

values.yaml

, you must enable

shared_data

mode and provide the primary S3 configuration. This configuration creates the default storage volume used by the cluster for its main data.

yaml
starrocksFeSpec:
  # Ensure the FE ServiceAccount is annotated for IRSA if not using hardcoded keys
  serviceAccount: "starrocks-fe-sa" 
  config: |
    run_mode = shared_data
    cloud_native_storage_type = S3
    aws_s3_path = my-primary-bucket/starrocks-data
    aws_s3_region = us-west-2
    aws_s3_endpoint = <https://s3.us-west-2.amazonaws.com>
    
    # Use Assumed Role for the primary storage
    aws_s3_use_instance_profile = true
    aws_s3_iam_role_arn = arn:aws:iam::123456789012:role/primary-starrocks-role
    
    # Automatically create the built-in storage volume on first start
    enable_load_volume_from_conf = true

starrocksCnSpec:
  # CN nodes (Compute Nodes) are required for shared-data mode
  serviceAccount: "starrocks-cn-sa"
  config: |
    run_mode = shared_data

2. Handle Multiple IAM Roles based on S3 Location StarRocks does not support mapping different IAM roles to different paths within a single static configuration file (

fe.conf

). Instead, it uses Storage Volumes (introduced in v3.1) to manage multiple S3 locations with distinct credentials. Since

CREATE STORAGE VOLUME

is a SQL command that modifies cluster metadata, it cannot be pre-configured in the Helm

values.yaml

static config. You must execute these commands after the cluster is running. Step 2a: Grant Permissions to the Cluster Ensure that the IAM role attached to your K8s nodes (or the ServiceAccount role used via IRSA) has the

sts:AssumeRole

permission for all the roles you intend to use. Step 2b: Create Additional Storage Volumes For every S3 location that requires a different IAM role, run a SQL command:

sql
-- For an additional location with a different role
CREATE STORAGE VOLUME specialized_volume
TYPE = S3
LOCATIONS = ("<s3://secondary-bucket/specific-path/>")
PROPERTIES (
    "aws.s3.region" = "us-west-2",
    "aws.s3.use_instance_profile" = "true",
    "aws.s3.iam_role_arn" = "arn:aws:iam::123456789012:role/secondary-location-role"
);

Step 2c: Use the Volume When creating a database or a table that should reside in that specific location, reference the volume:

sql
CREATE DATABASE sensitive_db 
STORAGE VOLUME specialized_volume;

3. Summary of Configuration Key Points *

run_mode = shared_data

: Must be set in both FE and CN configurations. CN nodes are mandatory in this mode. *

enable_load_volume_from_conf = true

: Allows the FE to use the

aws_s3_*

properties in the Helm config to bootstrap the cluster's root storage. * IRSA Recommendation: For Kubernetes deployments, it is recommended to use IAM Roles for Service Accounts (IRSA). Annotate your ServiceAccounts in the Helm chart, and set

aws_s3_use_instance_profile = true

. * Isolation: FE nodes and CN nodes both need direct access to the S3 buckets. The FE does not proxy data access for the CNs. Note: If you are using External Catalogs (e.g., Iceberg or Hive on S3), you can also specify different IAM roles directly in the

CREATE EXTERNAL CATALOG

statement using the

aws.s3.iam_role_arn

property. References • deployment/shared_data/s3.md