Rumit Jain
11/24/2025, 5:29 AMSHOW STORAGE VOLUMES; returns empty
• Creating a table fails with “Cluster has no available capacity”
• Storage volumes do NOT auto-create, even with:
enable_load_volume_from_conf=true
cloud_native_storage.enable=true
cloud_storage_type=S3
• Adding a "storage_volume" = "S3VOL" property still fails
If I deploy a BE node, everything works immediately — storage volume appears and table creation succeeds.
So my question is:
Does the current shared-data implementation still require BE nodes to exist even though they don’t store local data?
If yes, is FE+CN-only mode not yet supported in the Kubernetes operator, despite what the architecture docs describe?Kevin Cai
11/24/2025, 6:22 AMrun_mode for the FE must be shared_data and can't be changed after FE started, if the configuration is configured wrong, shall delete the setup and redo the fresh installation.Rumit Jain
11/24/2025, 6:24 AMmetrics:
serviceMonitor:
# Whether to expose metrics to Prometheus by ServiceMonitor.
# Note: make sure the prometheus operator is installed in your cluster.
# If prometheus is not installed by operator, you can add annotations on k8s service to expose metrics.
# See <https://github.com/StarRocks/starrocks-kubernetes-operator/blob/main/doc/integration/integration-prometheus-grafana.md#51-turn-on-the-prometheus-metrics-scrape-by-adding-annotations> for more details.
enabled: true
starrocksCluster:
# the namespace of starrockscluster cluster, if not set, the release namespace will be used.
namespace: "dockprivileged"
# specify the BE/CN deployment or not.
enabledBe: false
enabledCn: true
initPassword:
enabled: true
passwordSecret: starrocks-root-pass
starrocksFESpec:
service:
type: ClusterIP
ports:
- name: query
port: 9030
- name: http
port: 8030
# number of replicas to deploy for a FE statefulset.
replicas: 1
image:
# image sliced by "repository:tag"
repository: starrocks/fe-ubuntu
tag: "latest"
imagePullPolicy: IfNotPresent
resources:
requests:
cpu: 1
memory: 2Gi
# If you want to remove one resource limit, e.g., cpu, you can set it to cpu: "unlimited".
limits:
cpu: 1
memory: 2Gi
serviceAccountName: kubvir-starrocks-s3-sa
config: |
run_mode = shared_data
cloud_native_storage_type = S3
aws_s3_path = test-demo/starrocks/
aws_s3_region = us-east-1
aws_s3_endpoint=<https://s3.amazonaws.com>
enable_load_volume_from_conf=true
aws_s3_use_instance_profile=true
LOG_DIR = ${STARROCKS_HOME}/log
DATE = "$(date +%Y%m%d-%H%M%S)"
JAVA_OPTS="-Dlog4j2.formatMsgNoLookups=true -Xmx8192m -XX:+UseG1GC -Xlog:gc*:${LOG_DIR}/fe.gc.log.$DATE:time -XX:ErrorFile=${LOG_DIR}/hs_err_pid%p.log -Djava.security.policy=${STARROCKS_HOME}/conf/udf_security.policy"
http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
mysql_service_nio_enabled = true
sys_log_level = INFO
starrocksCnSpec:
# number of replicas to deploy for a BE statefulset.
replicas: 1
image:
# image sliced by "repository:tag"
repository: starrocks/cn-ubuntu
tag: "latest"
imagePullPolicy: IfNotPresent
resources:
requests:
cpu: 1
memory: 2Gi
# If you want to remove one resource limit, e.g., cpu, you can set it to cpu: "unlimited".
limits:
cpu: 1
memory: 2Gi
serviceAccountName: kubvir-starrocks-s3-sa
config: |
sys_log_level = INFO
# ports for admin, web, heartbeat service
thrift_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060Rumit Jain
11/24/2025, 6:26 AMmysql> CREATE TABLE IF NOT EXISTS crashdata ( CRASH_DATE DATETIME, BOROUGH STRING, ZIP_CODE STRING, LATITUDE INT, LONGITUDE INT, LOCATION STRING, ON_STREET_NAME STRING,
CROSS_STREET_NAME STRING, OFF_STREET_NAME STRING, CONTRIBUTING_FACTOR_VEHICLE_1 STRING, CONTRIBUTING_FACTOR_VEHICLE_2 STRING, COLLISION_ID INT, VEHICLE_TYPE_CODE_1 STRING, VEHICLE_TYPE_CODE_2 STRING );
ERROR 1064 (HY000): fail to create tablet: 10001: [Internal error: starlet err [RequestID=NAEK53WF6DP6WE96][StatusCode=403]Put object <s3://test-demo/starrocks/06d7b9e3-0247-4830-a813-8a455c4e1d7e/db10925/16432/16434/SCHEMA_0000000000004031> error: User: arn:aws:sts::111111111111:assumed-role/kubvir_dock_role/i-068674bf5c4ad1eb3 is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::test-demo/starrocks/06d7b9e3-0247-4830-a813-8a455c4e1d7e/db10925/16432/16434/SCHEMA_0000000000004031" because
It is taking Node IAM role and not the pod Service Account. Am i missing something?Kevin Cai
11/24/2025, 6:29 AMservice account running starrocks cluster, and set the storage volume to use sdk default behavior.Rumit Jain
11/24/2025, 6:31 AM