@Rocky what configs do we have for object store configuration on shared cluster in fe confs?

In a StarRocks shared-data cluster (v3.0+), the configuration for object storage in

fe.conf

consists of global toggles and provider-specific parameters. 1. Global Configurations These parameters enable shared-data mode and determine how StarRocks interacts with the storage backend. *

run_mode

: Must be set to

shared_data

. (Default is

shared_nothing

). *

cloud_native_storage_type

: Specifies the storage backend. Valid values:

S3

(default),

AZBLOB

,

ADLS2

,

GS

, or

HDFS

. *

enable_load_volume_from_conf

: (v3.1.0+) If

true

, StarRocks automatically creates a built-in storage volume using the configurations below and sets it as default. *

cloud_native_meta_port

: The RPC port for the meta service (default:

6090

). ────────── 2. Provider-Specific Configurations Depending on the

cloud_native_storage_type

you choose, you must provide the corresponding parameters. AWS S3 (and S3-compatible like MinIO) _Prefix: `aws_s3_`_ *

aws_s3_path

: The bucket and sub-path (e.g.,

my-bucket/data

). *

aws_s3_endpoint

: The S3 endpoint (e.g.,

<https://s3.us-west-2.amazonaws.com>

). *

aws_s3_region

: The AWS region (e.g.,

us-west-2

). * Authentication Options: * AK/SK:

aws_s3_access_key

and

aws_s3_secret_key

. * Instance Profile: Set

aws_s3_use_instance_profile = true

. * Assumed Role: Set

aws_s3_use_instance_profile = true

and provide

aws_s3_iam_role_arn

. * Default SDK:

aws_s3_use_aws_sdk_default_behavior = true

. Azure Blob Storage _Prefix: `azure_blob_`_ *

azure_blob_endpoint

: Storage account endpoint (e.g.,

https://<account>.<http://blob.core.windows.net|blob.core.windows.net>

). *

azure_blob_path

: The container and sub-path. * Authentication:

azure_blob_key

(Shared Key) or

azure_blob_sas_token

. *

azure_use_native_sdk

: (v3.4.4+) Set to

true

to use Managed Identities or Service Principals. Azure Data Lake Storage Gen2 (ADLS2) _Prefix: `azure_adls2_`_ *

azure_adls2_endpoint

: ADLS2 endpoint. *

azure_adls2_path

: The file system and sub-path. Google Cloud Storage (GCS) * Native Mode (v3.5.1+): Set

cloud_native_storage_type = GS

. * `gcp_gcs_path`: Bucket and sub-path. * `gcp_gcs_service_account_email`: Service account email for authentication. * S3-Compatible Mode: Set

cloud_native_storage_type = S3

and use

aws_s3_

prefixes with the GCS XML API endpoint (

<https://storage.googleapis.com>

). HDFS *

cloud_native_hdfs_url

: The HDFS path (e.g.,

hdfs://<nameservice>/starrocks/

). References • administration/management/FE_configuration.md • deployment/shared_data/s3.md • deployment/shared_data/gcs.md

@Rocky no for what builtin_storage_volume on object store is used fe/meta

In a StarRocks shared-data cluster, the

builtin_storage_volume

is the default storage location on your object store (S3, GCS, Azure, etc.) created via

fe.conf

. It is used to store the following components of your data: 1. Data and Metadata on Object Store Within the path defined by the storage volume, StarRocks organizes files into two main sub-directories (especially from v3.1.4 onwards): *

data/

: Stores the actual segment data files (the physical data of your tables). *

meta/

: Stores the tablet metadata files. This metadata tracks the versioning and mapping of segment files for each tablet. This allows CN nodes to know which data files to read without querying the FE for every single file list. 2. FE Local Metadata vs. Object Store It is important to distinguish between the two types of "meta" in a shared-data cluster: * Global Metadata (Local FE): The FE's core metadata (schema definitions, user permissions, etc.) is not stored in the

builtin_storage_volume

. It is still stored on the FE's local disk in the directory defined by

meta_dir

in

fe.conf

. * Tablet Metadata (Object Store): The specific metadata that links tablets to their physical segment files is stored in the

meta/

sub-directory of the

builtin_storage_volume

. 3. Relevant FE Configurations The FE manages the access to this object-store metadata via the Cloud-native Meta Service. *

cloud_native_meta_port

: The port (default

6090

) that FEs use to serve tablet metadata to CN (Compute) nodes. *

enable_load_volume_from_conf

: If set to

true

, the FE automatically creates this

builtin_storage_volume

using your S3/Azure/HDFS credentials and sets it as the default for all new tables. Summary: The

builtin_storage_volume

on the object store is used for table data (segments) and tablet-level metadata, while the FE's global cluster metadata remains on the FE's local disks. References • deployment/shared_data/s3.md • faq/shared_data_faq.md • administration/management/FE_configuration.md References • faq/shared_data_faq.md • deployment/shared_data/s3.md • quick_start/shared-data.md • deployment/shared_data/feature-support-shared-data.md