@Rocky generate a table for the configurations variable its default and production best pratice recommendation for all the fe , be and session variables for 4.0.starrocks decoupled shared data

For StarRocks 4.0 in a decoupled (shared-data) architecture, the configuration landscape emphasizes object storage integration, data caching (CN nodes), and FE-driven compaction. In this mode, Backend (BE) nodes are typically deployed as Compute Nodes (CN), which are stateless and focus on query execution and local caching. The following tables summarize the key configurations, their defaults in v4.0, and production recommendations for shared-data deployments. 1. FE (Frontend) Configurations These parameters are typically set in

fe.conf

. Configuration Variable | Default Value | Production Best Practice

run_mode

|

shared_nothing

| Set to

shared_data

. This is mandatory for decoupled mode.

cloud_native_storage_type

|

S3

| Set to

S3

,

AZBLOB

, or

HDFS

depending on your storage backend.

enable_file_bundling

|

true

| Keep

true

(v4.0 default). It bundles small metadata files into larger ones to reduce Object Storage API calls by up to 90%.

lake_compaction_max_tasks

|

-1

| Keep

-1

(Auto). FE will scale compaction tasks based on the number of CNs available.

lake_compaction_score_selector_min_score

|

10.0

| 10.0. Prevents FE from scheduling compaction for partitions with very few small files.

lake_compaction_history_size

|

12

| 100. Increase this to better track compaction history via

SHOW PROC '/compactions'

.

enable_load_volume_from_conf

|

false

|

true

if you want StarRocks to automatically create the built-in storage volume using

fe.conf

S3/HDFS properties.

brpc_connection_pool_size

|

16

| 64 or 128 for high-concurrency environments to reduce connection churn. 2. BE/CN (Compute Node) Configurations These parameters are set in

be.conf

(or

cn.conf

). In shared-data mode, the local disk is used primarily for Data Cache. Configuration Variable | Default Value | Production Best Practice

starlet_use_star_cache

|

true

|

true

. Essential for shared-data performance to avoid constant remote storage reads.

disk_high_level

|

90

| 90. (Renamed from

datacache_disk_high_level

in v4.0). Threshold for cache eviction.

disk_low_level

|

60

| 60. Threshold where automatic cache scaling-down stops.

datacache_mem_size

|

0

| 10% - 20% of RAM if you have sufficient memory. Enables an L1 memory cache for hot blocks.

compact_threads

|

4

| 4 to 8. Set higher if you have high ingestion rates and sufficient CPU.

max_cumulative_compaction_num_singleton_deltas

|

1000

| 100. Lowering this makes individual compaction tasks faster and less resource-intensive.

mem_limit

|

90%

| 90%. Ensure CN nodes have at least 64GB RAM in production.

write_buffer_size

|

100MB

| 256MB - 512MB for high-throughput ingestion to reduce the number of small files flushed to storage.

enable_poco_client_for_aws_sdk

|

false

|

false

(v4.0 default). Uses the native AWS SDK client for better stability. 3. Session Variables These are set via

SET GLOBAL

or

SET

and control query execution behavior. Variable Name | Default Value | Production Best Practice

enable_bucket_aware_execution_on_lake

|

true

|

true

(v4.0 default). Optimizes joins/aggs on data lake tables by using bucket info.

enable_rewrite_simple_agg_to_meta_scan

|

true

|

true

(v4.0 default). Accelerates simple

count(*)

or

max/min

by reading metadata instead of data files.

query_mem_limit

|

0

(No limit) | Set per user/session (e.g.,

32G

) to prevent a single query from crashing a CN node.

enable_query_queue_v2

|

false

|

true

. Recommended for concurrency control (note: becomes

true

by default in v4.1).

plan_mode

|

auto

|

auto

. Allow StarRocks to choose between local cache and distributed metadata plans for Iceberg/Lake queries.

enable_iceberg_column_statistics

|

false

|

true

if using Iceberg. Enables the CBO to use column-level stats for better plan generation.

what about special variables like enable_table_name_case_insensitive that should be set only before deployment @Rocky

In StarRocks 4.0, certain configurations are immutable once the cluster metadata is initialized. Setting these incorrectly at deployment often requires a full cluster re-installation to change. For a Decoupled (Shared-data) architecture, the following variables must be finalized in

fe.conf

(and `be.conf`/`cn.conf`) before the first start of the FE process. 1. Immutable FE Configurations (Deployment Only) These variables define the core identity and storage architecture of your cluster. Variable | Default | Importance | Why it must be set before deployment

enable_table_name_case_insensitive

|

false

| Critical (v4.0+) | Controls if names (catalog, db, table) are stored as lowercase. Cannot be modified after the first FE start. If changed later, FE will fail to start.

run_mode

|

shared_nothing

| Mandatory | Must be set to

shared_data

. Changing this later is not supported; StarRocks cannot "convert" between modes.

enable_load_volume_from_conf

|

false

| High | If

true

, StarRocks creates the

builtin_storage_volume

on first boot using credentials in

fe.conf

. If

false

, you must manually create a volume via SQL before loading data.

cloud_native_storage_type

|

S3

| High | Defines the primary storage backend (S3/AZBLOB/HDFS). While credentials can be updated, the storage type and root path (

aws_s3_path

) are effectively fixed for the lifetime of that storage volume.

meta_dir

|

/fe/meta

| High | The local path for FE metadata. While movable with care, it’s best practice to finalize this (ideally on high-performance SSD) before deployment.

cloud_native_meta_port

|

6090

| Medium | The RPC port for the internal cloud-native meta service. 2. Static BE/CN Configurations In decoupled mode, "BEs" are typically deployed as Compute Nodes (CN). Variable | Default | Recommendation | Reason

storage_root_path

|

/be/storage

|

/path/to/ssd/cache

| In shared-data mode, this is where the Data Cache is stored. Must be set before starting the CN. Changing this path requires clearing the cache and a node restart.

be_port

/

heartbeat_service_port

|

9060 / 9050

| Standardize | Once a node is added to the cluster via

ADD COMPUTE NODE

, changing its ports requires dropping and re-adding the node. 3. Session Variables (Global Initialization) While session variables can be changed at runtime, it is a production best practice to set these Globally immediately after the cluster is up to ensure consistent behavior across all users. Variable | Command to Run Post-Deployment | Purpose

sql_dialect

|

SET GLOBAL sql_dialect = 'starrocks';

| Defines the default SQL syntax. Note: if set to

trino

, identifiers become case-insensitive by default.

query_mem_limit

|

SET GLOBAL query_mem_limit = <bytes>;

| Prevents a single rogue query from consuming all CN memory. Recommended to set based on your smallest node's RAM.

enable_query_queue_v2

|

SET GLOBAL enable_query_queue_v2 = true;

| Enables advanced concurrency control (highly recommended for production v4.0). Summary Checklist for Shared-Data v4.0 Deployment: 1. Case Insensitivity: Decide if you need

enable_table_name_case_insensitive = true

. If you are migrating from systems that are case-insensitive (like some legacy warehouses), set this before starting FE. 2. Mode: Double-check

run_mode = shared_data

is in your config files. 3. Storage: Ensure your Object Storage path (

aws_s3_path

) is correct. If you change this later, existing data will become unreachable. 4. Hardware: Ensure CN nodes have sufficient local SSD space for

storage_root_path

to act as the cache layer. References • administration/management/FE_configuration.md • sql-reference/System_variable.md

@Rocky not the is mutable = no ones i am aksing about specially the variables that cannot be changed after deployment not the cluster fe restart changeable ones

In StarRocks 4.0 (Decoupled/Shared-data), configuration variables fall into two categories: those that can be tuned at runtime (mutable) and those that are "point-of-no-return" (immutable). For a production deployment, the variables in the table below are critical because they either cannot be changed after the cluster is initialized or changing them later will cause metadata corruption, data inaccessibility, or require a full cluster rebuild. 1. FE "Set-Once" Configurations (fe.conf) These parameters must be finalized before the first start of the FE. Changing them later typically requires re-initializing the cluster. Variable | Default | Production Best Practice | Reason for Immutability

run_mode

|

shared_nothing

|

shared_data

| StarRocks cannot convert between architectures. Changing this after deployment will prevent the FE from restarting.

enable_table_name_case_insensitive

|

false

| Decide before start | New in v4.0. If set to

true

, all object names (DB, Table, View) are stored in lowercase. If changed after metadata exists, existing objects become unreachable and FE will fail to start.

cloud_native_storage_type

|

S3

|

S3

,

AZBLOB

, or

HDFS

| Defines the storage backend protocol. You cannot switch from S3 to HDFS for an existing cluster without full data migration.

aws_s3_path

| (Empty) |

bucket/path

| This is the root path for your data. Changing this later makes all existing tables "orphaned" as their file paths are relative to this root.

meta_dir

|

./meta

|

/path/to/ssd/meta

| While it can be moved manually with a file copy, it is safer to define a high-performance SSD path from Day 1 to ensure metadata performance.

cloud_native_meta_port

|

6090

|

6090

| The internal RPC port for the meta service. Once nodes are communicating on this port, changing it requires a synchronized update across the cluster.

enable_load_volume_from_conf

|

false

|

true

| If

true

, StarRocks creates the

builtin_storage_volume

on first boot. If you change this later, it won't affect the existing volume, leading to confusion. ────────── 2. BE/CN Static Configurations (be.conf / cn.conf) In decoupled mode, BEs are Compute Nodes (CN). These are "static" in the sense that they require a process restart to change, but they don't break the cluster metadata if updated (unlike the FE variables above). Variable | Default | Production Best Practice | Note

storage_root_path

|

./storage

|

/path/to/ssd/cache

| In shared-data mode, this is the Data Cache path. Changing it requires a restart and will wipe the local cache (warm-up will be required).

be_port

|

9060

|

9060

| Once a CN is added via

ADD COMPUTE NODE

, changing its port requires dropping the node from the cluster and re-adding it.

priority_networks

| (Empty) |

10.x.x.x/24

| Critical for production. If a server has multiple IPs, set this to ensure the node registers the correct internal IP. Changing this later requires re-adding nodes. ────────── 3. Session / Global Variables Unlike the configuration files above, Session/Global variables are almost always mutable at runtime via

SET GLOBAL

. There are no "deployment-only" session variables that get locked forever. However, there are a few that behave as "effective defaults" for the cluster: Variable | Type | Production Best Practice | Note

sql_dialect

| Global |

starrocks

| Defines syntax behavior. Changing this globally later might break existing scheduled SQL tasks or dashboards that rely on a specific dialect (e.g., Trino).

query_mem_limit

| Global |

0.9 * Node_RAM

| Must be set globally immediately after deployment to protect CN nodes from OOM.

enable_query_queue_v2

| Global |

true

| Recommended to enable immediately for production concurrency management. Summary Checklist for v4.0 Shared-Data: