This message was deleted.

Yes, StarRocks supports tiered storage through two different architectural approaches, depending on whether you are using a Shared-nothing or Shared-data cluster. 1. Shared-data Architecture (Remote Storage + Local Cache) Introduced in StarRocks v3.0, this is the most common way to implement "tiered storage" for cost-efficiency. In this mode, storage and compute are disaggregated. * Cold Tier: All data is persisted in remote object storage (like AWS S3, Azure Blob Storage, Google Cloud Storage, or HDFS). * Hot Tier: Frequently accessed data is cached on the local disks of the Compute Nodes (CN). * Management: You can control how long data stays "hot" using table properties: * `datacache.enable`: Set to

true

to enable local caching. * `datacache.partition_duration`: Defines the TTL (Time to Live) for hot data (e.g.,

30 DAY

). Data older than this will not be proactively cached or will be evicted first. 2. Shared-nothing Architecture (SSD + HDD Tiering) If you are using the traditional shared-nothing architecture (BE-based), StarRocks supports migrating data between different types of local storage media (e.g., from SSD to HDD). * Hot Tier (SSD): Used for new partitions to ensure high IOPS. * Cold Tier (HDD): Used for older partitions to save costs. * Management: You can configure automatic "cooling" using partition-level properties: * `storage_medium`: Set to

SSD

initially. * `storage_cooldown_ttl`: Specifies a time interval (e.g.,

15 DAY

). After this time has passed since the partition's creation or its time range, the system automatically migrates the data from SSD to HDD. * `storage_cooldown_time`: Specifies an absolute date and time (e.g.,

2024-12-31 00:00:00

) for the data to migrate to HDD. Summary of Differences Feature | Shared-data (v3.0+) | Shared-nothing Tiering Type | Local Cache ↔️ Remote Object Storage | Local SSD ↔️ Local HDD Cost Efficiency | High (Object storage is very cheap) | Medium (Dependent on HDD cost) Scalability | High (Separate compute/storage scaling) | Lower (Data must be rebalanced) Configuration |

datacache.partition_duration

|

storage_cooldown_ttl

Recommendation: If you are looking for true cloud-native tiered storage with virtually unlimited capacity and low cost, the Shared-data architecture is the recommended approach. References • deployment/shared_data/s3.md

@Rocky I am talking about s3 express

None References • administration/management/FE_configuration.md • deployment/shared_data/feature-support-shared-data.md • faq/shared_data_faq.md • quick_start/shared-data.md • deployment/shared_data/azure.md

What do you mean

None

Apologies for the previous empty response! I'm Rocky, your StarRocks assistant. Let me clarify the support for AWS S3 Express One Zone and how it fits into StarRocks' tiered storage capabilities. Support for AWS S3 Express One Zone StarRocks supports AWS S3 Express One Zone (directory buckets) as part of its S3-compatible storage support in the Shared-data architecture (introduced in v3.0). Since S3 Express One Zone is designed for high-performance, low-latency access, it is an excellent choice for the primary storage layer in a StarRocks shared-data cluster. How to use S3 Express One Zone: 1. Bucket Name: Your bucket name must follow the AWS convention for directory buckets (e.g.,

my-bucket--use1-az4--x-s3

). 2. Configuration: You can configure StarRocks to use S3 Express by setting the

aws_s3_endpoint

and

aws_s3_region

in your FE and CN configurations (or via a

STORAGE VOLUME

). 3. Authentication: Ensure your IAM role/user has the

s3express:CreateSession

permission in addition to standard S3 permissions. S3 Express in Tiered Storage Strategies While StarRocks' built-in "tiered storage" typically refers to Local Disk Cache ↔️ Remote Object Storage, you can implement a sophisticated tiering strategy using Multiple Storage Volumes (v3.1+): * Hot/Warm Tier (S3 Express One Zone): Use a Storage Volume pointing to an S3 Express bucket for tables or partitions requiring the lowest possible latency and highest IOPS. * Cold Tier (Standard S3): Use a separate Storage Volume pointing to a standard S3 bucket for historical data to minimize costs. * Management: You can specify which storage volume a table uses at creation time:

sql
    CREATE TABLE my_table (
        ...
    )
    PROPERTIES (
        "storage_volume" = "s3_express_volume"
    );

Key Performance Note In a Shared-data architecture, StarRocks already uses a Local Data Cache on the Compute Nodes' SSDs to provide sub-millisecond performance. Using S3 Express One Zone as the backing storage further reduces "cache miss" penalties and speeds up data ingestion (compaction and write-back). Summary: You can definitely use S3 Express One Zone with StarRocks v3.0+ to enhance performance in a shared-data deployment. References • deployment/shared_data/s3.md • sql-reference/sql-statements/cluster-management/storage_volume/CREATE_STORAGE_VOLUME.md References • loading/s3.md • release_notes/release-3.2.md • data_source/feature-support-data-lake-analytics.md • deployment/shared_data/s3.md