@Rocky ur view on this please Given minio now is going to closed source (soon), has anyone used starrocks with nvidia aistore as its s3 file system? I'd love to know your experience with it. Thanks

NVIDIA AIStore (AIS) is an excellent candidate for a StarRocks backend, especially in high-performance or AI-centric environments. Since StarRocks is designed to be storage-agnostic via its S3-compatible interface, moving from other S3 providers to AIStore is technically straightforward. Here is the "Rocky" view on using StarRocks with AIStore: 1. Technical Compatibility StarRocks supports any S3-compatible storage for both Data Lake Analytics (External Catalogs) and Shared-Data Clusters (Compute Nodes with shared storage). In StarRocks v3.1.9+ and v3.2.4+, there is explicit support for "Generic S3-compatible" services. If AIStore uses a specific URI prefix that StarRocks doesn't automatically recognize as S3, you can register it in your BE/CN configuration: * Configuration (

be.conf

or

cn.conf

):

properties
    s3_compatible_fs_list = ais://

(Note: Use the prefix your AIStore implementation uses, often just

s3://

if mapped via its S3 gateway). 2. Why AIStore makes sense for StarRocks * Linear Scalability: AIStore is built to scale linearly across many nodes without a centralized bottleneck (like a metadata server). This complements StarRocks' distributed architecture perfectly. * High Throughput: AIStore is optimized for high-speed data movement (originally for training workloads). In a Shared-Data (Cloud-Native) StarRocks deployment, the bottleneck is often the speed at which Compute Nodes (CNs) can pull data from the object store into their local cache. AIStore’s performance profile excels here. * Local Caching: StarRocks Compute Nodes use a local disk cache for "hot" data. AIStore's ability to handle massive concurrent GET requests means that "cold" starts (initial cache warming) will likely be faster than traditional object stores. 3. Implementation Steps To use AIStore as your primary storage volume in a StarRocks shared-data cluster: 1. Create a Storage Volume:

sql
    CREATE STORAGE VOLUME ais_volume
    TYPE = S3
    LOCATIONS = ("<s3://your-ais-bucket/>")
    PROPERTIES
    (
        "aws.s3.endpoint" = "<http://ais-gateway-ip:8080>",
        "aws.s3.access_key" = "your_access_key",
        "aws.s3.secret_key" = "your_secret_key",
        "aws.s3.enable_path_style_access" = "true"
    );

2. Set as Default (Optional):

sql
    SET ais_volume AS DEFAULT STORAGE VOLUME;

4. Experience Note While AIStore is highly performant, ensure your S3 Gateway for AIStore is properly tuned. StarRocks performs many small metadata operations (LIST/HEAD) alongside large data reads. AIStore’s lightweight metadata handling generally performs well here, but you should monitor the latency of these operations compared to your previous setup. Verdict: If you are already running NVIDIA hardware or require a high-performance, open-source alternative for your data lake, AIStore is a very strong architectural fit for StarRocks. It avoids "lock-in" while potentially offering better throughput for the large-scale scans StarRocks is known for. References • data_source/feature-support-data-lake-analytics.md • quick_start/shared-data.md • deployment/shared_data/feature-support-shared-data.md • deployment/shared_data/s3.md