Hi @Rocky, @Kevin Cai, everyone, Any idea when the automated cluster snapshot feature will be GA?

The Automated Cluster Snapshot feature is currently available in Beta for shared-data clusters. As of September 2026, there has not been a formal announcement transitioning this specific feature to General Availability (GA). While it has seen significant enhancements in recent releases, the official documentation still labels it as a beta feature. Current Status & Recent Updates: * Introduced (v3.4.2): Basic support for automated snapshots was first added for shared-data clusters. * Enhanced (v4.0): Added the

ADMIN SHOW AUTOMATED CLUSTER SNAPSHOT

statement to monitor status and schedules. * Improved (v4.1): Added interval-based scheduling, allowing for more granular control over when snapshots are triggered. Important Notes: * Architecture: It is only supported on shared-data (compute-storage separated) clusters. * Beta Warning: Because it is in beta, the system typically only retains the single latest snapshot by default. It is recommended to verify your recovery workflows in a non-production environment first. If you need a definitive GA timeline for production planning, I recommend checking the StarRocks GitHub Roadmap or asking the maintainers in the

#starrocks-user

Slack channel. References • administration/cluster_snapshot.md • administration/management/FE_configuration.md • deployment/shared_data/feature-support-shared-data.md

Noted, I also confirmed this by doing some manual tests @Rocky. One more thing I want to is if we set a large time say 1hr for snapshot interval, how does it impact the data files referenced by the snapshot at that time because compaction would have happened within the 1hr of taking snapshot and I know

lake_autovacuum_grace_period_minutes

is the control for the time

Setting a large interval (e.g., 1 hour) for automated snapshots has specific implications for data retention and storage. In a StarRocks shared-data cluster, the system manages this through a combination of metadata pinning and the AutoVacuum process. 1. Data Pinning (Automatic Protection) The most important thing to understand is that an active Automated Cluster Snapshot "pins" the data versions it references. Even if you set a 1-hour interval and significant compaction occurs within that hour: * The FE/StarMgr keeps a record of all data files (rowsets) required by the currently valid automated snapshot. * The AutoVacuum process is aware of these references. It will not delete any data files required by the existing snapshot, even if those files have been superseded by compaction in the live cluster and their

lake_autovacuum_grace_period_minutes

has expired. * The "pin" is only released once a new automated snapshot is successfully created and the old one is dropped. 2. The Role of

lake_autovacuum_grace_period_minutes

This parameter acts as a buffer for the live cluster, not specifically for the snapshot. It ensures that files recently marked as "stale" (by compaction or schema changes) stay around long enough for active queries to finish. * If your snapshot interval is 60 minutes and

lake_autovacuum_grace_period_minutes

is 30 minutes, the data for your 12:00 PM snapshot is safe until the 1:00 PM snapshot completes. * Once the 1:00 PM snapshot is done, the 12:00 PM snapshot is deleted, and its unique data files will then be eligible for vacuuming after the 30-minute grace period (around 1:30 PM). 3. Impact of a Large Interval If you move from a 10-minute interval to a 1-hour interval: * Storage Overhead: You will likely see higher storage usage in your object bucket. Because you are only taking snapshots every hour, StarRocks must retain all data versions from the "start of the hour" state until the next successful snapshot. If your cluster is write-heavy with frequent compactions, this "ghost data" can add up. * RPO (Recovery Point Objective): Your potential data loss in a disaster increases. If the cluster fails at 12:59 PM, you can only restore to the 12:00 PM state. * Checkpoint Pressure: Since the snapshot triggers an FE image checkpoint, a very large interval means the FE has more journal logs to replay and consolidate during that checkpoint, which might lead to a longer "Snapshotting" phase. What else do you need? To ensure this setup is robust for production, consider the following: 1. Monitor Snapshot Success: Regularly query

information_schema.cluster_snapshot_jobs

to ensure the snapshots aren't failing due to storage timeouts or FE overhead. 2. Storage Volume Permissions: Ensure the FE has

DeleteObject

permissions on the storage volume. If the FE can't delete the old snapshot metadata, you might end up with multiple metadata images even though the system technically only "retains" one. 3. Manual Snapshots for Milestones: Since automated snapshots only keep the single latest version, use

ADMIN SET MANUAL CLUSTER SNAPSHOT

before major maintenance or application releases if you need a guaranteed fallback point that won't be overwritten by the next hourly cycle. References • administration/cluster_snapshot.md • administration/management/FE_configuration.md • faq/shared_data_faq.md References • administration/management/FE_configuration.md • faq/shared_data_faq.md • administration/cluster_snapshot.md

Noted but more on this @Rocky Lets say snapshot interval is 1hr and time of last snapshot is 9:00am. Now, our primary cluster goes down at 9:49am before next snapshot so all data referenced by the old snapshot will be retained but what if we finish setting up our new cluster infra after 2hr and spin up a new starrocks with the old snapshot by 11am, how long will the old data files then be retained?

In this scenario, where you restore a cluster at 11:00 AM from a 9:00 AM snapshot, here is exactly what happens to your data files and the retention logic: 1. Which files are "Safe"? When you spin up the new cluster at 11:00 AM with the 9:00 AM metadata: * The 9:00 AM State Files: Every data file (segment/SST) that was "current" at 9:00 AM is now the "current" data for your new cluster. These files are perfectly safe because they are referenced by the active metadata. They will only be candidates for deletion after the new cluster performs its own compactions. * The "Pinned" Snapshot: Upon startup, the new FE will see that the last successful snapshot is the one from 9:00 AM. It will continue to "pin" those files in object storage to ensure the snapshot remains valid until a new one is successfully taken. 2. The "Orphaned" Files (The 9:00 AM – 9:49 AM Window) This is the critical part. Any files created by compaction or loads between 9:00 AM and 9:49 AM on the old cluster are not part of the 9:00 AM metadata. To the new cluster, these files are "orphans" (garbage). * Cleanup Timing: As soon as the new cluster's AutoVacuum service starts (shortly after 11:00 AM), it will scan the object storage. It will see these 9:20 AM files, realize they are not in its 9:00 AM metadata, and check the

lake_autovacuum_grace_period_minutes

. * Result: Since it is now 11:00 AM, a file created at 9:20 AM is already 100 minutes old. If your grace period is 30 minutes, AutoVacuum will delete these orphaned files almost immediately after the new cluster starts. 3. When does the 9:00 AM Snapshot get released? * The Interval Trigger: Since the

automated_cluster_snapshot_interval_seconds

is 1 hour and the last snapshot was 2 hours ago (9:00 AM), the FE will immediately trigger a new metadata checkpoint and snapshot shortly after the cluster becomes healthy (usually within a few minutes of startup). * The Handover: Once the new snapshot (e.g.,

automated_cluster_snapshot_1105

) is successfully uploaded to object storage, the system will delete the 9:00 AM snapshot metadata. * The "Pin" Release: Only at this moment is the "pin" on the specific 9:00 AM state released. However, as mentioned, since those files are still your "current" data, they remain safe until the next round of compactions on the new cluster. Summary of Retention at 11:00 AM: Data Category | Status at 11:00 AM Restoration Files from 9:00 AM state | Retained. They are the current live data. Files from Snapshot A | Retained. Pinned until the first successful snapshot at ~11:05 AM. Files from 900–949 AM | Deleted quickly. They are not in the 9:00 AM metadata and exceed the 30-min grace period relative to 11:00 AM. Key Takeaway: If you restore a snapshot that is older than your

lake_autovacuum_grace_period_minutes

, any progress made (compactions/intermediate loads) between that snapshot and the crash is treated as garbage and cleaned up immediately by the new cluster. This is why it is generally recommended to keep the snapshot interval relatively frequent (e.g., the default 10 mins) or ensure your grace period is large enough to cover your recovery window (MTTR) window if you want to inspect those "orphaned" files. References • administration/cluster_snapshot.md • faq/shared_data_faq.md • administration/management/FE_configuration.md References • administration/cluster_snapshot.md • administration/management/Backup_and_restore.md • administration/management/FE_configuration.md

Hi @Rocky, are there Row-Level Security Policies in starrocks

Yes, StarRocks supports Row-Level Security (RLS), but the implementation depends on the edition you are using: 1. Enterprise Edition (Native RLS) In the Enterprise/Cloud edition, StarRocks provides native Row Access Policies. This allows you to define policies directly within StarRocks using SQL. * Syntax: You create a policy and then apply it to a table or view.

sql
    -- Create a policy
    CREATE ROW ACCESS POLICY region_policy
    AS (region_col VARCHAR) RETURNS BOOLEAN ->
    region_col = 'North America';

    -- Apply it to a table
    ALTER TABLE sales_data ADD ROW ACCESS POLICY region_policy ON (region);

* System View: You can monitor these using the

information_schema.policy_references

view. 2. Open-source Edition (Apache Ranger) The native

CREATE ROW ACCESS POLICY

syntax is generally not supported in the Open-source Edition. Instead, open-source users implement RLS through Apache Ranger integration (supported from v3.1.9 onwards). * How it works: You configure the StarRocks Ranger plugin. You then define "Row-level Filter" policies in the Ranger Admin UI. * Automatic Injection: When a user queries a table, the StarRocks FE communicates with Ranger to see if a filter should be injected into the SQL execution plan (e.g., automatically adding a

WHERE region = 'xxx'

clause). 3. Alternative: View-based RLS Regardless of edition, you can always implement a basic form of RLS using Views combined with the

current_user()

or

current_role()

functions. This is a common workaround if you do not want to manage an external Ranger service:

sql
CREATE VIEW secure_sales_view AS
SELECT * FROM sales_data
WHERE region IN (
    SELECT region FROM user_permissions 
    WHERE username = current_user()
);

Summary Table Feature | Open-source Edition | Enterprise Edition Native SQL Policies | No | Yes (

CREATE ROW ACCESS POLICY

) Apache Ranger Integration | Yes (v3.1.9+) | Yes View-based RLS | Yes | Yes References • administration/user_privs/authorization/ranger_plugin.md • faq/operation_maintenance_faq.md • sql-reference/sys/policy_references.md References • sql-reference/sys/policy_references.md • sql-reference/sql-statements/cluster-management/resource_group/CREATE_RESOURCE_GROUP.md • sql-reference/sql-statements/account-management/CREATE_USER.md • sql-reference/sql-statements/table_bucket_part_index/CREATE_TABLE.md • sql-reference/data-types/semi_structured/STRUCT.md