Hi @Rocky, I noticed that after restoring to the most recent snapshot, the data files written between the most recent restore time and before my source cluster went down are still in my storage bucket, I'm running starrocks version 3.5.20, is there some issue there? How can we clean up these old files?

In StarRocks (especially in shared-data/Lake clusters), seeing leftover data files after a restore is a known behavior. When you restore to a previous snapshot, you are rolling back the metadata (the pointers to the data), but the physical data files (segments) that were written to the bucket between the snapshot and the failure remain as "orphan files." These files are no longer referenced by the current system state, but they are not immediately deleted to ensure safety and allow for concurrent query consistency. How to Clean Up Old Files 1. Automated Cleanup (AutoVacuum) StarRocks has a background process called AutoVacuum that automatically identifies and deletes orphan files in your storage bucket. You should check the following FE configurations: *

lake_autovacuum_grace_period_minutes

: This determines how long historical or unreferenced data is kept. The default is usually 30 minutes (it was 5 minutes in older versions). If you want to speed up the cleanup of these "future" files, you can temporarily reduce this value:

sql
    ADMIN SET FRONTEND CONFIG ("lake_autovacuum_grace_period_minutes" = "5");

*

lake_autovacuum_parallel_partitions

: Ensures this is set to a value greater than 0 (default is 8) so the vacuum process can run. 2. Manual Cleanup Tool (

lake_datafile_gc

) If the automatic vacuum does not remove the files (e.g., if the metadata for those specific files was lost entirely during the cluster crash), you can use the

lake_datafile_gc

tool. This tool explicitly compares the files in your S3/OSS/HDFS bucket with the Tablet Meta in StarRocks to identify and delete unreferenced data. Basic Usage: 1. Download the

data_gc

tool (often found in the StarRocks release package or via the official community link). 2. Run the tool in "audit" mode first (to see what would be deleted):

bash
    ./bin/meta_tool.sh --operation=lake_datafile_gc \
      --root_path="<s3://your-bucket/path>" \
      --conf_file=./conf/be.conf \
      --do_delete=false

3. Once verified, set

--do_delete=true

to perform the actual cleanup. 3. Manual Compaction In some cases, triggering a manual compaction can help the system reconcile its state and mark old versions for vacuuming:

sql
ALTER TABLE <table_name> COMPACT;

Is there an "Issue"? It is generally *not a bug*; it's a design choice in the Lake architecture to prevent data loss. However, if the files persist for many hours or days despite the

AutoVacuum

settings, it may indicate that the vacuum process is being throttled or is encountering errors. You can check the FE/BE logs for "Vacuum" or "GC" related errors to confirm. Note: For shared-nothing (OLAP) clusters where you only use the bucket for

BACKUP

repositories, the bucket files are managed by the repository retention. You can clean up old snapshots using:

sql
DROP SNAPSHOT <snapshot_name> FROM <repository_name>;

References • knowledge/admin_and_operation/shared_data_compaction.md

Hi @Rocky, is the 4.0 series recommended for production?

Yes, the StarRocks 4.0 series is recommended for production. As of late 2025 and throughout 2026, the 4.0 series has reached a level of maturity suitable for enterprise environments. It introduced significant performance improvements (up to 60% faster query execution in some benchmarks) and key features like Decimal256, enhanced Apache Iceberg support, and Flat JSON V2. Key Points for Production Use: * Stability: Since its initial release in October 2025, several patch versions (e.g., 4.0.1, 4.0.2) have been released to address bugs and improve stability. * Production Focus: With the arrival of subsequent versions like 4.1 (which is explicitly marketed as "Built for Production"), the 4.x line is now the primary focus for new production deployments. * Maturity Levels: Most features in the 4.0 series are GA (Generally Available), meaning they are comprehensively tested and production-ready. Recommendations: 1. Use the Latest Patch: Always use the most recent patch version (e.g., 4.0.x) to ensure you have the latest bug fixes. 2. Check Feature Labels: While the core engine is production-ready, some specific new features might still be labeled as "Preview" or "Beta" in the documentation. Avoid using these for critical workloads unless necessary. 3. Upgrade Path: If you are moving from the 3.x series, review the Upgrade Notes in the release documentation, as 4.0 includes architectural improvements (like simplified Data Cache configurations and improved metadata handling) that may require minor adjustment to your configurations. If you are starting a new project, the 4.0 (or the even newer 4.1) series is generally the better choice over 3.x to take advantage of the latest performance optimizations and lakehouse integration features. References • release_notes/release-4.0.md • introduction/maturity.md References • release_notes/release-3.4.md • introduction/maturity.md • developers/versions.md • release_notes/release-2.5.md • introduction/versioning.md • release_notes/release-3.3.md • deployment/upgrade.md