@Rocky During one of my loads, it failed because of disk out of space. HOw can I clear the dormant data without waiting for GC

To clear dormant or temporary data after a failed load without waiting for the automatic Garbage Collection (GC), the method depends on whether you are using a Shared-nothing (local disks) or Shared-data (object storage) architecture. 1. Shared-nothing (Local Disk Storage) If your StarRocks cluster uses local BE disks, failed loads often leave behind data in the

trash

or

snapshot

directories. * Manual Deletion (Immediate): On each BE node, navigate to the storage root path (defined by

storage_root_path

in

be.conf

) and manually delete the contents of the

trash

and

snapshot

directories.

bash
    # Caution: This is irreversible
    rm -rf /path/to/storage/trash/*
    rm -rf /path/to/storage/snapshot/*

* Force GC via Config: You can decrease the retention time and the scan interval to trigger the system's GC process more frequently using the following SQL commands:

sql
    -- Set trash expiration to 1 second (default is 86400 / 24h)
    ADMIN SET FRONTEND CONFIG ("catalog_trash_expire_second" = "1");
    -- Set BE trash check interval and expiration (on every BE)
    -- You may need to run this for each BE or via admin set config
    ADMIN SET CONFIG ("trash_file_expire_time_sec" = "1");
    ADMIN SET CONFIG ("path_scan_interval_second" = "1");

Note: Remember to set these back to reasonable values (e.g., 86400) after the cleanup to avoid performance issues. 2. Shared-data (Object Storage) In shared-data clusters, failed loads or compactions can leave unreferenced files in S3/OSS. * Garbage Collection Tool: StarRocks provides a manual GC tool (

meta_tool.sh

with

lake_datafile_gc

) for shared-data clusters.

bash
    # Example usage (refer to your specific version for the exact script location)
    ./bin/meta_tool.sh --operation=lake_datafile_gc --root_path="<s3://your-bucket/path>" --do_delete=true --expired_sec=0

_It is highly recommended to run with

--do_delete=false

first to audit what will be deleted._ 3. Clear Data Cache If you are using the Data Cache feature for shared-data, the cache itself might be filling up the local BE disks. * Manual: Delete the

datacache

directory on the BE nodes. * SQL: Toggle the cache size to 0 and back to trigger an automatic purge:

sql
    -- Replace <BE_ID> with your target BE ID
    UPDATE be_configs SET VALUE="0" WHERE NAME="datacache_disk_size" and BE_ID=<BE_ID>;
    -- Set it back to your original size (e.g., 2T)
    UPDATE be_configs SET VALUE="2T" WHERE NAME="datacache_disk_size" and BE_ID=<BE_ID>;

4. General Recommendations * Drop Table Force: If you are deleting tables to free up space, use

DROP TABLE <table_name> FORCE

. This bypasses the recycle bin and deletes data immediately. * Monitor Disk Usage: If a disk reaches 95% usage, StarRocks may stop accepting new load tasks or even stop the BE process. Ensure you clear the

trash

directory manually if the BE cannot due be. immediately if you hit this limit. References • knowledge/admin_and_operation/shared_data_garbage_cleanup.md • using_starrocks/Cost_based_optimizer.md • faq/Others.md • administration/management/compaction.md