Unmerged Index Files root@fcc-be-3:/opt/starrocks# find /opt/starrocks/be/storage/data -type f -name "index." ! -name ".merged" -printf "%s\n" | awk '{s+=$1} END {printf "Permanent Index Total: %.2f GB\n", s/1024/1024/1024}' 616.60 GB Merged Index Files root@fcc-be-3:/opt/starrocks# find /opt/starrocks/be/storage/data -type f -name "index.*.merged" -printf "%s\n" | awk '{s+=$1} END {printf "Temporary Merge Total: %.2f GB\n", s/1024/1024/1024}' 173.09 GB is there a way to merge all the unmerged indexes? will it reduce disk size

There is a column name

INDEX_SIZE

in system table

be_tablets

, you can use this table to find out which table takes most of the disk space

I dont think that is showing correct data. For a particular BE ID (628937) as can be seen in screenshot total size is less than 200GB. Whereas on the disk its 787 GB

root@fcc-be-3:/opt/starrocks# find /opt/starrocks/be/storage/data -name "index.l*" -ls | awk '{s+=$7} END {printf "PK Index: %.2f GB\n", s/1024/1024/1024}'
PK Index: 787.21 GB

SQL Query -

SELECT      bt.TABLE_ID,     tc.TABLE_SCHEMA AS db_name,     tc.TABLE_NAME,     COUNT(*) AS tablet_count,     ROUND(SUM(bt.DATA_SIZE)/1024/1024/1024, 2) AS data_gb,     ROUND(SUM(bt.INDEX_DISK)/1024/1024/1024, 2) AS reported_index_gb,     SUM(bt.NUM_ROW) AS total_rows FROM information_schema.be_tablets bt LEFT JOIN information_schema.tables_config tc ON bt.TABLE_ID = tc.TABLE_ID WHERE bt.BE_ID = 628937 GROUP BY bt.TABLE_ID, tc.TABLE_SCHEMA, tc.TABLE_NAME ORDER BY tablet_count DESC;

One way to manually reclaim PK (Primary Key) index files is as follows: Identify all historical partitions (meaning those with no active imports) and locate the tablet IDs within them. You can then clean up the PK index files inside these tablet directories. Since PK index files are only utilized during data ingestion, they are largely redundant for historical partitions. Important: You must shut down the BE (Backend) before deleting any files, and only restart the BE once the cleanup is complete.

Issue was that even though we deleted all data from a partition, the index was not deleted. After deleting each partition disk size came down, so issue is fixed now. Thanks for the help