http://starrocks.io logo
Join Slack
Powered by
<@U0A71G31CDV> I have tablets distributed for all ...
# questions-and-troubleshooting
m
@Rocky I have tablets distributed for all compute nodes. how can I find size of tablets per compute node?
r
To find the size of tablets per compute node (BE or CN), you can query the 
information_schema.be_tablets
table. This table provides detailed information about every tablet across all nodes in the cluster. 1. Total data size per node To see the total size of all tablets aggregated by node, use the following query:
Copy code
sql
SELECT 
    BE_ID, 
    SUM(DATA_SIZE) / 1024 / 1024 / 1024 AS DATA_SIZE_GB, 
    COUNT(TABLET_ID) AS TABLET_COUNT
FROM information_schema.be_tablets
GROUP BY BE_ID;
2. Sizes of individual tablets on a node If you need to see the size of each individual tablet on a specific node (e.g., node ID 
10001
), use:
Copy code
sql
SELECT 
    TABLET_ID, 
    TABLE_ID, 
    DATA_SIZE / 1024 / 1024 AS DATA_SIZE_MB
FROM information_schema.be_tablets
WHERE BE_ID = 10001
ORDER BY DATA_SIZE DESC;
3. Quick overview of node capacity If you just want a high-level overview of how much data is stored on each backend without tablet-level granularity, you can use:
Copy code
sql
SHOW PROC '/backends';
The DataUsedCapacity column in the output shows the total data size per node. Note for Shared-Data clusters If you are using StarRocks in shared-data mode (v3.0+), data is stored in remote storage (S3/OSS/HDFS). In this case: • The 
DATA_SIZE
in 
be_tablets
represents the logical size of the tablet in remote storage. • If you are looking for the size of data currently cached on the compute nodes (CN), you should monitor the DataCache metrics or check the disk usage in the 
storage_root_path
defined in your CN configuration. References • knowledge/admin_and_operation/shared_data_compaction.md
m
@Rocky information_schema.be_tablets this table query is taking lot of time in my cluster and getting timedout of 1min. I'm running 4.0.6 cluster shared data mode.
r
If querying 
information_schema.be_tablets
is timing out, it is likely due to a very large number of tablets in your cluster. In shared-data mode (lake mode), gathering this metadata across all nodes can be expensive. Here are more efficient ways to find the size of tablets or the distribution of data per compute node (CN) in StarRocks 4.0: 1. High-Level Summary per Node If you only need the total data size and the number of tablets currently assigned to each node, use 
SHOW PROC
. This is much faster than querying system tables because it uses internal cached metrics.
Copy code
sql
SHOW PROC '/backends';
-- OR if you are using specific compute nodes
SHOW PROC '/compute_nodes';
Look for: * TabletNum: The number of tablets currently assigned to that node. * DataUsedCapacity: The total data size handled by that node. Note that in shared-data mode, this typically reflects the logical data size of the tablets the node is responsible for (e.g., for compaction tasks). 2. Querying 
be_tablets
by Individual Node If the global query (across all nodes) times out, try querying for one node at a time. This reduces the metadata gathering pressure on the FE. First, get your node IDs from 
SHOW BACKENDS
, then run:
Copy code
sql
-- Increase timeout for this session to allow more time for gathering
SET query_timeout = 300; 

SELECT 
    BE_ID, 
    SUM(DATA_SIZE) / 1024 / 1024 / 1024 AS DATA_SIZE_GB, 
    COUNT(TABLET_ID) AS TABLET_COUNT
FROM information_schema.be_tablets
WHERE BE_ID = <insert_node_id_here>
GROUP BY BE_ID;
3. Using 
partitions_meta
(Partition Level) If you are trying to find where the "bulk" of your data is, query 
information_schema.partitions_meta
. This table is often faster than 
be_tablets
because it stores aggregated partition-level information.
Copy code
sql
SELECT 
    DB_NAME, 
    TABLE_NAME, 
    SUM(DATA_SIZE) / 1024 / 1024 / 1024 AS SIZE_GB
FROM information_schema.partitions_meta
GROUP BY DB_NAME, TABLE_NAME
ORDER BY SIZE_GB DESC;
4. Tablet Size for a Specific Table If you want to see the tablet distribution and sizes for a specific large table, use 
SHOW TABLET
. You can filter by node (backend) to see which tablets are on which node:
Copy code
sql
SHOW TABLET FROM <db_name>.<table_name> 
WHERE backendid = <node_id>
ORDER BY data_size DESC 
LIMIT 10;
Why is it slow? In StarRocks 4.0 shared-data mode, the "location" of a tablet is more about which node is currently the owner of that tablet's metadata and background tasks (like compaction). * Too many tablets: If you have millions of tablets, the FE must crawl the metadata for every single one to answer your query. Consider using Random Bucketing or Automatic Bucketing in future tables to keep the tablet count optimal. * Recommendation: Check the 
TabletNum
in 
SHOW PROC '/backends'
. If a single node has >100,000 tablets, this is likely the cause of the timeout. Summary Recommendation Use 
SHOW PROC '/backends'
 for the quickest node-level summary. If you must use 
be_tablets
, wrap it in a script to query one 
BE_ID
at a time and increase your 
query_timeout
to 
300
(5 minutes). References • sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_PROC.md • sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_BACKENDS.md
m
@Rocky I want size of data per compute node. this query is also hitting timeout of 1mins select * from information_schema.be_tablets limit 2. what could be the reason?
r
The reason even 
SELECT ... LIMIT 2
is timing out is that the StarRocks Frontend (FE) must first gather the entire metadata list of all tablets from all nodes before it can apply the 
LIMIT
. In a cluster with a very high number of tablets (common in shared-data mode), this metadata collection process involves heavy internal RPC calls that easily exceed the default 60-second or 300-second 
query_timeout
, regardless of how many rows you actually want to see. Here are the most efficient ways to get this information without hitting timeouts: 1. The Fastest Method: 
SHOW PROC
 This command uses pre-calculated metrics and does not crawl every tablet, making it nearly instantaneous. For a summary of data size and tablet count per node:
Copy code
sql
SHOW PROC '/backends';
-- OR if you specifically use Compute Nodes
SHOW PROC '/compute_nodes';
* TabletNum: The number of tablets the node is responsible for. * DataUsedCapacity: The logical size of data associated with those tablets. 2. The Table-Level Distribution If you want to see how a specific table is distributed across your compute nodes:
Copy code
sql
ADMIN SHOW REPLICA DISTRIBUTION FROM <database_name>.<table_name>;
This returns a list of 
BackendId
, the number of replicas (tablets) on that node, and the percentage distribution. 3. More Performant Metadata Table: 
partitions_meta
 If you need SQL-like flexibility, use 
information_schema.partitions_meta
. It is much faster than 
be_tablets
because it stores aggregated partition-level data rather than individual tablet details.
Copy code
sql
SELECT 
    DB_NAME, 
    TABLE_NAME, 
    SUM(DATA_SIZE) / 1024 / 1024 / 1024 AS SIZE_GB
FROM information_schema.partitions_meta
GROUP BY DB_NAME, TABLE_NAME;
4. How to fix the 
be_tablets
Timeout If you absolutely must use 
be_tablets
(e.g., for custom scripts), you must increase the session timeout and query by specific node to reduce the workload:
Copy code
sql
-- Set timeout to 10 minutes for this session
SET query_timeout = 600; 

-- Query only ONE node at a time to prevent FE memory/RPC overload
SELECT 
    BE_ID, 
    SUM(DATA_SIZE) / 1024 / 1024 / 1024 AS DATA_SIZE_GB
FROM information_schema.be_tablets 
WHERE BE_ID = <backend_id>
GROUP BY BE_ID;
Why is this happening in StarRocks 4.0? In StarRocks 4.0 (shared-data mode), every tablet is backed by a metadata file in object storage. If your cluster has "tablet explosion" (e.g., millions of tablets due to high bucket counts and many partitions), the FE's task of synchronizing and reporting on every single tablet becomes a bottleneck. Recommendation: Check your total tablet count with 
SHOW PROC '/statistic'
. If it is in the millions, consider using Automatic Bucketing for your tables to reduce the metadata pressure. f the metadata pressure. References: • </docs/sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_PROC|SHOW PROC> • </docs/sql-reference/sql-statements/cluster-management/nodes_processes/SHOW_TABLET|SHOW TABLET> References • faq/shared_data_faq.md • release_notes/release-4.0.md • deployment/shared_data/feature-support-shared-data.md • knowledge/admin_and_operation/shared_data_compaction.md
2 Views
Open in Slack
PreviousNext
Powered by