@Rocky why would I see a 0B DataSize for tablets when I know that there is data

Seeing a

0B

(Zero Bytes)

DataSize

for tablets when you know data is present is a common occurrence in StarRocks, usually due to how metadata and statistics are updated. Here are the most likely reasons for this behavior: 1. Reporting Latency (Statistics Delay) StarRocks does not update table and tablet statistics in real-time for every single write to avoid performance overhead on the Frontend (FE). * The Delay: Statistics displayed in commands like

SHOW DATA

or

SHOW TABLET

are updated periodically (often every few minutes or after a specific number of versions). * Verification: Run a

SELECT count(*)

on the table. If it returns the expected row count, your data is safely stored and searchable; only the metadata reporting is lagging. 2. External Tables If you are looking at an External Table (e.g., Hive, Iceberg, Hudi, or MySQL external tables), StarRocks does not store the data locally on its Backends (BEs). * Since the data resides in an external system (like S3, HDFS, or a different database), the "Data Size" for the StarRocks metadata will often show as

0B

because no local StarRocks data files (segments) exist for those tablets. 3. Data in MemTable (Unflushed Data) In some scenarios, especially with very small or frequent streaming loads (like via Routine Load or Stream Load), data might still be sitting in the MemTable on the BE and hasn't been flushed to a segment file on disk yet. * Metadata statistics usually only count data that has been "persisted" into segment files. Once a flush occurs (manually or via the system threshold), the size will eventually update. 4. Shared-Data Architecture (StarRocks 3.x+) If you are using a Shared-Data cluster, the tablets are stored in object storage (like S3/OSS). * The FE might not have yet synchronized the latest metadata from the object store. * You can sometimes see

0B

if the local cache on the BEs hasn't been populated or if the FE's view of the "visible version" hasn't yet calculated the size of the new segments in object storage. 5. Primary Key Tables For Primary Key tables, the

DataSize

reporting can sometimes be counter-intuitive because of how StarRocks handles updates. * If you are performing many

UPSERT

operations, the reported size might not change immediately because the system is managing "Delete Vectors" and "Persistent Indexes" in the background. The actual disk usage update is often deferred until a compaction task runs. How to get more accurate info: To see more granular or up-to-date information, try these commands: 1. Check the Partition Metadata: This is often more reliable than

SHOW DATA

.

sql
    SELECT * FROM information_schema.partitions_meta
    WHERE TABLE_NAME = 'your_table_name';

2. Check BE-specific Tablet Info: This queries the BEs directly for what they see on disk.

sql
    SELECT * FROM information_schema.be_tablets
    WHERE TABLET_ID = <your_tablet_id>;

3. Force a check on a specific tablet:

sql
    SHOW TABLET <tablet_id>;

(In the output, look at the

Version

and

RowCount

fields. If they are increasing, data is being recognized even if DataSize is lagging.) Summary: If

SELECT count(*)

works, you don't need to worry. The

0B

is almost certainly a temporary metadata reporting delay or a side effect of using external storage type (External/Shared-data). References • best_practices/primarykey_table.md • table_design/table_types/primary_key_table.md

@Rocky does table pruning prevent partition pruning?

No, Table Pruning does not prevent Partition Pruning. In fact, they are separate optimizations that work at different levels of the query planning process. Here is how they relate and why they don't conflict: 1. Different Levels of Optimization * Table Pruning (Higher Level): This is a Join optimization (introduced in v3.1) that occurs during the logical planning phase. If StarRocks determines that a table in a JOIN (like an Inner or Left Join) is not needed to produce the final result—usually because of foreign key constraints and because no columns from that table are selected—it removes the entire table from the execution plan. * Partition Pruning (Lower Level): This occurs during the physical planning/scan phase. It uses predicates in your

WHERE

clause (e.g.,

WHERE date >= '2023-01-01'

) to skip reading specific partitions within a table. 2. How They Interact * If a table is pruned: The entire table is removed from the query plan. Because the table is no longer being scanned, "Partition Pruning" for that specific table becomes irrelevant—you've already achieved the ultimate pruning by skipping the table entirely. * If a table is NOT pruned: If the table remains in the plan, StarRocks will proceed to evaluate if it can perform Partition Pruning (skipping partitions) and Bucket Pruning (skipping tablets) based on your query filters. 3. Does one "Block" the other? There is no scenario where enabling Table Pruning would disable the engine's ability to prune partitions on the tables that remain in the query. Summary of Differences Feature | Level | Requirement | Result Table Pruning | Join/Logical | Foreign Key constraints, Cardinality-preserving joins | Entire table removed from the plan. Partition Pruning | Scan/Physical | Predicates on the Partition Key | Specific data folders/files skipped. Pro Tip: If you want to see which one is happening, use the

EXPLAIN

command: * If you don't see an

OlapScanNode

for a table that was in your SQL, Table Pruning occurred. * If you see

partitions=1/20

in the

OlapScanNode

, Partition Pruning occurred. References • using_starrocks/accelerate-cardinality-preserving-joins.md • best_practices/partitioning.md References • using_starrocks/accelerate-cardinality-preserving-joins.md • unloading/Spark_connector.md

@Rocky how can i set the start date of a time_slice