The choice between partitioning by
week with 6 buckets versus
month with 24 buckets depends primarily on your
data volume,
cluster size, and
query patterns.
In StarRocks, the most critical factor is the resulting
tablet size (a tablet = 1 bucket within 1 partition).
1. The Golden Rule: Tablet Size
*
Recommended Size: 100 MB to 1 GB (compressed data).
*
Maximum Size: Try to keep tablets under 10 GB. Tablets larger than this can cause slow data compaction and imbalanced query performance.
*
Total Tablet Count: Avoid exceeding 100k–200k tablets per BE node to prevent FE metadata overhead.
2. Comparison of your options
Assuming a month is roughly 4 weeks, both strategies result in the same total number of buckets over time (24 per month). The difference lies in
granularity and
parallelism:
Feature |
Week + 6 Buckets |
Month + 24 Buckets
Pruning | Finer. Can skip data at a weekly level. Better for "last 7 days" queries. | Coarser. Any query touching a single day must scan the entire month's buckets.
Parallelism |
Low. Only 6 buckets can be processed in parallel per partition. If you have 10+ BE nodes, some will be idle. |
Medium. 24 buckets allow better distribution across larger clusters.
Metadata | Higher. 4x more partitions than the monthly strategy. | Lower. Fewer partition objects for the FE to manage.
3. Which one should you choose?
*
Choose Weekly (6 buckets) if:
* Your total data per month is relatively small (e.g., < 20 GB).
* You frequently query specific weeks or recent data, and you want to minimize the data scanned.
_ You have a very small cluster (e.g., 1–3 BE nodes). A rule of thumb is to have at least
2 _ Number of BEs
buckets per partition to ensure all nodes participate in the query.
*
Choose Monthly (24 buckets) if:
* You have a larger cluster (e.g., 5–12 BE nodes). 24 buckets allow for better parallelism.
* Your data volume is high (e.g., > 100 GB per month).
* Your queries usually span long time ranges (e.g., "last 3 months") where weekly pruning doesn't offer a significant advantage.
Recommendation
If you are on
StarRocks v2.5.7 or later, it is highly recommended to use
Automatic Bucketing. You can omit the
BUCKETS
keyword, and StarRocks will automatically calculate the optimal number based on your BE node count and historical data volume:
sql
DISTRIBUTED BY HASH(join_key) -- No BUCKETS specified
To give you a more precise recommendation, could you share:
1. How much data (GB) do you ingest per week/month?
2. How many BE nodes are in your cluster?
3. What is the typical time range of your queries?
References
•
table_design/data_distribution/Data_distribution.md
•
deployment/plan_cluster.md
•
faq/Deploy_faq.md
•
best_practices/bucketing.md
•
knowledge/usage/best-practice.md