<@U0A71G31CDV> is it recommended to give partition...
# questions-and-troubleshooting
m
@Rocky is it recommended to give partition by week with 6buckets or month with 24 buckets in a table?
r
The choice between partitioning by week with 6 buckets versus month with 24 buckets depends primarily on your data volume, cluster size, and query patterns. In StarRocks, the most critical factor is the resulting tablet size (a tablet = 1 bucket within 1 partition). 1. The Golden Rule: Tablet Size * Recommended Size: 100 MB to 1 GB (compressed data). * Maximum Size: Try to keep tablets under 10 GB. Tablets larger than this can cause slow data compaction and imbalanced query performance. * Total Tablet Count: Avoid exceeding 100k–200k tablets per BE node to prevent FE metadata overhead. 2. Comparison of your options Assuming a month is roughly 4 weeks, both strategies result in the same total number of buckets over time (24 per month). The difference lies in granularity and parallelism: Feature | Week + 6 Buckets | Month + 24 Buckets Pruning | Finer. Can skip data at a weekly level. Better for "last 7 days" queries. | Coarser. Any query touching a single day must scan the entire month's buckets. Parallelism | Low. Only 6 buckets can be processed in parallel per partition. If you have 10+ BE nodes, some will be idle. | Medium. 24 buckets allow better distribution across larger clusters. Metadata | Higher. 4x more partitions than the monthly strategy. | Lower. Fewer partition objects for the FE to manage. 3. Which one should you choose? * Choose Weekly (6 buckets) if: * Your total data per month is relatively small (e.g., < 20 GB). * You frequently query specific weeks or recent data, and you want to minimize the data scanned. _ You have a very small cluster (e.g., 1–3 BE nodes). A rule of thumb is to have at least
2 _ Number of BEs
buckets per partition to ensure all nodes participate in the query. * Choose Monthly (24 buckets) if: * You have a larger cluster (e.g., 5–12 BE nodes). 24 buckets allow for better parallelism. * Your data volume is high (e.g., > 100 GB per month). * Your queries usually span long time ranges (e.g., "last 3 months") where weekly pruning doesn't offer a significant advantage. Recommendation If you are on StarRocks v2.5.7 or later, it is highly recommended to use Automatic Bucketing. You can omit the
BUCKETS
keyword, and StarRocks will automatically calculate the optimal number based on your BE node count and historical data volume:
Copy code
sql
DISTRIBUTED BY HASH(join_key) -- No BUCKETS specified
To give you a more precise recommendation, could you share: 1. How much data (GB) do you ingest per week/month? 2. How many BE nodes are in your cluster? 3. What is the typical time range of your queries? References • table_design/data_distribution/Data_distribution.md • deployment/plan_cluster.md • faq/Deploy_faq.md • best_practices/bucketing.md • knowledge/usage/best-practice.md