http://starrocks.io logo
Join Slack
Powered by
Hi folks, We are looking to onboard hundreds of c...
# questions-and-troubleshooting
j
Hi folks, We are looking to onboard hundreds of customers with one customer per database and in each database there will be a few hundreds views. How starrocks shared data mode handle the huge number of objects ? Is there some limitations on number of tablets or table or database or views or per cluster ? What can be the impact of having such situation? Would the FE node be overwhelmed with the size of metadata? Any suggestions / configurations to make this works Thanks
r
For partition count, < 100K per cluster. For tablet count, 200k per BE. Too many partitions will overwhelm the FEs. It depends on your data size, but eventually you want to aim around 1-10 gb for each tablet
j
Thank you Robert
So for production cluster, if we have a hundred of clients which has 67 mat views each, that’s already 6700, now I let starrocks creating the right number of buckets for each mat view, and often it creates 6 buckets, that’s already 40200 tablets. I have noticed that the tablets increases over time. Is it due to the mat view refreshes? How come they are not deleted? Should only one version of the mat view be kept
With just 2 customers I have noticed that on 3 compte nodes, we had 55k tablets on each one of them. And the analyze statistics job was always failing because of the meta folder not found. We didn’t know how to fix this but running analyze mat view wasn’t working anymore. We refreshed the mat views every hour
We don’t use base table and directly use s3 delta lake tables
r
Yes, MV refresh will replace the affected partitions (as in delete old partitions and construct new ones). When a new partition is created, the bucket count varies depending on many factors. I'd assume you're using random bucketing on your MV, which lead to the tablet and bucket count growing. If this is true, and you want to keep your tablet count constant, use hash bucketing instead.
j
I am using hash bucketing but without setting the number of bucket as it can change for customers. And I still see the number of tablets growing after each mat view refresh. What can be done? Is there a way to keep only 2 or 3 versions and recycle / delete the old version? We don’t need time travel
r
You have to explicitly mention the bucket count. MV doesn't support time travel regardless - it only has one version. Did you partition your mv?
j
Ah that’s the reason, ok. No the mv is not partitioned as the base table isn’t partitioned
We use liquibase to deploy into starrocks and for some client they ma have a huge amount of data and for some not so much for the same MV, how to handle this then?
What is the best approach here in terms of tablets? Should we worry about it ? FYI in our test cluster we have 255k tablets and all our views are actually the number of buckets specified, so I am not sure what do to actually. Also if we specify the number of buckets how to handle a growing mat view? We might have to increase the number of buckets in the future
Thanks for your help
r
How much disk is each bucket taking? Not sure how many customer you have, but ~250k tablet count seems a lot. This seems like an odd approach, giving each customer a single mv. Is it possible to combine all customer into single partitioned mv (maybe you can find a column that have categories, like dob, address, etc. as the partition key) with fixed bucket count? This will help keep the bucket count under control.
j
We are using shared data mode, so data is on s3, hence I don’t really understand the tablets concept for CN nodes because they are supposed to be stateless. We have one database per user because of PCI4 rules and we connect the BI tool against that customer specific db with a customer specific user read only on this db. In our data lake we have sometimes one bucket per customer or more in one bucket with one subfolder per customer
Open in Slack
PreviousNext
Powered by