Hi all,
I have a high level question. When you sta...
# generalc
Clovis Leoncio
12/12/2023, 1:40 PMHi all,
I have a high level question. When you start using Pinot and providing data for a lot of teams services and even outside your organization (and by that I mean department) what would be the recommended topology? Smaller clusters with their own data, kinda like a microservice paradigm or a large cluster for everyone kinda like a data lake?
I've been trying to find some article or guidance about that but couldn't.
j
Johan Adami
12/12/2023, 2:03 PMyou can do either. but it’s likely more expensive (cost, time, maintenance) to provide it like a microservice. You need controllers and zookeepers for each cluster. You need to keep an inventory of clusters and know how to route users to the right one. You need to upgrade them all one after the other. It’s doable, but it’s a lot of overhead.
Johan Adami
12/12/2023, 2:05 PMsince you’re “starting”, I would advise picking a low cardinality (1-5) “thing” to divide use cases by. In our case, we picked “how important is the user/use case”. With those 3-5 clusters, we could focus on making sure SLCs made sense, reliability was good, there was a clear order of upgrades, etc.
Johan Adami
12/12/2023, 2:06 PMand then make sure each user is at least in their own server tenant and ideally their own broker tenants
c
Clovis Leoncio
12/12/2023, 4:46 PMWe're not actually starting, I misspoke. What I meant was start exposing to more and more teams, some outside the org. We're planning expansion. The idea of having different Pinot "microservices" is that each team needing to use Pinot would have their own cluster. My team would help onboarding, creating infrastructure etc but would not own maintenance of each cluster. I do prefer not having this spread out but I'm worried about scaling it for different use cases
j
Johan Adami
12/12/2023, 5:16 PMi haven’t really seen a good model where other teams “own the database” outside of things like small RDS instances for internal applications. It’s hard to be an expert in a product and also know how to manage a complex database
Johan Adami
12/12/2023, 5:17 PMand that’s really for any type of DB, not just pinot
Johan Adami
12/12/2023, 5:18 PMthis is exactly why we put users on clusters we owned based on how important their use case was. we own the DB, they own how they use it, we’re both aligned on how careful we have to be with it
c
Clovis Leoncio
12/12/2023, 5:43 PMThank you for your input. t really seems to make sense centralizing this in just one team
n
Neha Pawar
12/13/2023, 5:26 AM
speaking from experience at LinkedIn, one big cluster for mixed set of usecases (as large as 80) with centralized expertise, was preferred over having a data mesh with expertise within each sub cluster. scaling decisions taken centrally, proved to be a lot more economical and operationally efficient
thanks 1