Hi @Koorous Vargha, in earlier meetings, you mentioned about capacity: • biggest cluster: 300 servers, 280 routers (roughly 1:1 ratio). Trend is toward fewer, beefier boxes (vertical scaling of hardware). • Server hardware: 256 GB RAM, 32 cores, 6.4 TB NVMe SSD. • Router hardware: shared boxes with multiple router pods. Each pod: 24 CPUs, 36 GB RAM. Could you share what you configure for Controller and Zookeeper? I assume ZK doesn't need NVMe, EBS gp3 should suffice, but curious how the setup looks like in terms of number of instances, cores, RAM and disk. Thank you

Controller: • 6 replicas • 18 vCPUs • 53 GB memory Please keep in mind we have 16 Venice clusters, so you probably don't need something this beefy. For the ZooKeeper setup, I wasn't able to find what we're running, since it is managed by another team. @Ali do you happen to know?

we run the mostly OSS ZK , ensembles of 5 instances .

@Ali thanks. Regarding capacity - I assume it's similar to Controller, but also needs disk. I guess NVMe is an overkill, so EBS should be fine

From what I've found, it looks like our ZK is on NVMe. It might be overkill for Venice?

ZK latency is tied to write latency across ensemble. We need low latency writes and quick propagation of the state of ZK . we run it on NVMEs,

@Ali but disk requirements are not that big, right? It's just metadata. Even for your case, probably 64GB SSD would be plenty. Sorry for bugging, it's just that ZK is the most important component of Venice, so want to make sure that we avoid any future vertical scaling on those.

you are correct, size is not the limit, the write latency is