we are going to be running a pinot (and friends) instance on client's machine as our application is primarily on-prem (with uncertain plans for cloud).
How much resource usage should we expect with around 10gigs of json data?
https://docs.pinot.apache.org/operators/tutorials/deployment-pinot-on-kubernetes#pinot-server
> E.g. Assuming data is 100 GB on disk, the container size is 4 CPU, 10GB Memory.
am I right to assume around 1Gig of memory would be consumed? We will be concretely checking the limits ourselves but it would be nice to have a baseline
r
Randika Navagamuwa
05/23/2024, 7:23 AM
We can expect up to 90% compression with JSON data, and the complete resource utilization will depend on Ingestion Rate, Read QPS, and the Data size. If you are wondering about capacity planning this blog is a good place to start.