we are going to be running a pinot (and friends) i...
# general
t
we are going to be running a pinot (and friends) instance on client's machine as our application is primarily on-prem (with uncertain plans for cloud). How much resource usage should we expect with around 10gigs of json data? https://docs.pinot.apache.org/operators/tutorials/deployment-pinot-on-kubernetes#pinot-server > E.g. Assuming data is 100 GB on disk, the container size is 4 CPU, 10GB Memory. am I right to assume around 1Gig of memory would be consumed? We will be concretely checking the limits ourselves but it would be nice to have a baseline
r
We can expect up to 90% compression with JSON data, and the complete resource utilization will depend on Ingestion Rate, Read QPS, and the Data size. If you are wondering about capacity planning this blog is a good place to start.