Hi Everyone, I want to know about compression ratio in Apache Pinot. For Example, If I have a 10GB of JSON file containing records, having 100 columns, to save it in Pinot server, how much memory is required (considering there will be only 1 replica) ? Also in-memory segments gets flushed to segment store once threshold reached. So how much storage should be provisioned for deep store in controller ?

Compression depends on a variety of factors (eg cardinality of columns, data type, type of indexing used, etc). But even so, when compared to text based input (eg CSV/JSON), the compressions should be quite a lot. If you want an accurate number, just take one sample JSON, and create a pinot segment out of it (using pinot-admin)