Hello :wave: Has anyone got experience with Pinot ...
# general
j
Hello 👋 Has anyone got experience with Pinot on ADLS (Gen 2) ? Specifically: • Any idea on minimum IOPS for running Pinot smoothly on lowish load ? (i.e. is a standard Storage account "enough ? If so, how "far" can we push it ?) • Is it recommended to create a dedicated PVC for
controller.local.temp.dir
?
🌟 1
x
• pinot uses ADLS as deep store( for backup) so it’s not your query path, all the segments are required to be copied to pinot server local disks. • For
controller.local.temp.dir
it’s mostly used as temporary data store for segments uploaded to controller, which is not required if you use ADLS as deep store, so you typically just keep it to local temp should be ok. • Pinot server uses PVC to serve data, please ensure you give enough disk space and SSD for query performance. On Azure, by default we use standard AzureDisk as PVC. You can also try AzureFile if you don’t care about the perf much.
✔️ 1
j
Thanks @User
Does a Standard SSD with 500 IOPS seem enough for lowish loads or it is not a good idea ? https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types#standard-ssd I get that we could have better performance (latency & concurrency) with higher grade disks of course
x
depends on your use cases, standard SSD is good for typical use cases.
j
We'll start with that, thanks again