Hi All, can anyone explain exact meaning/difference of 'controller.data.dir' configuration in Controller and 'pinot.server.instance.dataDir' and 'pinot.server.instance.segmentTarDir' in Server's configuration. Thankyou in advance :)

pinot.server.instance.dataDir is where the server will store uncompressed segments with indexes, and it will be used for query serving. this is typically a server’s local disk controller.data.dir is where compressed segments will be stored. they may or may not have all indexes. this is usually for permanent backup anc recovery, and not used in query serving path e.g. used in case of capacity changes, node failures, rebalance. This is typically some remote fs like S3, GCS in prod setups (or can be controller’s local disk in quickstart, quick pocs)

@Neha Pawar is there a best practices document or blog one can refer for these configurations in a prod environment?

Hi @Neha Pawar thankyou for response. one more configuration which is bit confusing to me. 'pinot.server.instance.segmentTarDir' what's the purpose of this configuration in Server's config. As i noticed since doing PoC on Pinot there is not a single file is created on given local path. Please can you explain this also.