I noticed that the Pinot server's /debug/memory/offheap API shows 512MB mmap files for each real-time consuming segment (path: data/table_name/consumers/segment_name) The same was mentioned in the javadoc of MmapMemoryManager

We create files of length 0.5g (or the requested buffer length, whichever is higher), and map areas of the file for each allocation request within a segment

Can someone confirm if at least 512MB is allocated per Kafka partition consumer, and is there a way to configure this size to reduce RES memory usage incase of high partition count ?

Mmap files don't use heap or direct memory

Thanks for the quick reply, Mayank