Hassan Ait Brik
04/07/2025, 9:30 PM[2161.229s][warning][os,thread] Attempt to protect stack guard pages failed (0x00007f73f9a6b000-0x00007f73f9a6f000).
#
# A fatal error has been detected by the Java Runtime Environment:
# Native memory allocation (mprotect) failed to protect 16384 bytes for memory to guard stack pages
# An error report file with more information is saved as:
# /opt/pinot/hs_err_pid1.log
#
# If you would like to submit a bug report, please visit:
# https://github.com/corretto/corretto-17/issues/
jvmOpts: "-Xms512M -Xmx10G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:MaxDirectMemorySize=16g .....
resources:
requests:
cpu: 500m
memory: 28Gi
limits:
memory: 28GiMayank
Mayank
Xiang Fu
Xiang Fu
Xiang Fu
Hassan Ait Brik
04/08/2025, 9:27 AMvm.max_map_count on our new Azure VM instance.
Details:
• Previous environment:
• On our previous machine (Azure ds3_v2), the vm.max_map_count was set to 524288. This high limit allowed Apache Pinot to open a large number of memory-mapped files (mmap), which is critical for efficiently handling large volumes of data and for operations such as upserts.
• Current environment:
• On our new machine (Azure d8s_v3), we found that the vm.max_map_count was set to a much lower value of 65530. (default) This limit was reached quickly during the server’s operation, resulting in excessive mmap consumption and leading the operating system to kill the Pinot server process.
Solution:
To resolve the issue, we increased the vm.max_map_count parameter on the new machine to match the value of the previous, proven configuration.Xiang Fu
Xiang Fu
Xiang Fu
vm.max_map_count more often blocked the cloud deploymentMayank
Hassan Ait Brik
04/08/2025, 6:34 PMMayank
Hassan Ait Brik
04/09/2025, 9:08 AMZhuangda Z
04/11/2025, 9:03 PMZhuangda Z
04/11/2025, 9:13 PM