piby
04/04/2025, 1:05 PMpinotdb.exceptions.DatabaseError: {'errorCode': 235,
'message': 'ServerSegmentMissing:\n'
'Found 1 unavailable segment for table table_name: '
'[table_name__0__2__20250307T0805Z]'}
We use multistage engine and we see this happening frequently.
This causes downtime for our tables if one query consumes a lot of memory leading to server restart.
Is this a bug in Pinot? We are currently considering setting the replica to 1 instead of 3 as it is not providing us high availability currently as it is supposed to.Xiang Fu
Xiang Fu
Xiang Fu
piby
04/05/2025, 12:36 PMpiby
04/05/2025, 12:47 PMJAVA_OPTS: -XX:+ExitOnOutOfMemoryError -Xms50G -Xmx50G -Djute.maxbuffer=100000000 -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xlog:gc*:file=/opt/pinot/gc-pinot-server.log -javaagent:/opt/pinot/etc/jmx_prometheus_javaagent/jmx_prometheus_javaagent.jar=8008:/opt/pinot/etc/jmx_prometheus_javaagent/configs/pinot.yml -Dlog4j2.configurationFile=/opt/pinot/etc/conf/pinot-server-log4j2.xml -Dplugins.dir=/opt/pinot/plugins
Now that I think of it, maybe we should try removing -XX:+ExitOnOutOfMemoryError ? 🤔Xiang Fu
Xiang Fu
piby
04/06/2025, 5:13 PMXiang Fu
Xiang Fu
piby
04/07/2025, 5:43 AMpiby
04/07/2025, 5:49 AMXiang Fu