I have a table on production, it was not able to commit the consuming segments from last 3 days, my kafka has retention of 2 days, so now i have deleted my consuming segments and tried to invoking the

RealtimeSegmentValidationManager

task, after that also triggered this api tables/{{table_name}}/resumeConsumption?consumeFrom=largest still consuming segments are not spawning. Please help to resolve this

Try the controller endpoint (

/debug/tables/{tableName}

) for debug logs. If there is nothing, you can look at pinot server logs to check what's happening during Kafka consumer creation.

[ { "tableName": "append_aaa_events_REALTIME", "numSegments": 23354, "numServers": 36, "numBrokers": 3, "segmentDebugInfos": [], "serverDebugInfos": [], "brokerDebugInfos": [], "ingestionStatus": { "ingestionState": "HEALTHY", "errorMessage": "" }, "tableSize": { "reportedSize": "11 TB", "estimatedSize": "11 TB" } } ]

Please check the controller log after triggering the API and see the status there

suggest to use controller to first pause table, after 5 mins, resume consumption with latest offset.