Hello, I’ve set replicas per partition to 1 for ll...
# troubleshooting
t
Hello, I’ve set replicas per partition to 1 for llc streaming ingestion. Whenever pinot fails to ingest records from kafka, (in our case it is schema registry restarts) it throws error and set the segment state to offline. Even after the issue is resolved, I don’t see the consumption being resumed/retried. I tried triggering the reload of the offline segments but it did not have any affect. What else can I do to resume consumption?
Recreating the table solves the issue on non-prod environments. But I wonder how to tackle this issue in production environments.
c
@Tanmay Movva I believe this issue is tracking this problem: https://github.com/apache/incubator-pinot/issues/6308
n
For consuming segments that get marked offline, there is a periodic task that will correct them. Once it runs, the consumption will be restored. That task runs every hour by default. You can increase the frequency via controller configs if you want it to run sooner
t
@Neha Pawar I see that the consumption has restored but a new consuming segment is created instead of the offline segment regaining the consuming state. Is this expected?