Hello, is it possible to reset pinots state that remembers where in kafka ingestion should continue from? On one of our clusters, we deleted and recreated kafka from where pinot was consuming from. Now pinot is not consuming anything (presumably because it is looking for a high offset value which is not present anymore). We dont want to lose previously ingested data.

One workaround would be to move those segments to offline table and start reatlime from scratch. Also, not sure, if there is a way in Kafka to override the offset that may be an easier way

does changing segment.realtime.startOffset = "0" in zookeeper work? i may have tried that a long time ago

Theoretically, it should work, but I usually try out other solutions before touching ZK

simple script to update zookeeper worked