Hello everyone 👋 I have recently started experimenting on Pinot in order to have run in production. I have a question about disaster recovery in case metrics are wrong for some reason and we'd like to recompute them. In the case of a kafka realtime table, is there a way to pause the Kafka Consumers, truncate the table and then restart consumption from scratch? Or should I just consider a blueGreen way here (create new table with correct metrics and redirect the queries to it when it's up to date)?

How long do you keep the data in kafka? What is your plan to fix bad data when it’s past retention time

We have infinite retention. The source of the data is Postgres, we have a debezium that is streaming changes into kafka. then we have flink jobs that compute metrics from these events and produce into another topic. this last topic should end up in pinot

In that case, both should possible, truncating and blue green. Truncating won’t require client side change to change the table name.