Hello, is there any way to delete the tableconfig without deleting segments and create the same table with the same segments from the disk for realtime? I happen to execute a wrong clusterconfig rest call and broke the broker UI . I tried updating it again but no luck. So planning to recreate the tableconfig without losing data.

No that I am aware of (for realtime). What exactly broke in the broker?

The Cluster Manager section stopped loading. I can see some js error in the javascript console. All ingestion and searches are working fine.

Restart controller?

not helping, seems like the bad config is somehow coming from zookeeper.

Yeah, can't think of an easy fix

will do the hard way.

If you know which zk record is breaking, you may manually fix it via the zookeeper browser (I assume this is the hard way)

Hard way was delete and recreate table, I think

For offline segments, you an re-load them from the Deleted_Segments folder in your PinotFS after recreating the table. For realtime tables, you can consume from earliest available row in the stream. Note that when it starts to consume feverishly, query performance will suffer. Also, if the data has already been retained out from the underlying stream, then it is gone forever, sorry.

@Jackie @Mayank correct delete and recreate unfortunately.

Note that this restore option is only for offline tables. You can check the root directory of your PinotFS (underwhich there are table directories), and there should be a folder for deleted segments

ohh ok , Is it possible to copy segments created by realtime manually to Deleted_Segments folder then create a offline table from it. Then update the offline table config to realtime?

Well, the realtime segments are also available in Deleted_Segments folder. But they cannot be uploaded into the table in any easy manner. If it is a production issue ($$$ at stake) , you can probably recover, but it needs a lot of manual work.

Actually my stream retention matches pinot retention so there is no data loss. However what is the best way to do proper backup and restore ? is S3 or EFS based deep store a good way to go?