Hi ALl, i am working on backup and restore task, i took a backup of segments, modified or altered value of one of the column inside the segments. Recreated the segment from csv file. Post to that, i am uploading the segments via REST API -> ../segments?tableName=data_record&tableType=OFFLINE .... The API results with "SUCCESS" error message. Post to that , in Apache pinot GUI when i select the table, it shows totaldocs and that is also shows proper value. Post to this verification, when i query select * from table_name or select * from table_name limit 100. I am getting no records found. Can you correct me or share me the exact steps to ensure that the import of data is proper and there is no data loss and i am able to view the data....

I am unclear on what problem you are trying to solve? Is this about DR?

Yes. To support DR, I have a script that takes the DEEPSTORE SEGMENTs, and below are the steps. 1. Read the deepstore segments 2. Read each docs, modify the column values if required. - Yes we will modify "id" column with offset addition say 10000. 3. Once the rows are created, we will recreate modified segments again. 4. Copy the modified segments to target server where DR Pinot is running. .We use REST API -> ../segments?tableName=data_record&tableType=OFFLINE to upload the segments to the target PINOT server. 5. REST API shows successful and totaldocs count is also matched. 6. In the target PINOT SQL, select * from tablename shows empty and reports no rows found. 7. We would like to know what could be the issue and how to debug it? Note: To ensure that the workflow fine, we removed step #2, #3 and used the segment as it is without modified... As a result, the API is success. But while query the DB, it shows it is empty no records found.... The solution expected here is how to export the segment and upload the segments to target server without any issues or loss of data....

I wouldn't do DR this way, you are basically re-indexing all the data. Here are some of the options for DR: • Multi-region deployment that can be active-active or active-passive. • Automation to save and restore ZK snapshot. As long as there is data in deep-store and metadata of ZK is snapshotted, you can have automation to resurrect the Pinot cluster from these two.