https://pinot.apache.org/ logo
#troubleshooting
Title
# troubleshooting
a

Awadesh Kumar

03/09/2022, 1:00 PM
Hi all, I deleted all the segments from a pinot table using below endpoint:
http://{base_url}/segments/trips?type=REALTIME&retention=0d
Now Pinot table has stopped receiving the data from the kafka topic. We didn't change anything in table configuration. Any possible reason for this?
👀 1
m

Mayank

03/09/2022, 2:23 PM
What was the reason for deleting all data? And where do you want Pinot to start consuming from? As a quick fix you can try to delete and recreate the table.
Cc: @Neha Pawar @Navina perhaps we should not allow deleting of consuming segments by default?
a

Awadesh Kumar

03/09/2022, 2:59 PM
Hi Mayank, We made
singleValueField
as
false
for a field of the table. There were negative values(default long value) for this field in the table which is invalid in our use case. So for this clean-up we deleted all the segments as it was testing data. After the segments deletion, data is not getting consumed in the table. I know it wouldn't be a recommended approach for a live system. Is there any better way to update/delete specific records from query console.
m

Mayank

03/09/2022, 3:24 PM
What you are describing is a rebootstrap case. For offline tables you can simply push data and overwrite. For real-time table, there is no push support yet (coming soon), so for such backward incompatible changes this is the only way I can think of
a

Awadesh Kumar

03/09/2022, 3:28 PM
ok thanks for the update. Any reason for data is not getting consumed in the table? Any fix except recreating tables?
n

Neha Pawar

03/09/2022, 4:01 PM
The last segment of each position is in consuming state.If consuming segment is deleted, the consumption stops. We have an open issue to add recovery mechanism for this.
a

Awadesh Kumar

03/10/2022, 3:57 AM
Got it, Thank you