https://pinot.apache.org/ logo
#general
Title
# general
n

Noureddine

08/06/2021, 2:10 PM
Hi guys, I am looking for a way to delete a record in a pinot table. I am aware that Pinot doesn't support this type of operation, but in our use case we have deletes once in a while so I am wondering if there is a Hack on how to delete a record without rebuilding the whole Table ?
m

Mayank

08/06/2021, 2:49 PM
Is this for purging for GDPR ?
Typically, you would replace the segment that contains the record you want to delete
n

Noureddine

08/06/2021, 3:35 PM
ok thanks
k

Ken Krugler

08/06/2021, 3:49 PM
@User I started writing a response here:
Assuming every record has a unique id, then I can think of a hack (similar to how Lucene handles deletes) where you add a
deleted
integer field, and group by the unique id, include
max(deleted)
in the result, and the client ignores any results with that value > 0. So then you can add a record with the same unique id and
deleted = 1
, and it gets ignored. Yes, it’s a horrible hack 🙂
👍 1
Actually you do this with existing upsert support, plus a “deleted” flag, and just always filter out deleted entries. See https://docs.pinot.apache.org/basics/data-import/upsert
n

Noureddine

08/08/2021, 1:39 PM
thanks for you answer. I thought of this solution too, but I am just wondering by how much is it gonna slow my requests.so I definitely need to benchmark it before implementing it .