https://pinot.apache.org/ logo
#general
Title
# general
y

Yugandhar

08/24/2020, 4:40 PM
Does it support any exactly once guarantees like Druid ?
m

Mayank

08/24/2020, 4:43 PM
If so, it is unclear to me if exactly once implies de-duping duplicate Kafka messages (even without failures). Or does it mean avoiding duplicates when recovering from failure? If latter, Pinot has always had that. If former, Pinot does not de-dup duplicate events during ingestion, atm.
y

Yugandhar

08/24/2020, 5:14 PM
I mean does it process every "kafka offset id" only once ?
m

Mayank

08/24/2020, 5:18 PM
It may reprocess to recover from failure, however, the every incoming event is indexed and stored only once (your queries won't have duplicate events because of Pinot).
y

Yugandhar

08/24/2020, 5:49 PM
ok, thanks
m

Mayank

08/24/2020, 5:50 PM
👍