Hi Team, Background: I am evaluating pinot for re...
# general
s
Hi Team, Background: I am evaluating pinot for replacement of existing key value store. I have use-case where only 2% of the records get updated once inserted. I was reading about Stream Ingestion with Upsert (https://docs.pinot.apache.org/basics/data-import/upsert), but this is only for real-time ingestion and one the segment is completed its stored in deep store(Azure blob, HDFS etc). Question: If I have a update request for a record which exist in deep store, can pinot pull that segment from deep store and perform upsert on it. If not any workaround ? One idea I was having is to make id as foreign key and use some GUID as primary key which will allow me to insert duplicates against id. And during read I will add where id = <my_id>. Not sure if this can work and have any known complication. Thanks, Danish