Hi Team,
Background:
I am evaluating pinot for replacement of existing key value store. I have use-case where only 2% of the records get updated once inserted.
I was reading about Stream Ingestion with Upsert (
https://docs.pinot.apache.org/basics/data-import/upsert), but this is only for real-time ingestion and one the segment is completed its stored in deep store(Azure blob, HDFS etc).
Question:
If I have a update request for a record which exist in deep store, can pinot pull that segment from deep store and perform upsert on it. If not any workaround ?
One idea I was having is to make id as foreign key and use some GUID as primary key which will allow me to insert duplicates against id. And during read I will add where id = <my_id>. Not sure if this can work and have any known complication.
Thanks,
Danish