Hi Team, I have an upsert table with uniq key (STRING, 64 chars). Ingest rate ~4–5k events/sec. Quick math for the raw ID payload: 64 bytes per row 86,400 sec/day 4–5k rows/sec → ~345–432M rows/day • ⇒ ~22–28 GB/day (≈26 GB/day avg) only for raw uniqId bytes My question: how do you roughly estimate RAM usage for upsert/dedup in Pinot (PK → docId map, validDocIds, etc.) given: • key type: 64-char STRING • high cardinality • hot window target: 3 days or more if it passible Any rule-of-thumb or example numbers would help

Generally speaking you end up with may be 100M max keys per server (64 GB) as you want to have enough heap for other things. YMMV

We've observed 300 bytes per primary key as our baseline