When we enable upsert on a realtime table, and the retention is 1 day, even after 5 days, I am receiving primary key count equal to the data of the last 5 days. When does Pinot delete the upsert keys from the heap once the corresponding segments are deleted due to retention? If it doesnt, does it mean that my memory usage will keep increasing as time goes by?

Can you check how many segments are created? A segment will be deleted if all the records past the retention.