Couple open questions about upsert tables and the metadataTTL config: 1. If no

metadataTTL

is set, do primary keys then expire and free up memroy at the end of the data TTL (set by the

segmentsConfig

retention settings)? Or do primary keys live forever in this case? 2. Should

metadataTTL

always be set to longer than the segment config retention setting? I imagine if you went to update a record that fell out of the

metadataTTL

, but the old segment data was still around, you could have duplicate data at that point?

1. Segment retention will remove keys that will be obsolete after removing the segment. 2. If TTL < segment retention, then you need to guarantee that PKs are not be reused, otherwise, yes, you will get duplicates.