Hey quick question, when the data is moved to deep storage (after the retention period) is it still queryable or do we need to reload that back like a batch ingestion job?
r
Rong R
10/14/2022, 2:06 PM
i think there's a misconception here: deep store is actually used for backup and restore segments e.g.
• segments are not being transfer over after retention period, they are uploaded to deep store right after segment generation
• other replica segments on different server downloads them from deep store
for more info see: https://docs.pinot.apache.org/basics/components/deep-store
r
Rohit Anilkumar
10/14/2022, 2:17 PM
Oh so in that case, after retention period, its just removed from the data nodes?
l
Luis Fernandez
10/14/2022, 2:51 PM
they are not longer queryable once the retention hits
r
Rong R
10/14/2022, 3:05 PM
there's several concept of "retention", do you know which config key you are referring to?
r
Rohit Anilkumar
10/14/2022, 7:00 PM
Let me read this. Maybe that will clear my doubt. Thanks