Is there way to set segmentGranularity by data int...
# general
y
Is there way to set segmentGranularity by data interval automatically ? Also, I want to apply for segmentGranularity as day on recent data and segmentGranularity to month for old data. it would be great to apply in auto compaction.
Should I set to skip offset to 90D and trigger manual compaction on the interval between 90D before and now?
j
you can edit the auto-compaction spec to specify a granularitySpec ... this is a standard way to raise segmentGranularity and/or queryGranularity.
y
I meant by period. [Now-24 , Now]. realtime task [D-90 , Now-24 ]. segmentGranularity of 1D [ ~ , D-90 ). segmentGranularity of 1 Month
j
Hi Youngsol, you can't specify multiple granularities in a single spec ... you can only have compaction/reindex jobs change the granularity for specific time intervals they operate on. There is the notion of multi-stage auto-compaction that may be added to the product in the future as part of data lifecycle feature development ... but for now the common practice is to ingest at one granularity, and then manually run compaction/reindex jobs at the appropriate time to convert aging time intervals to the new desired granularity. I know people are doing what you are describing ... all using these batch jobs.
y
Thanks Kowtko.