This message was deleted.

MSQ SQL ingestion makes this fairly easy: https://docs.imply.io/latest/druid/multi-stage-query/ You would basically issue a SQL statement like:

INSERT into datasource2 
select * from datasource1 
 where __time >= CURRENT_TIMESTAMP - INTERVAL '14' DAY 
   and __time < CURRENT_TIMESTAMP - INTERVAL '7' DAY 
PARTITIONED BY ALL

And if you need to replace data in datasource2 then you would use the REPLACE/OVERWRITE syntax I'm not sure I have the syntax exact, but hopefully this gives you enough to work with.

The web-console also provides a guided experience to help with generating the re-index spec to load data from one datasource into another (see the

Reindex from Druid

tile)

@Suneet Saldanha but this does not give an option to reindex periodically correct

Job scheduler is a feature being discussed, but not in the product right now. So for now you would have to submit jobs through the API and coordinate the scheduling your own (e.g. cron) Fyi auto-compaction runs on a schedule, but that is specifically for single datasource compaction jobs, I don't know that you could jury-rig that to do something else (am guessing not).

This information helps. Thanks @John Kowtko