This message was deleted.

When rollups are enabled (I believe this setting is per segment) then the requred metadata is stored with the field to enable further rollups/merging.

the metadata for monthly segment would know its seen X and Y before? I suppose this is something that is testable too.

Oh, no. When you ingest data you ingest in either append or replace mode. If append mode then it is assumed the metrics will be added to. If replace, then you are replacing all data within a segment of time.

i suppose the scenario I'm suggesting is that the second time Druid sees X and Y, it will reingest it into "NONE" granularity at first, and then later it will try to rollup to "MONTHLY" where it has previously rolled the first time it saw X and Y

I am not sure of the rules around ingesting data with different levels of query granularity ... my guess here is that Druid will not know there is replacement data ... once the data is rolled up you cannot "un-roll" it to find individual records to replace. As far as the merging process, here is the doc page section on query granularity handling during compaction: https://druid.apache.org/docs/latest/data-management/compaction.html#query-granularity-handling ... and two sections down: https://druid.apache.org/docs/latest/data-management/compaction.html#rollup

thanks, your guess is similar to my suspicion.

For Druid the preferred approach would be: • keep the data at the more granular level • use the "latest() aggregate function to always grab the most recent value for X and Y during query time • optionally create a summary table (datasource) that you regenerate periodically using that "latest" data query. The only other alternative I know of would be to orchestrate updates to the original data ... that is a batch operation and takes a bit of work, but can be done if you can't use one of the other approaches.