This message was deleted.
# general
s
This message was deleted.
j
For appending data the segment granularity of the must match. But within a datasource different time chunks can be set up with different segment granularity. So, for example, if you leave your ingestion at DAY level and never ingest old (i.e. yesterday's) data, then on the first of each month you can compact/reindex all of last month's data from DAY to MONTH. If you want to change your current ingestion from DAY to MONTH granularity, then you first have to compact/reindex the current month of data from DAY to MONTH so it will match the MONTH granularity of the ingestion task.
a
I will not do old data.... I am actually doing ingestion using Kafka stream in real time. Assume I do modify the data-source spec for tomorrow 02-June onwards. then in this case will it work ?
I mean old data in day wise segmented and from tomorrow onwards in month wise segment !? or this is not the case we can do !
I mean druid require entire data-source to reindexed to month wise ?!
before modifying the spec !
j
You just need to make sure there are no DAY segments in the month of June 🙂 if there is anything existing, then reindex it to MONTH, even though it may be a small segment, doesn't matter, the granularity just has to match what's in your supervisor spec.
Btw the downsize of this is that you will not easily be able to compact the current month time chunk so it will become very fragmented as the ingestion tasks cycle and create new segments every (hour?). This could affect performance if the fragmentation is extreme.
a
Ok