This message was deleted.
# generals
Slackbot
06/01/2023, 2:11 PMThis message was deleted.
j
John Kowtko
06/01/2023, 2:36 PMFor appending data the segment granularity of the must match.
But within a datasource different time chunks can be set up with different segment granularity.
So, for example, if you leave your ingestion at DAY level and never ingest old (i.e. yesterday's) data, then on the first of each month you can compact/reindex all of last month's data from DAY to MONTH.
If you want to change your current ingestion from DAY to MONTH granularity, then you first have to compact/reindex the current month of data from DAY to MONTH so it will match the MONTH granularity of the ingestion task.
a
Ashok Kumar Ragupathi
06/01/2023, 2:39 PMI will not do old data.... I am actually doing ingestion using Kafka stream in real time. Assume I do modify the data-source spec for tomorrow 02-June onwards. then in this case will it work ?
Ashok Kumar Ragupathi
06/01/2023, 2:40 PMI mean old data in day wise segmented and from tomorrow onwards in month wise segment !? or this is not the case we can do !
Ashok Kumar Ragupathi
06/01/2023, 2:40 PMI mean druid require entire data-source to reindexed to month wise ?!
Ashok Kumar Ragupathi
06/01/2023, 2:41 PMbefore modifying the spec !
j
John Kowtko
06/01/2023, 2:53 PMYou just need to make sure there are no DAY segments in the month of June 🙂 if there is anything existing, then reindex it to MONTH, even though it may be a small segment, doesn't matter, the granularity just has to match what's in your supervisor spec.
John Kowtko
06/01/2023, 2:54 PMBtw the downsize of this is that you will not easily be able to compact the current month time chunk so it will become very fragmented as the ingestion tasks cycle and create new segments every (hour?). This could affect performance if the fragmentation is extreme.
a
Ashok Kumar Ragupathi
06/01/2023, 2:57 PMOk