This message was deleted.

Is this real-time ingestion?

yes @Vijay Narayanan we are trying for real time with high ingestion

With real-time ingestion it will take sometime for the existing tasks to finish publishing and only then will the new spec take effect. Have the current tasks shutdown and new tasks started?

yes i suspended the datasource till all the realtime segments get pushed and then resumed the ingestion

Check the payload of the tasks. Does it reflect the changes you made?

yes it has been updated,

@Bipin Kumar Tiwari with realtime ingestion, segments will be published at the

taskDuration

specified in the config, or according to the conditions described in

intermediateHandoffPeriod

https://druid.apache.org/docs/latest/development/extensions-core/kafka-supervisor-reference.html#kafkasupervisorioconfig

@Suneet Saldanha i have changed the task duration to 1 day, the difference that is see in segment sections(router console) for that particular data source has start and end as 1 hour duration for realtime segment of late data and current day realtime segment it has of 6 hour duration

can you paste a redacted version of your spec and a screenshots of the segments tab in the web console. It will be easier to tell what is going on if we can see the config

Hey @Suneet Saldanha @Vijay Narayanan, Thanks for the input, after stopping the ingestion and doing the compaction on that segment for the day, i was remove that issue and now segments get created normally for day as expected