This message was deleted.

Normally, an ingestion task will publish a segment and then wait for the segment to make its way into a historical before exiting. The completion timeout is the maximum time that the task will wait for the handoff to complete. I few reasons why this might not complete: • retention rules do not assign the new segment to a historical • coordinator too busy - usually related to too many segments • zookeeper - the segment announcement mechanism failing - only if you are using zookeeper

yea. just for some. almost 2-3 tasks each hours. do you recommend to tweak

completion timeout

or

task duration

?

and all data is recent or does it have late arriving messages?

we might have few late msg but very negligible

just to verify, what is your retention rule? it could be that a single late message that is older than the retention rule will create a segment that is never assigned to a Historical.

we dont have any lateMessageRejectionPeriodConfigured for this datasources. and we dont have any retention configured yet

I mean that if your retention rule for the datasource is say 1 week and a message arrives today that is older than 1 week, then the task will create a segment for that older timestamp and publish it, but it will not get picked up by the coordinator/historical because it is older than the retention.

It make senses. Ill check and apply the retention and lateMessageRejectionPeriod Configured. Thanks