Hi team, I’ve a question about segment build process with startree index. If a segment build failed, pinot would stop consuming that kafka partition. Through calling /consumingSegmentsInfo api, this segment shows not_consuming. It will continue consuming by calling /segments/{tableNameWithType}/{segmentName}/reset api. But I’m a little confusion about the offset. I thought it would from the offset where it failed building that segment. But it may not seem so. Based on my test, through checking server log, a segment building failed on Jul 17, and it stopped consuming data since then. I reset this segment at 8:00 am on Jul 18, then new data is ingested since this time. Data earlier this time isn’t ingested.

It should consume from the previous checkpointed offset. The

smallest

setting in table config is not for this scenario but bootstrapping of the table.

The previous checkpointed offset, is it the same as the one where the segment failed built?

The one failed is not checkpointed. The last successful segment commit is

I thought so. But just could not explain data earlier this time wasn’t ingested.