Hello, we’re using

RealtimeToOfflineSegmentsTask

for our hybrid table and we ran into a problem in our dev environment. We have time gaps in our data ingest and when it’s larger than the bucket time period, the minion task runs into an error of creating empty segment. After exception the minion fails to update the watermark, thus we’re ending up with a stuck task (trying to create an empty segment over an over again for this specific period). While it’s unlikely to run into this empty segment problem in production, we’re wondering what’s the recommended way to overcome this issue in a dev environment?

I’m also trying this feature in our test environment. Watermark is stored as ZK metadata. You can try updating the metadata manually. Try setting it to the next available segment start time stamp.

thanks for the suggestion Laxman! @Tamás Nádudvari do you mind creating a github issue for this? will see if it can be fixed easily