Hello Pinot friends. Has anyone had issues with du...
# general
s
Hello Pinot friends. Has anyone had issues with dupes when using the roundBucketTimePeriod configuration on the RealTimeToOfflineSegmentsTask? Our realtime data is in milliseconds and we want to trim that to hours in the offline table. We are ingesting an event_id field so I can tell when querying the table I get dupes for a period of time. It seems to be related to us trimming the time to an hour. When I remove the roundBucketTimePeriod I no longer get dupes. As if the controller finally recognizes the correct time boundary.
p
if you query and include $segmentName are the dupes in the same segment or between realtime and offline?
s
They are between realtime and offline