Hi Team, i've created realtime-offline flow. i've...
# general
v
Hi Team, i've created realtime-offline flow. i've pushed some data 1st day and it got converted to offline 2nd day. and than i pushed more data to but its not pushing to offline even i can see 6 segments are completed but its not pushing to offline table. can somebody help me on this? Thanks.
m
@Haitao Zhang @Seunghyun
s
@vishal sharma realtime to offline will try to move data to offline once ‘bufferTimePeriod’ is past. What’s your configuration for that? https://docs.pinot.apache.org/operators/operating-pinot/pinot-managed-offline-flows
v
hi @Seunghyun
bufferTimePeriod already passed but not able to see any segment move to offline
s
@vishal Can you post your
RealtimeToOffline
configuration? Also, if you have the access to the controller side log, Please grep for
RealtimeToOfflineSegmentsTaskGenerator
and post to us if it provides some reasoning for skipping the segmtns or if there's any
warn
or
error
logs