https://pinot.apache.org/ logo
p

Phúc Huỳnh

04/14/2021, 3:14 AM
Hello guys, i’m have some issue with RealtimeToOfflineSegments task. So i create 2 hybrids table from same days. 1 for QC env and 1 for UAT env. • In the tables managements. It’s seem
RealtimetoOfflines
task in QC env has been stop. but i cannot find any errors log. • Another the question is the realtime segments will be remove after convert to Offline Segments Table ?
x

Xiang Fu

04/14/2021, 4:18 AM
have you checked logs in minion?
p

Phúc Huỳnh

04/14/2021, 4:20 AM
here is the logs of minion pods.
sometime, my zookeepers pods rollout because vm preemptible in gcloud
x

Xiang Fu

04/14/2021, 4:21 AM
ic, maybe have 3 pinot-zookeepers for HA?
p

Phúc Huỳnh

04/14/2021, 4:22 AM
i already setup 3 zookeepers for HA
x

Xiang Fu

04/14/2021, 4:26 AM
ok
so if there is no task logs on minion, then it means the minion tasks are not scheduled
can you check controller log and look for
RealtimetoOffline
log?
p

Phúc Huỳnh

04/14/2021, 4:35 AM
only 1 logs grep by “realtime”
x

Xiang Fu

04/14/2021, 5:17 AM
hmm, is this task scheduled? can you check minion apis through controller swagger ui?
p

Phúc Huỳnh

04/14/2021, 6:28 AM
Minion apis is Task in swagger ?
Could u answer question 2:
Copy code
Another the question is the realtime segments will be remove after convert to Offline Segments Table ?
x

Xiang Fu

04/14/2021, 7:10 AM
I don’t think so. This task requires a hybrid table and it will create segments and push to offline table. You can set a fairly low retention for realtime table but longer for offline table .
have you found the issue ?
p

Phúc Huỳnh

04/15/2021, 4:05 AM
hmm, i’m try but cannot find related log
x

Xiang Fu

04/15/2021, 4:19 AM
i see, can you try to tail the controller log and call the
/tasks/schedule
POST endpoint using swagger ui?