Hi, Pinot Teams. Nice to meet you. I'm sorry for asking a question as soon as I joined the channel. Explain : I am testing the recipe page below. (merge-segments-realtime) https://dev.startree.ai/docs/pinot/recipes/merge-segments-realtime MergeRollupTask was not executed, so I triggered it using the curl command as shown below.

$ curl -X POST "<http://localhost:9000/tasks/schedule?taskType=MergeRollupTask&tableName=merge-small-seg_REALTIME>"  -H "accept: application/json"                                                                                                                                                                                    
{"MergeRollupTask":"Task_MergeRollupTask_7b816fb6-fb78-425a-8ab0-27a7427d5728_1712031213827"}                                                                                    
$ curl -X POST "<http://localhost:9000/tasks/schedule?taskType=MergeRollupTask&tableName=merge-small-seg_REALTIME>"  -H "accept: application/json"
{"MergeRollupTask":"Task_MergeRollupTask_fa634216-130d-4db3-8a1f-cd696868fe91_1712031420296"}

Question : 1. When curl is called twice as shown below, the value of 'Num Tasks in Queue' in the UI is displayed as 2. MergerRollupTask seems to be a single task executed over time, but why are multiple tasks created? For example, if one task is created, the other one is not needed, so it may be possible to prevent it from being created. 2. Anyway, I would like to know what it means to have two tasks. Could you please explain the difference from when there is only one? 3. After MergeRollUp, I checked that the segment file was located in the data dir of pinot-server. I think the Minion Process performs the merge. By what process is the data located in

pinot-server

? And in the log file of the minion process, I can see

"datadir" : "/var/pinot/minion/data"

. What is datadir used for? Thank you so much. Really so much :)

Cc @Hubert Dulay

If there is a logical problem or strangeness with my question, I would appreciate it if you could point it out.

Hi, can you make sure you pull the latest changes to this repo? This recipe was updated recently. Thanks

Yes, thank you so much. I pulled it from the repo. My question does not seem to be related to the repo being updated to the latest version. Could you please check the contents of the 'question' again rather than the part that is not running? Thank you. It will be of great help in applying it to the product environment.

I believe the dataDir is where the Minion puts its data that it is working on when running. It gets cleaned when the minion stops. @Mayank to confirm

I will take a look and get back with answers soon, thanks!

Thank you so much for the quick reply. Thank you both, @Hubert Dulay, @Aishik. The weekend is coming. Have a happy time with your lover and family.