This message was deleted.
# general
s
This message was deleted.
g
For
index_parallel
this is controlled by split hint: https://druid.apache.org/docs/latest/ingestion/native-batch/#split-hint-spec
by default, it's 1 GB per task so you would not get multiple tasks for 400 MB. but you can change it
I personally recommend MSQ, where this is instead controlled by
maxNumTasks
and
taskAssignment
, see https://druid.apache.org/docs/latest/multi-stage-query/reference#context-parameters
The logic for how many tasks would be launched is described on that page
u
Thanks @Gian Merlino! This task was to compare the performance between the index_parallel and MSQE operations. Hence we needed the task to spin up similar number of workers.