i am replying to the last message you sent so we can start a thread

this is the real key here:

2022-09-12T11:07:39,921 ERROR [forking-task-runner-8] org.apache.druid.indexing.overlord.ForkingTaskRunner - Process exited with code[137] for task: index_kafka_nbr_8322bd4a68e5423_pclejmmh

exit code 137 is signal 9 a.k.a. SIGKILL — the task JVM was killed abruptly

generally most common reason for this is it tried to use too much memory — so the thing to do would be verify your memory settings & make sure we aren't exceeding limits

most relevant settings are here

druid_indexer_runner_javaOptsArray: '["-server", "-Xmx3g", "-Xms256m", "-XX:MaxDirectMemorySize=3g", "-Duser.timezone=UTC", "-Dfile.encoding=UTF-8", "-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager"]'

and

druid_indexer_fork_property_druid_processing_buffer_sizeBytes: 256MiB

and

druid_worker_capacity: "16"

and

druid_processing_numThreads: "6"
  druid_processing_numMergeBuffers: "2"

https://druid.apache.org/docs/latest/operations/basic-cluster-tuning.html has some details about how to interpret these parameters but simple version is: • each mm will run up to 16 tasks (

druid.worker.capacity

) • each task will use up to 3 GB of heap (

-Xmx3g

) and ~2.5 GB offheap (6 threads + 2 merge buffers; times 256MB; plus a little extra) • so if all 16 tasks are running, that's potentially 88 GB of memory usage on the mm do you have that much memory available on the machine?

generally most common reason for this is it tried to use too much memory

btw, you can verify this in `dmesg`: it will say something about the oom-killer killing a

java

process

@Gian Merlino you were completely right! Thanks of u, it is solved! Thank u so much!