This message was deleted.
# general
s
This message was deleted.
j
I believe this is a direct (off-heap) memory allocation issue ... this is used by merge operations. Can you share your ingestion spec? There are a few parameters that could be relevant here.
s
Too big to show here so I did a gist: https://gist.github.com/RelativisticMechanic/16d4be75fa57a016d5760f4577d30f49 There are a lot of columns with doubles, could that be an issue?
j
In the tuningConfig try setting MaxColumnsToMerge to something small like 4 to see if that helps. ref: https://druid.apache.org/docs/latest/ingestion/native-batch.html#tuningconfig
s
Also how are your peon JVMs configured?
druid.indexer.runner.javaOptsArray
in the Middle Manager config? That's where you specify the heap and direct memory limits for the peon tasks that execute the ingestion, if it is not specified it will take the JVM settings for the middle manager.