This message was deleted Apache Druid #troubleshooting

Join Slack

This message was deleted.

# troubleshooting

Slackbot

03/28/2023, 9:39 AM

This message was deleted.

Gian Merlino

03/28/2023, 9:54 AM

A few ideas 1) You can raise your direct memory limit for tasks by adjusting

-XX:MaxDirectMemorySize

druid.indexer.runner.javaOpts

middleManager/runtime.properties

(whichever conf directory you're using) 2) you can use the new

bin/start-druid

script in Druid 25+ which sizes direct memory automatically; it may be a better fit for you 3) if you're using native batch ingestion, you can use SQL-based ingestion (https://druid.apache.org/docs/latest/multi-stage-query/index.html) which has greatly improved memory management and is less likely to run into various kinds of OutOfMemoryErrors

Saurav Chirania

03/28/2023, 9:56 AM

Thanks, after doing the change you mentioned in point (1), I will have to restart druid?

Gian Merlino

03/28/2023, 10:00 AM

yep you would

Saurav Chirania

03/28/2023, 10:00 AM

ack, thanks

Saurav Chirania

04/18/2023, 12:39 PM

Hi, what is the default value of

-XX:MaxDirectMemorySize

when we are not passing any value?

Gian Merlino

04/20/2023, 4:39 AM

if you're using the

bin/start-druid

script it's set automatically to a reasonable value

Gian Merlino

04/20/2023, 4:39 AM

you can see it in the messages printed when the script starts, like:

Copy code

[Mon Apr  3 14:20:49 2023] Running command[router]: bin/run-druid router /Users/gian/Documents/implytesting/apache-druid-26.0.0-SNAPSHOT/conf/druid/auto '-Xms256m -Xmx256m -XX:MaxDirectMemorySize=128m'

Gian Merlino

04/20/2023, 4:39 AM

if you're not using

bin/start-druid

then default is same as

-Xmx

(this is a JVM thing)

2 Views

Open in Slack

Previous Next