I have a weird byte buffer issue I've never ran into before. My peons are using all of the default values but I have one ingestion that fails to allocate byte buffer memory:

2026-10-09T17:50:02,440 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.
java.lang.RuntimeException: java.lang.OutOfMemoryError: Cannot reserve 65536 bytes of direct buffer memory (allocated: 2621378747, limit: 2621440000)
	at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:636)
	at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:293)
	at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.runTask(SeekableStreamIndexTask.java:179)
	at org.apache.druid.indexing.common.task.AbstractTask.run(AbstractTask.java:175)
	at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:478)
	at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:450)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.OutOfMemoryError: Cannot reserve 65536 bytes of direct buffer memory (allocated: 2621378747, limit: 2621440000)
	at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:121)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332)
	at org.apache.druid.segment.data.CompressionStrategy$LZ4Compressor.allocateInBuffer(CompressionStrategy.java:338)
	at org.apache.druid.segment.data.CompressedVSizeColumnarIntsSerializer.<init>(CompressedVSizeColumnarIntsSerializer.java:132)
	at org.apache.druid.segment.data.CompressedVSizeColumnarIntsSerializer.<init>(CompressedVSizeColumnarIntsSerializer.java:95)
	at org.apache.druid.segment.data.CompressedVSizeColumnarIntsSerializer.create(CompressedVSizeColumnarIntsSerializer.java:63)
	at org.apache.druid.segment.nested.GlobalDictionaryEncodedFieldColumnWriter.openColumnSerializer(GlobalDictionaryEncodedFieldColumnWriter.java:307)
	at org.apache.druid.segment.nested.GlobalDictionaryEncodedFieldColumnWriter.writeTo(GlobalDictionaryEncodedFieldColumnWriter.java:244)
	at org.apache.druid.segment.nested.NestedDataColumnSerializer.writeTo(NestedDataColumnSerializer.java:442)
	at org.apache.druid.segment.column.ColumnDescriptor.writeTo(ColumnDescriptor.java:95)
	at org.apache.druid.segment.IndexMergerV9.makeColumn(IndexMergerV9.java:322)
	at org.apache.druid.segment.IndexMergerV9.makeIndexFiles(IndexMergerV9.java:246)
	at org.apache.druid.segment.IndexMergerBase.merge(IndexMergerBase.java:464)
	at org.apache.druid.segment.IndexMergerBase.multiphaseMerge(IndexMergerBase.java:257)
	at org.apache.druid.segment.IndexMergerBase.persist(IndexMergerBase.java:154)
	at org.apache.druid.segment.IndexMerger.persist(IndexMerger.java:237)
	at org.apache.druid.segment.realtime.appenderator.StreamAppenderator.persistHydrant(StreamAppenderator.java:1688)
	at org.apache.druid.segment.realtime.appenderator.StreamAppenderator$2.call(StreamAppenderator.java:692)
	... 6 more

I see this in the logs as well:

2026-10-09T17:48:17,987 INFO [main] org.apache.druid.query.DruidProcessingConfig - Detected max direct memory size of [2,621,440,000] bytes
2026-10-09T17:48:17,988 INFO [main] org.apache.druid.query.DruidProcessingConfig - Auto sizing buffers to [655,360,000] bytes each for [1] processing and [2] merge buffers. If you run out of direct memory, you may need to set these parameters explicitly using the guidelines at <https://druid.apache.org/docs/latest/operations/basic-cluster-tuning.html#processing-threads-buffers>.
2026-10-09T17:48:18,130 INFO [kafka-producer-network-thread | producer-1] org.apache.kafka.clients.Metadata - [Producer clientId=producer-1] Cluster ID: redpanda.f9befd29-9798-4536-a70b-ecc579609964
2026-10-09T17:48:18,594 WARN [main] org.apache.druid.k8s.discovery.K8sDiscoveryConfig - IF coordinator pods run in multiple namespaces, then you MUST provide coordinatorLeaderElectionConfigMapNamespace
2026-10-09T17:48:18,594 WARN [main] org.apache.druid.k8s.discovery.K8sDiscoveryConfig - IF overlord pods run in multiple namespaces, then you MUST provide overlordLeaderElectionConfigMapNamespace
2026-10-09T17:48:18,886 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[0] of size[655,360,000]
2026-10-09T17:48:19,322 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new result merging buffer[1] of size[655,360,000]
2026-10-09T17:48:19,646 INFO [main] org.apache.druid.offheap.OffheapBufferGenerator - Allocating new intermediate processing buffer[0] of size[655,360,000]

Is my only recourse to manually configure the buffer size for peons since the "auto sizing" appears to be incorrect here?

Yes, it isn't uncommon to change these settings either in the MM or in specific Supervisor specs, e.g.

context: { "druid.indexer.runner.javaOptsArray": [ "-XX:MaxDirectMemorySize=NNg" ] }

In Imply Druid many of these properties are already set in predefined profiles per instance type/size.

This is fantastic!! Can

javaOptsArray

also be set in the

taskContext

(e.g. for compaction)?