This message was deleted.
# troubleshooting
s
This message was deleted.
y
The java stack btw:
Copy code
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
	at java.nio.Bits.reserveMemory(Bits.java:175) ~[?:?]
	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118) ~[?:?]
	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317) ~[?:?]
	at org.apache.druid.segment.CompressedPools$4.get(CompressedPools.java:102) ~[druid-processing-25.0.0.jar:25.0.0]
	at org.apache.druid.segment.CompressedPools$4.get(CompressedPools.java:95) ~[druid-processing-25.0.0.jar:25.0.0]
	at org.apache.druid.collections.StupidPool.makeObjectWithHandler(StupidPool.java:171) ~[druid-core-25.0.0.jar:25.0.0]
	at org.apache.druid.collections.StupidPool.take(StupidPool.java:154) ~[druid-core-25.0.0.jar:25.0.0]
	at org.apache.druid.segment.CompressedPools.getByteBuf(CompressedPools.java:110) ~[druid-processing-25.0.0.jar:25.0.0]
	at org.apache.druid.segment.data.DecompressingByteBufferObjectStrategy.fromByteBuffer(DecompressingByteBufferObjectStrategy.java:49) ~[druid-processing-25.0.0.jar:25.0.0]
	at org.apache.druid.segment.data.DecompressingByteBufferObjectStrategy.fromByteBuffer(DecompressingByteBufferObjectStrategy.java:28) ~[druid-processing-25.0.0.jar:25.0.0]
Seems to me like data sketches uses a lot more direct memory than expected!
j
IIRC, the sketch’s internal hash table can be up to 2x nominalEntries, so 256kiB since each entry is 64 bits, plus something like 32 extra bytes for extra internal use (storing configured nominal entries, current size, etc)
A direct sketch should be identical to the serialized image of a sketch