Hi, we are seeing below error while running

index_hadoop

job

2024-04-09T03:06:09,556 WARN [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobResourceUploader - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2024-04-09T03:06:09,562 WARN [task-runner-0-priority-0] org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2024-04-09T03:06:10,052 WARN [DataStreamer for file /tmp/hadoop-yarn/staging/d_cidruid/.staging/job_1712631488122_0002/job.split] org.apache.hadoop.hdfs.DataStreamer - DataStreamer Exception
java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
        at org.apache.hadoop.crypto.CryptoStreamUtils.freeDB(CryptoStreamUtils.java:40) ~[hadoop-common-2.8.5.jar:?]
        at org.apache.hadoop.crypto.CryptoOutputStream.freeBuffers(CryptoOutputStream.java:291) ~[hadoop-common-2.8.5.jar:?]
        at org.apache.hadoop.crypto.CryptoOutputStream.close(CryptoOutputStream.java:225) ~[hadoop-common-2.8.5.jar:?]
        at java.io.FilterOutputStream.close(FilterOutputStream.java:188) ~[?:?]
        at java.io.FilterOutputStream.close(FilterOutputStream.java:188) ~[?:?]
        at org.apache.hadoop.hdfs.DataStreamer.closeStream(DataStreamer.java:987) ~[hadoop-hdfs-client-2.8.5.jar:?]
        at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:625) ~[hadoop-hdfs-client-2.8.5.jar:?]
        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:801) ~[hadoop-hdfs-client-2.8.5.jar:?]
Exception in thread "DataStreamer for file /tmp/hadoop-yarn/staging/d_cidruid/.staging/job_1712631488122_0002/job.split" java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
        at org.apache.hadoop.crypto.CryptoStreamUtils.freeDB(CryptoStreamUtils.java:40)
        at org.apache.hadoop.crypto.CryptoInputStream.freeBuffers(CryptoInputStream.java:683)
        at org.apache.hadoop.crypto.CryptoInputStream.close(CryptoInputStream.java:317)
        at java.base/java.io.FilterInputStream.close(FilterInputStream.java:180)
        at org.apache.hadoop.hdfs.DataStreamer.closeStream(DataStreamer.java:996)
        at org.apache.hadoop.hdfs.DataStreamer.closeInternal(DataStreamer.java:839)
        at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:834)
2024-04-09T03:06:10,073 WARN [DataStreamer for file /tmp/hadoop-yarn/staging/d_cidruid/.staging/job_1712631488122_0002/job.splitmetainfo] org.apache.hadoop.hdfs.DataStreamer - DataStreamer Exception
java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
        at org.apache.hadoop.crypto.CryptoStreamUtils.freeDB(CryptoStreamUtils.java:40) ~[hadoop-common-2.8.5.jar:?]
        at org.apache.hadoop.crypto.CryptoOutputStream.freeBuffers(CryptoOutputStream.java:291) ~[hadoop-common-2.8.5.jar:?]
        at org.apache.hadoop.crypto.CryptoOutputStream.close(CryptoOutputStream.java:225) ~[hadoop-common-2.8.5.jar:?]

any recent changes after this issue started ? might related to https://issues.apache.org/jira/browse/HADOOP-12760

no changes , i got some workaround from this thread https://apachedruidworkspace.slack.com/archives/C0309C9L90D/p1686706735072409

👍

even after updating jvm i am still seeing the error