This message was deleted.
# devs
Slackbot
08/06/2023, 2:48 AMThis message was deleted.
o
Oleg Yamin
08/07/2023, 6:44 PM@Maytas Monsereenusorn Not if you are using HDFS as deep storage, since hdp only works on Java 1.8, So had to go back to using 1.8. Otherwise, All services ran fine on druid 0.23.0 on Java 17, until the middle manager started trying to write to HDFS 😞 Kept getting errors like NoSuchMethodError:
Copy code
2023-06-14T01:25:12,327 WARN [DataStreamer for file /druid/task-logs/index_kafka_.LOGGING _3d968870f9eb302_phgoknak] org.apache.hadoop.hdfs.DataStreamer - DataStreamer Exception
java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'Oleg Yamin
08/07/2023, 6:45 PMI posted about it also about a month ago, but never got any replies or possible solutions :(
m
Maytas Monsereenusorn
08/07/2023, 6:46 PMAh. I was able to run Druid 26.0.0 + https://github.com/apache/druid/pull/14384 on Java 17 but we are not using HDFS as deep storage
o
Oleg Yamin
08/07/2023, 6:48 PMerr. yeah I was testing it with newest version 26. Sorry...
But yeah.. if you are not using HDFS as storage, I dont think you will have any issues... although.. make sure you add all the exception rules to the jvm config.. and you will be fine
m
Maytas Monsereenusorn
08/07/2023, 6:49 PM👍
Maytas Monsereenusorn
08/07/2023, 6:49 PMI tested with Druid built on Java 8 but was wondering if there will be any benefit to also compile and build on 17 as well?
o
Oleg Yamin
08/07/2023, 6:56 PMI would think mostly performance improvements and new enhancements that you get with 17... and it is LTS.. so thats a plus too.
Oleg Yamin
08/09/2023, 9:32 AM@Maytas Monsereenusorn where you able to get everything going on Java 17? And if you did what did you use for deep storage?
m
Maytas Monsereenusorn
08/09/2023, 5:24 PMI was. I am using s3
k
Karan Kumar
08/14/2023, 3:20 PM@Oleg Yamin Which hadoop version do you use?
o
Oleg Yamin
08/14/2023, 3:40 PMI tested it on 3.3... but Apache Hadoop 3.3 and upper supports Java 8 and Java 11 (runtime only). So no matter what i tried it kept failing to write due to missing classes
k
Karan Kumar
08/16/2023, 10:48 AMhttps://issues.apache.org/jira/browse/HADOOP-17177 Looks like the relevant jira.
o
Oleg Yamin
08/16/2023, 6:29 PMYeah... they actually added bunch of support for 17 on new 0.26 version, but I think until HDP starts officially working on Java 17, I dont think there is not much we really can do from Druid side.