Hi team, I'm getting this error in broker, control...
# troubleshooting
m
Hi team, I'm getting this error in broker, controller, minion, server. Kindly help on this.
l
You're probably using the wrong version of the java VM in your PATH, Check the requirements of Pinot and make sure you have that Java version instralled and `java -version`shows this version. (you have to change the PATH and the JAVA_HOME variable)
m
This is in GKE environment
l
sdkman.io is a good tool to install and switch between java VMs
ah
can't help you there, sorry
m
anyways, this is because of java version mismatch?
CC: @Sadim Nadeem
l
it looks like you are using a switch that is not known by your java VM, that would be my first hypothesis
m
@Mayank??
x
I think you can remove this option from the jvmOps in
values.yaml
file
or just change the image tag to
latest-jdk11
m
ok let me try
I have tried with image tag, same issue
I'll try to remove JVMOpts line from values.yaml
now
@Manju Priyadharshini!
s
@Xiang Fu we can see the latest pinot image is updated on docker hub for latest tag and causing some trouble*(could not create jvm ..unrecognized vm option 'PrintGCDateStamps')* .. is it related to the latest migration to java11 for pinot?or is there some issue in gke cluster which is causing this error in jvm start .. please guide
x
latest image is built in jdk8, I think helm scripts is using vm options available in java 11
s
is there some tag for jdk8 or some release on dockerhub which is stable for other java versions .. we dont see the history of the pinot images that were earlier pushed on dockerhub and the image is replaced .. the older image was working fine in old gke cluster .. once we moved to a new gke cluster .. facing issue i guess. . @Mohamed Sultan please add
latest image is built in jdk8, I think helm scripts is using vm options available in java 11 -->ohhk ..what will be the vm options for jdk8 @Xiang Fu
x
Can you check git commit history and see the change of java 11 upgrade
s
the image on dockerhub was updated almost 10 hours ago .. is that the root cause of the issue we are facing maybe because of java version changed
x
Default is jdk8
Jdk11 images have a suffix in the tag
s
@Mohamed Sultan tried with that .. but again same issue .. is there some tag for jdk8 image
atleast latrest or latest-jdk11 should work .. means one out of these two
x
Latest image is jdk8
Latest-jdk11 is jdk 11
s
ok so for latest-jdk11 .. all vm options must be removed|
means no xmx and xms shhould be provided and then tested?
x
No
You just change the image tag
From latest to latest-jdk11 and try
s
yeah .. we tried that but it didnt resolved
x
No need to modify the jvmOpts
s
ok
x
Oh?
What’s the issue
s
let me recheck with @Mohamed Sultan
m
I have changed to latest-jdk11 and removed jvmopts line from values.yaml
I think there is no issue now
This is the change from old jvmOpts for jdk8
Just FYI
i think there is some bug in latest pull
28 Actually, Kafka works fine with newer versions of Java. I had the same problem, and found an error in the 
kafka/bin/kafka-run-class.sh
 script, where the Java version was incorrectly parsed. This line grabs too much of the version string:
Copy code
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([^.-]*).*"/\1/p')
This makes the 
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]]
 condition fail to identify the correct Java version, and adds some unsupported GC options. Changing the line above to this solved my problem:
Copy code
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([^.-]*).*/\1/p')
can u confirm @Xiang Fu
cc: @Daniel Lavoie @Mayank @Ken Krugler @Mohamed Hussain
m
CC: @Shailesh Jha
x
This is always override by helm
JvmOpts
Check helm repo
This will override default jvm opts from admin script
s
@Xiang Fu @Mohamed Sultan
@Xiang Fu: for latest-jdk11 .. pinot is working fine on gke .. only concern is we are not providing any jvm options in helm .. so from which file will it be picking default jvm options and how to give our own jvm options since its failing if we provide it in helm.. as we need to check metrics in prometheus and also increase xmx
x
you can provide the jvm opts
just remove the ones that complaining
it will override the existing jvmopts