This message was deleted.
# general
s
This message was deleted.
a
@Saydul Bashar if you can help here.
j
Any help would be immensely appreciated. I actually, after enough grasping at random straws, was able to get it to work by passing in JAVA_OPTS with all the java/jetty options for JKS KeyStore password. NONE of the druid native ways to provide this information worked. I attempted: • Standard environment variable approach • Password provider approach • Verifying all jks keystore and truststores • many more I can't even remember
s
Hi @Jason Witkowski I am not an expert on the Druid mTLS settings. The last time I tried to use mTLS, it wasn't ready for production. I am hoping that those issues have been fixed in the recent release. From what you are saying, things are only working when you pass the information via JAVA_OPTS. If that is the case then there is a good chance that the check that is failing is failing for the processes. The standard environment variable approach will not work for these cases because most methods of setting environment variables set them at the server or pod level but not at the process level. So, in most cases, you will find that even though you think you are setting the env variable at the top level somewhere the env variables are not being set at the process level. I dealt with a similar issue a while ago. I basically went to the process that starts all druid processes and added the environment variable information there. That resolved the issue. I hope this helps.
The process that starts all the processes is run-druid. you can check the environment variables set for a process by doing
cat /proc/<process-id>/environ
j
I can see from the logs above that the process recognizes the environment variables
it iterates through them in
druid.sh
and prints them to the runtime properties file under /tmp/*, which I can confirm contains the correct values
Unless i'm missing something?
The worst part is when I undo it all to get it working without mTLS, i accidentally left some of the env vars there for the jks password. It processed those enough to throw the error:
Copy code
Caused by: java.lang.NullPointerException: must specify a trustStorePath
Because the trustStorePassword field was set. But if i specify a thrustStorePath then it tells me the password is null lol
s
Hi @Jason Witkowski I reread my messages above and it seems like I am saying that the env variable should be available for the run-druid process. Actually from the error you showed above it seems like the env variable should be available in the java task. I see what you mean by the inconsistencies of the variables and how they are set. Happy to jump on a call with you and see if we can fix this together. Here is my calendar link: https://calendly.com/saydul-bashar/30min
j
omg thank you! I tossed a meeting for your earliest time.
If you free up earlier and want to get it out of the way I can be completely flexible to your schedule. This issue has been blocking me for a full week at this point lol
s
You are welcome šŸ™‚. That time suits me perfectly. I can't promise that we will get to the bottom of this but we will definitely try šŸ™‚
šŸ™ 1
j
We figured it out after a whole week
The documentation claims that
KeyManagerPasssword
is optional and not required.
Copy code
druid.server.https.keyManagerPassword	The Password Provider or String password for the Key Manager.	none	no
But that is not true.
So we are supplying trustStore password, keyStore password, and keyManager password. This gets past the TLS start-up
I don't actually know what a
keyManager
is. It's not something we're explicitly using so I'm guessing its something more native to Java
s
Thank you for the update @Jason Witkowski. Congratulations šŸ™‚. Glad to hear that your hard work paid off. This is a great question for the dev team. If you have any time raising a GitHub issue would be amazing. Do you still want to jump on a quick call so that we can say hi to each other or are you too tired with all the debugging so far and we should sync up some other time?
I mean our scheduled call should I keep it or cancel it?
j
Lets reschedule if you'd like to keep in touch. I'm going to celebrate Friday after this win šŸ™‚
I really appreciate your willingness to hop on a call.
s
Lol šŸ™‚. I know the feeling. Celebrate your win. I will cancel our meeting for today. You have my calendar link. Let's catch up some other time. I wasn't much help this time around but if you have data modelling questions or druid tuning feel free to reach out šŸ™‚.
šŸ™ 1