http://coldfusion.com logo
Join Slack
Powered by
Did anything change between 5.3.9.166 and 5.4.1.8 ...
# lucee
a
Did anything change between 5.3.9.166 and 5.4.1.8 regarding the JDBC drivers? Or anything to do with how the networking part of cfquery etc works? In my case I am using the 
-nginx
docker images. On 5.3.9.166 everything works as expected. On 5.4.1.8, 
<cfquery>
reqs are just sitting there doing nowt until the request times out. On dev we're using the Docker network, and that was fine on 5.4.1.8; I've deployed to one of our prod boxes, where the DB is across the host network, and 
<cfquey>
and its ilk suddenly just hang. The only change is 5.3.9.166 to 5.4.1.8. I do not currently have much else to go on other than that; and haven't 100% tied it down to this (although I'm at about 95% currently)
z
MySQL? The driver was updated
šŸ‘ 1
j
i don't use the extension for my (mariadb) driver--i just use a vendor jar that i bake in with my build. (don't remember why i'm not using the extension--this goes way back). however, i did try to update the vendor driver recently, and there was a major release between my current version and the latest. i did have issues with the latest, which i didn't have time to troubleshoot, so i used the latest version from the major release that i'm currently on. not sure that's much to go on, but if the extension has a major leap, then maybe that explains things. in which case you could just pin 5.3.9.166's extension version and see if that gets things working.
ā­ 1
a
OK, cool. That gives me something to work on. I need this working so can sink some more time into it (a dedicated day or so anyhow). If I find anything, I'll report back.
šŸ‘ 1
gonna try to at least replicate locally tomorrow, which'll expedite test/experiment iterations.
Can replicate on a completely different server now. @zackster how can I revert the MySQL JDBC drivers to test if that's the issue? Is there a jar file I can swap? Is it as easy as that? I see the 5.4.1.8 container has a 
/opt/lucee/server/lucee-server/bundles/com.mysql.cj-8.0.33.jar
and the 5.3.9.166 one has 
/opt/lucee/server/lucee-server/bundles/com.mysql.cj-8.0.19.jar
Can I safely stick the latter in in place of the former, and expect it to work? (I will just try in the meantime, in case the answer is "yeah, it should do")
z
yoi can just change the datasource definition bundle and lucee will auto download
a
ooh, OK
Cool
z
are you using application level dsn?
a
But hang on. If we have this:
Copy code
this.datasources[server.system.environment.app_name] = {
			class: 'com.mysql.cj.jdbc.Driver',
			bundleName: 'com.mysql.cj',
			bundleVersion: '8.0.19',
Does that mean that we would be using the old drivers even if Lucee itself has upgraded to new ones?
And... yes, all DSNs set via Application.cfc
What's the URL Lucee downloads from? Because I really bet that whatever it is is not open on our servers...
This could be the problem. If Lucee sees we're specifying 8.0.19, and it only has 8.0.33 so is trying to download 8.0.19, and our firewall is going "the fuck you are"
z
possibly, if that's the case try setting this value, to get an error (we can also fix this to be more sane) but try setting 
lucee.enable.bundle.download=false
a
OK, testing
this could be good news.
z
yeah, i shortened the timeout to be 5 s to connect, we had a related bug a few years ago, when some of the bunldes jars were'nt in the build and someone had lucee hanging for 15m on startup behind a firewall
a
This looks to be the problem
Which is good.
However I think Lucee needs to handle a failed download better, maybe? We get no feedback ANYWHERE about this being the issue, and this has, as a result, wasted about a week of my time.
z
100%
a
So be it... this is my job... but if Lucee logged "I can't phone home", then I'd've had this sorted very quickly
It's also cost us a coupla grand in support with our service provider.
Again, "so be it", but these shortfalls are not "zero-cost"
z
it does log, now i know there's a problem i can fix it
a
Is it documented anywhere (and loudly, and clearly) that if the DSN config says "use driver x" then Lucee will attempt to D/L it? It never even occurred to me to check this. Also, can I just NOT specify that sort of granularity in the DSN config? I don't particularly want to use a specific version of the driver. It's - to me - Lucee's job to handle that.
ie: like instead of this:
Copy code
class: 'com.mysql.cj.jdbc.Driver',
bundleName: 'com.mysql.cj',
bundleVersion: '8.0.19',
Can I just do something like this:
Copy code
driver: 'mysql'
?
z
if you don't have the correct extension version that you are using
just leave out the bundle version
a
where that implies "just use the one you have"
OK, good to know
(these config settings predate me, and I've never bothered to know the minutiae of what's needed / not needed / etc)
z
so you confirmed it, can you file me a bug
a
Absolutely
I am just gonna continue to dance for a bit, then will sort my servers out and get on with it.
Do I need to even specify the 
bundleName
? For one thing, it's repeating what's in the driver line anyhow, so I... suspect... there's not a lot of optionality there anyhow?
it does log, now i know there's a problem i can fix it
There is nothing in any log file that indicates "I needed to phone home but could not". All I get is the exception from the request timeout in exception.log, and another similar entry in requestimeout.log
exception.logfrom_requesttimeout.log
That's it.
(the server context application.log has some application errors caused by the failed start-up, but those are all like "key not found in struct" and shiz like that)
Do I need to even specify the 
bundleName
?
Looks like it's also surplus to requirement.
z
so on my laptop with wifi turned off, i get for both application.datasouces or passing in a datasource directly into cfquery the following error
<cfscript>
ds = {
class: "com.mysql.cj.jdbc.Driver",
bundleName: "com.mysql.cj",
bundleVersion: "8.0.35",
connectionString: "jdbc:<mysql://localhost:33306/preside-demo?characterEncoding=UTF-8&serverTimezone=US/Eastern&maxReconnects=3>",
username: "preside-demo",
password: "encrypted:xyz",
// optional settings
connectionLimit:-1, // default:-1
liveTimeout:15, // default: -1; unit: minutes
alwaysSetTimeout:true, // default: false
validate:false, // default: false
};
</cfscript>
<cfquery datasource="#ds#" name="zac">
select 1
</cfquery>
5.4.2.20 has only xml changes and admin stuff compared to 5.4.2.17
that request timeout entry has the clue
Copy code
at lucee.loader.engine.CFMLEngineFactory.downloadBundle(CFMLEngineFactory.java:726)
        at lucee.runtime.osgi.OSGiUtil._loadBundle(OSGiUtil.java:633)
there's a 10s connection timeout since 5.3.9 https://luceeserver.atlassian.net/browse/LDEV-3355
a
I do not get anything like that, and the request sits there until the normal req timeout
I just get a req timeout on the 
<cfquery>
tag
z
yeah i suspect it's due to the difference between how a firewall and being actually offline and leaving the request hanging
ideally your firewall would be singing the chorus to this old aussie classic

https://www.youtube.com/watch?v=3-VZP1pCIL8ā–¾

šŸ˜œ 1
a
The Angels. Blimey.
TBH I don't think Lucee should be doing this. It should just error out with "you ain't got that lib", like with any other lib I might not have. It should not be making uncontrolled changes like that in the background. It's on me to configure my servers correctly; not for Lucee to hide it from me. It's not appropriate.
z
that the enable bundle download directive
šŸ‘ 1
a
Where's that? I doubt I have switched it on (then again, I would not have put that shit in my DSN config either so who knows what other stupid shit has been done in this app) Is it on by default unless otherwise set off?
z
yes, otherwise in place upgrades don't work
5.4.3.0 has the extra timeout, 6 already had them somehow
a
yes, otherwise in place upgrades don't work
At a superficial level, that seems like bad coupling. There's quite a difference between "someone clicking a button to opt-in to a download", and "Lucee will do stuff by itself in the background without being asked"
z
updating in place doesn't include any jars
a
I do not know why that might be a response to my comment.
z
because i'm explaining why it is the way it is
a
You might think you are. But I am not sure you're using enough words for me to understand šŸ˜‰
z
so when update via the admin, the update is just the core engine, any updated jars required are downloaded dynamically, most of the time there are no new ones, but we do update them semi regularly the only bundles you can download from the update provider are allow listed via a cache from releases and extra maven mappings
that way upgrading from 5.4.2.17 to 5.4.3.0 is a 9mb download and not a 20mb download
a
Noted and makes sense. Doesn't really address my point though. I'll contextualise and clarify (well: try to clarify ;-)) Z:
that the enable bundle download directive
https://docs.lucee.org/guides/Various/system-properties.html#lucee_enable_bundle_downloadluceeenablebundledownload
A:
Is it on by default unless otherwise set off?
Z:
yes, otherwise in place upgrades don't work
A:
At a superficial level, that seems like bad coupling. There's quite a difference between "someone clicking a button to opt-in to a download", and "Lucee will do stuff by itself in the background without being asked"
To try again: * someone clicking a "please update Lucee" is intrinsically opting-in to downloading taking place. They've consented. Do it. * Lucee magically and invisibly deciding to download new libraries without being asked. Those are two completely different things. The only commonality is "stuff gets downloaded". They should not be controlled by the same setting. It is entirely realistic to allow an active-opt-in inplace upgrade, whilst at the same time prohibiting Lucee from doing something it's not been asked to do.
z
it's akin to box install or npm installl
a
It's not, cos 
box install
is something I opt in to do as well! I need to type in 
box install [enter]
! 
LUCEE_EXTENSIONS
is also opt-in, as I need to give a list of values. And by doing that I am opting-in to the documented process taking place. What the current situation is like with these drivers is like if I didn't set a value for 
LUCEE_EXTENSIONS
and Lucee just decided to go get any extension on the fly just cos my code happened to reference it. Instead of erroring-out and going "you need to install the extension to use it". All the examples you are giving require an opt-in. This driver download thing requires me to opt-out, and it sounds like if I do opt-out, it disables a bunch of unrelated stuff too?
j
sorry to hijack, but i've got a beef in the same ballpark. i can "install" the ORM extension but it doesn't really completely install until some code calls it. so a build isn't a clean build--it tees things up for installation at (container) runtime (but not until an orm function is called). the other extensions i'm used to take care of all of their business at build-time, which is good. (except maybe some quirky stuff with the extension that adam's referring to). (i might be wrong about this next one, since i'm going on memory): relatedly, in logs, (iirc), i can see log4j get downloaded installed at runtime. i should be able to have a build completely build at build time. runtime installation shenanigans are really dirty, IMO. i'd rather have that kind of stuff fail at build time and fail my pipeline, so it never gets deployed. i don't want there to be any possibility of installations to fail at runtime. i'd be thrilled to find out that I'm wrong, tho, so set me straight if i am.
i remember an old thread on the dev site where adam was initially baffled by the behavior i describe (of ORM i think), but was appeased when someone (maybe zac) said something like "that's just the way osgi works." i wasn't convinced at the time, tho.
z
yes as you can see in the admin, bundles are dynamically only loaded on use
wanna warm em? call em
j
that's what i'm doing in my build and it's ugly. i install the extension, i place a page in the web root which calls a simple orm function, i loop a wait until a localhost curl returns a success, i delete the page, and i continue with the build. i think that's excessive and weird for a 
Dockerfile
. i feel like there ought to be a better way. (i stand by the notion that installations should happen at build-time.)
šŸ’Æ 1
z
use server.cfc or web.cfc
a
Right we've finally got our SANDBOX env updated to 5.4.1.8. Only took 1.5 weeks all up. Issues: ā€¢ magic unexpected Lucee downloads that silently fail. ā€¢ docker version incompat with the version of Ubuntu the new Lucee image uses. QA still need to regression test that said.
2 Views
Open in Slack
PreviousNext
Powered by