Hello all - Background I recently had to install M...
# sql
r
Hello all - Background I recently had to install MySQL 5.7 to run parallel with MySQL 8 due to differences in query results between the versions. I was successfully able to get everything set up in Lucee 5.3 to point to MySQL 5.7. Problem I am running a query of query and using a group by on only one field. It is actually grouping by month and year with the "monthly" field (ie values look like Mar'21 or Jun'22) without using an aggregate function on any fields in the SELECT clause. It works perfectly on my local machine. Example:
Copy code
SELECT 		marketValue,
			marketPercent,
			d_as_at_date,
			monthly
FROM 		qMonthlyPortfolioSum
GROUP BY 	monthly
ORDER BY	d_as_at_date
My colleague is stating that he receives the following error:
Copy code
Not in aggregate function or group by clause: org.hsqldb.Expression@6b6ba733 in statement [SELECT marketValue, marketPercent, d_as_at_date, monthly FROM qMonthlyPortfolioSum GROUP BY monthly ORDER BY d_as_at_date]
I cannot figure out why he is receiving an error when I am not on the query of query when we both have the same version of MySQL. I then looked in the Lucee Admin and noticed that the driver installed is version 8.0.28. I attempted to down grade MySQL Application to the highest version 5 available, which is version 5.1.40 and also version 6, but anything lower than 8.0.19 errors when validating the datasource in Lucee Admin. There is not a version 5.7 available to choose in the list of MySQL drivers for Lucee. I have even put the query of query together with the real query and executed the script against MySQL 5.7 the DBMS and it runs fine outside of Lucee as well. Would anyone have any sort of ideas as to the discrepancies between the errors on the local environments? Is it possible that I may not be running MySQL in some kind of restrictive mode and that maybe my colleague might have something restricted that would cause an error on his side?
s
There are a few things at play here, and im unsure where you error is happening. so lets break it down, can both of you dump out the results of the query directly from the database (before you run a query of query on it)?
The error itself is in reference to the fact that all select columns that do NOT have an aggregate called on them (sum, avg, etc.) must also be present in the Group By.
where it is occurring is the question. if its from the database this makes sense. mysql 5.7 allowed you NOT to list all columns in the Group By, 8 requires a setting to allow for this
r
Unfortunately, my colleague is out for the weekend
but, what you just mentioned is good info, thank you
👍 1
the group by occurs in a QofQ
s
ok well that helps
r
For testing, I did pull out the source query and the QofQ, combined them and ran it on both server versions 5.7 and 8. Both work fine for me
Basically, I cannot replicate this error no matter what I change regarding versions.
s
the mention of
org.hsqldb.Expression
means that its getting passed to the in memory database for processing. (some queries are completely run internally with java instead of hsql)
r
I was just told in the Lucee channel that the Lucee minor versions make a difference on the QofQs. I do not believe that my colleague and I are on identical minor versions, but we are both on Lucee 5.3.
ahh that's interesting! I thought hsql was always getting used for every query.
and both of us are on the hsql 1.8. I did check that.
this helps piece things together
s
yes, there may have been an update to hsqldb that is to blame for this. Lucee 5.3.8 was a significant upgrade to QofQ by @bdw429s. it allowed more support and faster response of queries since they dont even get passed to
hsqldb
r
excellent, thank you so much for that
I'm going to downgrade Lucee and see if I can replicate the error that my colleague receives
I appreciate the help @Scott Steinbeck!
s
no problem, cheers
🤘🏼 1
b
@ryan • Always include full stack traces with your error reports. • If the error is happening on a Query Of Query, all of the MySQL details you provided are not only irrelevant, but also just a red herring. A QoQ behaves the same regardless of where the query object came from, or if it even came from a JDBC call at all (as opposed to
queryNew()
@Scott Steinbeck I do think HSQLDB was updated in the last Lucee release. @zackster would know for sure, or you could just search the Lucee ticket tracker
I don't see a ticket right off for it, but I could be searching for the wrong text.
That said, the error is pretty obvious-- SQL doesn't allow you to select columns that you aren't grouping by. MySQL allows you to do this (and just gives you a random value for non-grouped columns). HSQLDB obviously isn't fond of such a thing.
I am curious why the QoQ is hitting HSQLDB at all. it seems like a very simple query and the native QoQ should have processed it. I'm pretty sure the native QoQ will let you select columns you haven't grouped in a similar manner to MySQL.
@ryan’s SQL works fine for me on a QoQ, so he'll need to provide an actual test case that shows the error before we can debug any further https://trycf.com/gist/4e516170500e52165bcd1a087047c816/lucee5?theme=monokai
I'm guess as to what his column types are, plus his error could be related to the data in his query 🤷
Trying that SQL on Lucee 4.x does throw the error you reported. prior to Luce 5.3.8, the native QoQ had no support for grouping or aggregates at all, so that code would only work on 5.3.8 and up if it selects columns it's not grouping on. So if your coworker is on an old version such as 5.3.6 or 5.3.7, then yeah, that wouldn't work for them.
r
Thanks for confirming. I'm in the middle of downgrading lucee. My colleague is on lucee 5.3.3.62
b
Holy cow, that's old
Why not just have him upgrade?
r
everything is a process Lol
b
Any QoQ using group by will 100% run on HSQLDB on a version of Lucee that old
And HSQLDB is not as forgiving as MySQL or the newer native QoQ implementation added in 5.3.8
r
I have asked them to upgrade MySQL as well, because I have hit issues I never imagined hitting because they are still on MySQL 5.7 rather than 8. MySQL 5.7 reaches EOL next year.
b
Sure, but don't confuse MySQL issues with QoQ issues. Those are two separate things.
r
yeah, I didn't know that til now
b
If you're getting an error on a Query of Query, adjusting your MySQL version, etc, etc will be pointless
Like Scott said, QoQ received a MAJOR overhaul in Lucee 5.3.8
r
I thought that QofQ used whatever driver is installed to the datasource, but nope.
b
If you guys use it for anything, you eed to update
No QoQ is 100% in-memory manipulation of a query object
👍🏼 1
There were some very serious performance issues with QoQ in Lucee prior to 5.3.8
Lucee's QoQ is now actually faster than Adobe's!
🎉 2
The graphs and explanations are all in that post
nevermind, the graphs are missing after my blog upgrade, lol. let me see if I can fix those
r
Thanks for this, Brad. I'm going to push to have them upgrade Lucee, but as I have realized with trying to get them to upgrade MySQL, it will be placed as a task in the distant future. Lol
s
I didn’t realize you upgraded your blog! Looks nice
r
I'm going to have to reconfigure the QofQ for now
b
Gavin helped me get it to a current version of ContentBox, lol
s
for something that simple you could do it all in SQL or handle it with a queryReduce
r
Great article, @bdw429s. I'm going to refer to this article to my colleague to push for the lucee upgrade. Thanks, again.
👍 1
@Scott Steinbeck I will look into queryReduce, thank you 🙂
b
Lucee 5.3.9 is also VERY stable as well so it's a good point to update to.
👍🏼 1
Every known regression in the backlog was squashed for that release
r
great to know
I will relay the your points
so thankful for your creation of commandbox, @bdw429s It amazes me even after all of these years. 🙂
metal 1
so simple to up/down grade