Hi folks. I'm pretty fed up with the reliability/r...
# docker
j
Hi folks. I'm pretty fed up with the reliability/repeatability problems of installing Lucee extensions at build time in the official Lucee image. Collectively, I've probably wasted a work week or three on it over the last several years. I know that the big competitor to the official Lucee image is the Ortus one, though I've avoided it so far because it is (or was) pretty convoluted under the hood, but I'm more open to it now that I've steadily soured to the official image. Can someone who knows the Ortus image explain to me the mechanics of its extension installation, and how it knows that extension installation (both *.lex and repo-based) is complete before it shuts down the servlet container? (That's always been my problem with the official image: No simple way to know that the extension installation is complete before shutting down the servlet, which makes it so that 1% of your builds have a badly-built image, with no outward indication that anything went wrong.)
b
@jamiejackson As much as I'd like you to use the Ortus image, it really doesn't do anything directly to solve your question. All the same Lucee features which can install extensions on start are available in the Ortus image because in reality, it's Lucee itself doing this work.
For example, if you want to warm up Lucee in the Ortus image, it's still up to you to • start the server • do stuff like copy lex files (possible in opposite order) • detect when they are possibly installed • shut down lucee
Now, the "real" answer to your question IMO is the Lucee env var/sys prop for the warmup flag https://docs.lucee.org/guides/Various/system-properties.html#lucee_enable_warmupluceeenablewarmup
This is something I lobbied for a long time to get built into the Lucee core which will automatically shut down Lucee as soon as all extensions are finally installed.
I'm actually wanting to say I ran into an issue with the Ortus image getting confused the last time I tired to use it, but I'd have to check again since I recall talking to Jon at the time about a change we needed to make.
j
Thanks for the input @bdw429s. I wonder why I'm the only one I've heard complain about this, over the years. I hate my workaround but it's the only thing that got me 100% reliable images. 😞 https://github.com/jamiejackson/lucee5-install-extensions/
b
Either way, that's the "best" wait to warm up • set
LUCEE_ENABLE_WARMUP
env var • get all lex files in place • start server • server will shutdown once it's done • remove env var
j
oh, wait, you're saying that there is a solution already. rereading
b
The ticket is marked as added in 5.3.6, but I think a terrible, broken version was added there and then later fixed quietly without any notice in a later version lie 5.3.7 or 5.3.8 https://luceeserver.atlassian.net/browse/LDEV-1196
j
what about ORM, tho? my workaround also does this awful, but necessary thing, where i start up lucee, run a little page that uses an orm function, queries the admin for the presence of the hibernate extension, then shuts down lucee. ...because last i checked, that extension doesn't get installed until it's called via an orm function, which i thought was crazy.
b
I'm not aware of any such behavior
j
and you're an orm user?
b
I will say at Ortus we've had special "issues" with Orm not always working on the server's first start, but I'm not sure if that behavior was related to what you're describing
Ortus does use ORM on many projects, but not all of them. I don't personally use it a lot due to the inherent complications it always seems to bring.
j
I will say at Ortus we've had special "issues" with Orm not always working on the server's first start, but I'm not sure if that behavior was related to what you're describing
I bet it's the same thing. It's the same thing I experienced until I added my workaround. Check out bullet 1 in my "refs" section. In which micha talks about this "just-in-time" installation, which never sat right with me in a container context. https://dev.lucee.org/t/please-explain-hibernate-orm-installation-initialization-internals/4917
b
We used to have to • start lucee • stop lucee • start lucee again • THEN run test suite using ORM
j
thanks for the LUCEE_ENABLE_WARMUP tip. i'll see if it helps me simplify any of this. i have a related but tangential question about setting the password from `password.txt`but i'll post in a new thread.
👍 1
j
Might be worth taking a look at this: https://github.com/isapir/lucee-docker. I've used this Dockerfile in the past (haven't touched in a while though), and could just pass in extensions, passwords, warmup settings, etc. using ENV vars
b
☝️ All that image does is • set the same env vars Jamie is already using • starts the server with the
LUCEE_ENABLE_WARMUP
flag enabled • starts Lucee and lets it shutdown when it's done
For older versions if Lucee it simply sleeps for 20 seconds and then shutsdown
So yeah, that should work, but it's not anything different than what we're already discussing
j
@jamiejackson In the Ortus image, this is pretty simple:
Copy code
FROM ortussolutions/commandbox:lucee5

#Example using Hibernate Extension ( GUIDS HERE : <https://download.lucee.org/> )
ENV LUCEE_EXTENSIONS=FAD1E8CB-4F45-4184-86359145767C29DE;version=3.5.5.87

RUN ${BUILD_DIR}/util/warmup-server.sh
The environment variable above will flag the extension for installation. If you are using an extension hosted on forgebox, you can also use
box
to install it:
Copy code
RUN box install 5C558CC6-1E67-4776-96A60F9726D580F1@2.0.0+6428
j
hi @jclausen & @jumpmaster, afaik, these solutions still rely on the lucee mechanism for installing plugins, which i've found to be unreliable in a docker build context.
@bdw429s, i'm not spotting the magic in LUCEE_ENABLE_WARMUP, unless
shutdownFelix()
inherently waits for working threads to complete or something: https://github.com/lucee/Lucee/blob/011bf2f2663722ff4fc1bd94257b35fde47323ef/core/src/main/java/lucee/runtime/engine/CFMLEngineImpl.java#L1754
b
rely on the lucee mechanism for installing plugins
To be be clear, there is no other mechanism. The only way to install an extension into Lucee is to ask Lucee to install it for you. The question is usually whether Lucee is finished doing so before you halt the warmup process.
j
yup
b
i'm not spotting the magic in LUCEE_ENABLE_WARMUP,
It's really just that it shuts down the server when it's done so you don't have to guess. All of the env var checks and
deploy/
folder checks in Lucee are now synchronous (as of 5.3.7) and happen as soon as Lucee boots and it won't process any request until it's done. The scenario you're trying to avoid is
Copy code
start lucee
sleep 20
stop lucee
because what if it takes 21 seconds to finish installing all the extensions? Well, you shut down too soon and the extensions aren't correctly installed. So with the flag it's just
Copy code
set lucee_enable_warmup true
start lucee
and that's it. it shuts down when it's one and you don't have to guess.
j
synchronous is good. i'll trust you that LUCEE_ENABLE_WARMUP truly waits for the extensions to install before shutting down, which will simplify some things. i'm still. going to do the extra step to warm up the hibernate/orm extension, since that needs an orm function to be called before it completes (regrettably).
b
That's not a bad idea at all
Since Lucee won't process requests until the extensions are installed, you can probably run
curl
until it comes back with a 200
j
yeah, that's basically what my thing does.
👍 1
i've got a catalina start/stop wrapped around this:
Copy code
#!/usr/bin/env bash

set -e

echo "create a test app"
mkdir -p /var/www/wwwroot/test_app

cat > /var/www/wwwroot/test_app/Application.cfc <<EOF
component {
  this.ormenabled = true;
}
EOF

cat > /var/www/wwwroot/test_app/index.cfm <<EOF
<cfscript>
ormReload();
</cfscript>
EOF

echo "wait until extension is installed"
until $(curl --output /dev/null --silent --head --fail "<http://localhost:8888/test_app/>"); do
  printf '.'
  sleep 1
done
echo ""

echo "clean up test app"
rm -rf /var/www/wwwroot/test_app
things are much simplified with
LUCEE_ENABLE_WARMUP
thanks a lot for the help, @bdw429s
👍 1