I'm getting a stack overflow error when I try to `...
# box-products
s
I'm getting a stack overflow error when I try to
server start
on a remote EC2 instance. It has something to do with Globber.cfc, I'll put part of the verbose error in the thread. Any ideas? This server runs fine locally. Same version of CommandBox (5.9.0+00721) on both.
Copy code
at lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1775)
        at models.globber_cfc$cf.udfCall4(/globber/models/Globber.cfc:456)
        at models.globber_cfc$cf.udfCall(/globber/models/Globber.cfc)
        at lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
        at lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
        at lucee.runtime.type.UDFImpl.call(UDFImpl.java:223)
        at lucee.runtime.type.scope.UndefinedImpl.call(UndefinedImpl.java:786)
        at lucee.runtime.util.VariableUtilImpl.callFunctionWithoutNamedValues(VariableUtilImpl.java:787)
        at lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1775)
        at models.globber_cfc$cf.udfCall4(/globber/models/Globber.cfc:433)
        at models.globber_cfc$cf.udfCall(/globber/models/Globber.cfc)
        at lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
        at lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
        at lucee.runtime.type.UDFImpl.call(UDFImpl.java:218)
        at lucee.runtime.type.EnvUDF.call(EnvUDF.java:109)
        at lucee.runtime.functions.closure.Filter._inv(Filter.java:309)
        at lucee.runtime.functions.closure.Filter.invoke(Filter.java:195)
        at lucee.runtime.functions.closure.Filter._call(Filter.java:91)
        at lucee.runtime.functions.query.QueryFilter._call(QueryFilter.java:58)
        at lucee.runtime.functions.query.QueryFilter.call(QueryFilter.java:40)
        at lucee.runtime.functions.query.QueryFilter.invoke(QueryFilter.java:64)
        at lucee.runtime.interpreter.ref.func.BIFCall.getValue(BIFCall.java:142)
        at lucee.runtime.type.util.MemberUtil.call(MemberUtil.java:126)
        at lucee.runtime.type.QueryImpl.call(QueryImpl.java:2032)
        at lucee.runtime.util.VariableUtilImpl.callFunctionWithoutNamedValues(VariableUtilImpl.java:787)
        at lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1775)
        at models.globber_cfc$cf.udfCall4(/globber/models/Globber.cfc:456)
        at models.globber_cfc$cf.udfCall(/globber/models/Globber.cfc)
        at lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
        at lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
        at lucee.runtime.type.UDFImpl.call(UDFImpl.java:223)
        at lucee.runtime.type.scope.UndefinedImpl.call(UndefinedImpl.java:786)
        at lucee.runtime.util.VariableUtilImpl.callFunctionWithoutNamedValues(VariableUtilImpl.java:787)
        at lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1775)
        at models.globber_cfc$cf.udfCall4(/globber/models/Globber.cfc:433)
        at models.globber_cfc$cf.udfCall(/globber/models/Globber.cfc)
        at lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
        at lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
        at lucee.runtime.type.UDFImpl.call(UDFImpl.java:218)
        at lucee.runtime.type.EnvUDF.call(EnvUDF.java:109)
        at lucee.runtime.functions.closure.Filter._inv(Filter.java:309)
        at lucee.runtime.functions.closure.Filter.invoke(Filter.java:195)
        at lucee.runtime.functions.closure.Filter._call(Filter.java:91)
        at lucee.runtime.functions.query.QueryFilter._call(QueryFilter.java:58)
        at lucee.runtime.functions.query.QueryFilter.call(QueryFilter.java:40)
        at lucee.runtime.functions.query.QueryFilter.invoke(QueryFilter.java:64)
        at lucee.runtime.interpreter.ref.func.BIFCall.getValue(BIFCall.java:142)
        at lucee.runtime.type.util.MemberUtil.call(MemberUtil.java:126)
        at lucee.runtime.type.QueryImpl.call(QueryImpl.java:2032)
        at lucee.runtime.util.VariableUtilImpl.callFunctionWithoutNamedValues(VariableUtilImpl.java:787)
        at lucee.runtime.PageContextImpl.getFunction(PageContextImpl.java:1775)
        at models.globber_cfc$cf.udfCall4(/globber/models/Globber.cfc:456)
        at models.globber_cfc$cf.udfCall(/globber/models/Globber.cfc)
        at lucee.runtime.type.UDFImpl.implementation(UDFImpl.java:112)
        at lucee.runtime.type.UDFImpl._call(UDFImpl.java:350)
My server.json:
Copy code
{
  "app": {
    "libDirs": "lib",
    "cfengine": "lucee-light"
  },
  "openbrowser": false,
  "dotenvFile": ".env",
  "env": {
    "APPMODE": "${APP_ENV}",
    "APPNAME": "myapp",
    "LUCEE_EXTENSIONS": "E6634E1A-4CC5-4839-A83C67549ECA8D5B;name=MongoDB;version=3.12.8.133,CED6227E-0F49-6367-A68D21AACA6B07E8,37C61C0A-5D7E-4256-8572639BE0CF5838,B737ABC4-D43F-4D91-8E8E973E37C40D1B;version=1.0.0.46-SNAPSHOT,D46B46A9-A0E3-44E1-D972A04AC3A8DC10;version=1.0.19.24,66E312DD-D083-27C0-64189D16753FD6F0;version=1.1.0.19"
  },
  "dockEnable": false,
  "trayEnable": false,
  "web": {
    "rewrites": {
      "enable": "true",
      "config": "config/docker/rewrites.xml",
      "configReloadSeconds": 30
    },
    "rulesFile": "config/docker/rules.json"
  },
  "jvm": {
    "heapSize": 2048,
    "minHeapSize": 2048
  },
  "fusionreactor": {
    "enable": true,
    "licenseKey": "${FR_LICENSEKEY}",
    "debugEnable": false,
    "port": "8088",
    "password": "${LUCEE_ADMIN_PASSWORD}",
    "cloudGroup": "myapp,${FR_CLOUDGROUP}",
    "defaultApplicationName": "myapp",
    "licenseDeactivateOnShutdown": true,
    "ChatEnabled": false
  },
  "trace": "false",
  "name": "frontend"
}
b
Without seeing the full stack trace, it's impossible to know what use of the globber is having issues here
s
the full stack trace is so huge I can't even scroll back to the beginning in my console
b
The
"rulesFile":"config/docker/rules.json"
setting can be a globbing pattern, so it COULD be related to that.
Can you remove that and see if it goes away?
s
yep, standby
b
If a blank server works, then I'd add in stuff until you find the bad apple
the
.env
path may also use globbing-- I forget off the top of my head
Lol, here's the comment above the line that's recursing
Copy code
// If we're inside a **, then we just blindly recurse forever
				if( arguments.path.type == 'dir' && remainingPattern.startsWith( '**' ) ) {
					processPattern( thisPattern, local.thisPath, skipExcludes )
s
I removed both env and rules and now it works. Removing rules only did not work
yes, I saw that, 🤣
b
Whatever logic is supposed to be ending the recursion isn't working
What's interesting is you aren't even using any wildcards in your rules file path
s
Imma try putting rules back. I'm not sure I even need the .env? Since commandbox-dotenv should be loading those vars in?
b
Yeah, pretty sure
Copy code
"dotenvFile":".env",
is redundant since that's already the default file it looks for
So, is the
config/docker/rules.json
relative to the folder the
server.json
lives in?
s
yes
b
That logic is here BTW
Copy code
if( serverJSON.keyExists( 'web' ) && serverJSON.web.keyExists( 'rulesFile' ) ) {
			if( isSimpleValue( serverJSON.web.rulesFile ) ) {
				serverJSON.web.rulesFile = serverJSON.web.rulesFile.listToArray();
			}
			serverInfo.webRules.append( serverJSON.web.rulesFile.reduce((predicates,fg)=>{
				fg = fileSystemUtil.resolvePath( fg, site.serverConfigFileDirectory );
				return predicates.append( wirebox.getInstance( 'Globber' ).setPattern( fg ).matches().reduce( (predicates,file)=>{
						if( lCase( file ).endsWith( '.json' ) ) {
							return predicates.append( deserializeJSON( fileRead( file ) ), true );
						} else {
							return predicates.append( fileRead( file ).listToArray( chr(13)&chr(10) ), true );
						}
					}, [] ), true );
			}, []), true );
		}
s
OK, I added rulesFile back and it still works without dotenvFile
b
Oh, hmm. So is the dotenv the one causing issues?
Or the server rules?
s
the dotenv
could it have to do with the fact that the actual ./.env is symlinked to ../shared/.env?
I use Capistrano for deploys and it creates symlinks for shared config files, like .env
b
Hmm, symlinks always come with the possibility of screwing stuff up, lol
The dotenvfile is treated like so
Copy code
// Then look for a key in the actual server.json
        if ( serverJSON.keyExists( "dotenvFile" ) && len( serverJSON.dotenvFile ) ) {
        	serverEnvFile = serverEnvFile.listAppend(
        		serverJSON.dotenvFile.listMap( ( f ) => variables.fileSystemUtil.resolvePath( f, getDirectoryFromPath( serverInfo.serverConfigFile ) ) )
        	);
        }
So it's a list of files, which are each expanded based on the directory of the server.json file
I've had endless loops in Globber before-- often times due to slash mishandling or bad assumptions in a drive root, but in order to fix, I basically need to be able to reproduce
s
OK. I'm now trying to just load the app to make sure it's working without that dotenvFile key. I gotta fix a couple other things in my security group that are keeping me from verifying it
b
OK, doing a quick local test with that same config and file location
s
locally (Mac) this works fine. the Ec2 instance is Ubuntu
b
Yeah, works here too
Also, that setting is totally ignored
This code gets hit
Copy code
// If it's the same as the one we loaded above, ignore it
				if ( f == webRootEnvFile ) {
					return;
				}
dotenv recognizes that it's the same file it already loaded based on the web root convention
Out of curiosity, what do you get if you run this in CommandBox
Copy code
ls /path/to/your/.env
Since CommandBox's
ls
uses the Globber as well, that should be the rough equivalent
s
Copy code
commandbox-dotenv: Loading environment variables from [/home/deploy/dealstream/releases/20230510190336/.env] for command [ls]
May 10,2023  19:05:36   H         0.3 KB   .env
b
Hmm šŸ¤”
And that's the symlink'd file?
s
yes
ok, so turns out I'm not getting the error anymore but also the server is not actually starting
I recieve the typical response to server start:
b
I tried making a symlink locally on Windows, but I still can't reproduce
s
Copy code
√ | Starting Server
   | √ | Setting Server Profile to [development]
   | √ | Loading CFConfig into server   | √ | Loading FusionReactor
but when I do
server status
it's stopped. with this info:
Copy code
frontend (stopped)  127.0.0.1:8080 --> /home/dealstream/current/
  CF Engine: lucee-light 5.3.10+120
  Last Started: 10-May-2023 19:08:29

  Last status message: 
  NOTE: Picked up JDK_JAVA_OPTIONS:  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.annotation=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.module=ALL-UNNAMED --add-opens=java.base/java.lang.ref=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.math=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.net.spi=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.channels=ALL-UNNAMED --add-opens=java.base/java.nio.channels.spi=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.nio.charset.spi=ALL-UNNAMED --add-opens=java.base/java.nio.file=ALL-UNNAMED --add-opens=java.base/java.nio.file.attribute=ALL-UNNAMED --add-opens=java.base/java.nio.file.spi=ALL-UNNAMED --add-opens=java.base/java.security=ALL-UNNAMED --add-opens=java.base/java.security.cert=ALL-UNNAMED --add-opens=java.base/java.security.interfaces=ALL-UNNAMED --add-opens=java.base/java.security.spec=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.text.spi=ALL-UNNAMED --add-opens=java.base/java.time=ALL-UNNAMED --add-opens=java.base/java.time.chrono=ALL-UNNAMED --add-opens=java.base/java.time.format=ALL-UNNAMED --add-opens=java.base/java.time.temporal=ALL-UNNAMED --add-opens=java.base/java.time.zone=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED --add-opens=java.base/java.util.function=ALL-UNNAMED --add-opens=java.base/java.util.jar=ALL-UNNAMED --add-opens=java.base/java.util.regex=ALL-UNNAMED --add-opens=java.base/java.util.spi=ALL-UNNAMED --add-opens=java.base/java.util.stream=ALL-UNNAMED --add-opens=java.base/java.util.zip=ALL-UNNAMED --add-opens=java.base/javax.crypto=ALL-UNNAMED --add-opens=java.base/javax.crypto.interfaces=ALL-UNNAMED --add-opens=java.base/javax.crypto.spec=ALL-UNNAMED --add-opens=java.base/javax.net=ALL-UNNAMED --add-opens=java.base/javax.net.ssl=ALL-UNNAMED --add-opens=java.base/javax.security.auth=ALL-UNNAMED --add-opens=java.base/javax.security.auth.callback=ALL-UNNAMED --add-opens=java.base/javax.security.auth.login=ALL-UNNAMED --add-opens=java.base/javax.security.auth.spi=ALL-UNNAMED --add-opens=java.base/javax.security.auth.x500=ALL-UNNAMED --add-opens=java.base/javax.security.cert=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED --add-opens=java.desktop/com.sun.java.swing.plaf.nimbus=ALL-UNNAMED --add-opens=java.desktop/com.sun.java.swing.plaf.motif=ALL-UNNAMED --add-opens=java.desktop/com.sun.java.swing.plaf.nimbus=ALL-UNNAMED --add-opens=java.desktop/com.sun.java.swing.plaf.windows=ALL-UNNAMED --add-opens=java.desktop/sun.java2d=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-opens=java.base/sun.security.rsa=ALL-UNNAMED --add-opens=java.base/sun.security.pkcs=ALL-UNNAMED --add-opens=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.security.util=ALL-UNNAMED --add-opens=java.base/sun.util.cldr=ALL-UNNAMED --add-opens=java.base/sun.util=ALL-UNNAMED --add-opens=java.base/sun.util.locale.provider=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED  --add-exports=java.desktop/sun.java2d=ALL-UNNAMED --add-exports=java.base/sun.util=ALL-UNNAMED
Not sure where I would look to see why it's failing to start. I'm used to looking in tomcat/catalina.out for this type of stuff.
(I'm in the process of migrating from nginx/tomcat based hosts to commandbox)
b
Do a
--console --verbose
start
Your start above was async, so if there was an error, you wouldn't see it
šŸ‘ 1
Also, those stupid Java args push all the useful text out of what we track
I've already adjusted that on the CommandBox bleeding edge
s
ok. it has to do with my webroot, which is also a symlink 😩
b
ugh
So you have a symlink inside of a symlink?
s
uh, yeah
/home/deploy/app/current
is a symlink to
/home/deploy/app/releases/20230510XXXXX
and inside that are a few symlinked files.
This is how Capistrano does deploys/rollbacks
b
Many a clever thing are done with symlinks, they can also screw up a lot of tools
s
i guess I probably need to orchestrate it to copy those files into an actual directory and use that as webroot instead. I had to do that with Tomcat too, because Tomcat does not pick up changes to symlinked directories.
b
I'm guessing somewhere in the globber, the symlinked path is getting used instead of the real path or vice versa and which is screwing up logic that expects the path to nest
s
I was hoping to avoid that but hey
b
If you can create some standalone repro case, I can see if it's possible to detect this in Globber
s
at this point I don't think the Globber thing is my issue anymore. If I end up orchestrating copying of all these files, .env won't be a symlink within a symlink anymore and putting that dotenvFile key back in would probably work. I'll let you know.
b
Well, that may work around the issue, but I'd also like Globber not to try on symlinks and fall on its face šŸ™‚
šŸ‘ 1
I tried creating a symlinked folder with another symlinked file inside of it, but I still can't repro
Of course, I'm on Windows, which may work differently than Linux in that regard
s
yep. I'm trying to figure out how to create a reproducible case for you
b
Oohh, I think I got it!
s
I mean, I'm failing to think of a way 🤣
b
listing the file was fine, but starting a server in that symlinked folder while trying to load that symlinked file has locked me up!
šŸŽÆ 1
Ugh, now I have to kill the CLI eye roll🤣
Well, what the heck. Now it's not hanging and I can't figure out what changed šŸ¤”
The absolute path to
.env
was getting expanded to
\\.env
which was clearly wrong, but after putting in some debugging and restarting the CLI, it's working now and expanding to the correct path
šŸ¤” 1
s
ha. I finally have the server standing up but I can't access it from where I am. I think probably because by default the server binds only to 127.0.0.1? I should add host="0.0.0.0", right? (trying to find this in the docs)
b
Yes
is this Docker or just a based
box server start
you're doing on your own?
Because our docker images automatically set the host to
0.0.0.0
for you
s
on my own
b
Ok, then you'll want to bind to 0.0.0.0 to hit the ports externally
s
I hope to eventually move to docker, thus the config path names. This is an intermediate step
b
Yeah, in docker, your rollback mechanism is the entire image you create
So it's just a matter of deploying the previous image version.
s
yep. looking forward to that. I have docker stuff working locally, just have to grok how best to deploy to AWS.
I'm so done with boutique servers
d
@seandaniels I’m doing something along these lines to run CF in docker on AWS - using cloud formation to script the infrastructure once you have an idea how it all goes together - a good starting point is https://aws.amazon.com/blogs/containers/running-adobe-coldfusion-applications-on-amazon-ecs-with-blue-green-deployments/
I went the whole hog and use code build to build and test the images into AWS ECR, code pipeline to then notice and deploy the changes using code deploy for the blue green deploy above - MUCH better then stand alone servers I think
s
I have another app I'm using this same approach for @dougcain. I find that deployments take a really long time though. I'm used to being able to push code changes in < 30 seconds, and the blue/green thing seems to take 5+ minutes, plus requires manual intervention to "cutover" once the new containers are deployed. Do you find the same or am I missing something?
Oh wait, interesting. This is a different article, and possibly approach than what I used. I will look into this! Thank you.
d
the blue green deployment usually takes a couple of minutes as we have a test traffic stage to test and prime the new container before full traffic is routed. The actual deployment has been seamless since I have been running it with no manual step. My main pain point is when we build the container and run our tests (2000+) before its deployed - this can take 30 minutes - due to the adobe cf compiler being dog slow when running test box - we have spent months trying to get Adobe to realise this is a problem and fix it šŸ˜ž We have had to adjust our expectations of deploying the production app to take the 30 minute build cycle into account - would be nice if it was quicker but its automated, known and testable.
i think @bdw429s has had a similar battle with Adobe and compiler issues
b
Yes, Adobe's compilers is dog slow, at least compared to Lucee's
I've run many tests which show in a first-boot scenario given a stock ColdBox app • Lucee will be serving requests in about 4 seconds • Adobe will be serving requests in around 60 seconds, maybe more.
It's all 100% compilation time (confirmed with fusionreactor stack traces)
The core of the issue is Adobe uses try/caught exceptions for flow control in their parser which is DOG slow because it's throwing and catching exceptions thousands and thousands of times as it tokenizes the code.
I spent several weeks going back and forth with an engineer to who told me to my face (over slack šŸ˜† ) that the code in the
coldfusion.compiler...
package was "JDK code" and adobe didn't control it. They refused to even acknowledge ownership of their code, let alone the issue.
Yes, I'm still bitter 😁
d
@bdw429s had a similar experience with a lot of test cases to no avail - @Mark Takata (Adobe) is this something your aware of? It makes a big difference when trying to use containers with CF on things like AWS fargate etc.