http://coldfusion.com logo
Join Slack
Powered by
We have a ColdFusion 2021 server that's been runni...
# adobe
a
We have a ColdFusion 2021 server that's been running like a champ for years and all of a sudden, an hour or so ago, it started showing this message in FusionReactor
Copy code
Waiting on lock <0x4f9cef99> (a java.util.concurrent.Semaphore$FairSync)
the server has been rebooted several times to no avail.
p
Windows/Linux?
any stacktrace
a
Copy code
"ajp-nio-127.0.0.1-8120-exec-24" Id=198 TIMED_WAITING on java.util.concurrent.Semaphore$FairSync@4f9cef99 
   java.lang.Thread.State: TIMED_WAITING
        at java.base@11.0.12/jdk.internal.misc.Unsafe.park(Native Method)
        - waiting on java.util.concurrent.Semaphore$FairSync@4f9cef99
        at java.base@11.0.12/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11.0.12/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1079)
        at java.base@11.0.12/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1369)
        at java.base@11.0.12/java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:415)
        at coldfusion.CfmServlet.lock(CfmServlet.java:311)
        at coldfusion.CfmServlet.service(CfmServlet.java:230)
        at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:311)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:46)
        at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at coldfusion.inspect.weinre.MobileDeviceDomInspectionFilter.doFilter(MobileDeviceDomInspectionFilter.java:57)
        at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:47)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
        at jdk.internal.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
        at java.base@11.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base@11.0.12/java.lang.reflect.Method.invoke(Method.java:566)
        at com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:134)
        at com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doNext(FusionReactorRequestHandler.java:698)
        at com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.doFusionRequest(FusionReactorRequestHandler.java:123)
        at com.intergral.fusionreactor.j2ee.filter.FusionReactorRequestHandler.handle(FusionReactorRequestHandler.java:736)
        at com.intergral.fusionreactor.j2ee.filter.FusionReactorCoreFilter.doFilter(FusionReactorCoreFilter.java:36)
        at jdk.internal.reflect.GeneratedMethodAccessor635.invoke(Unknown Source)
        at java.base@11.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base@11.0.12/java.lang.reflect.Method.invoke(Method.java:566)
        at com.intergral.fusionreactor.j2ee.filterchain.WrappedFilterChain.doFilter(WrappedFilterChain.java:71)
        at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)
        at java.base@11.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base@11.0.12/java.lang.reflect.Method.invoke(Method.java:566)
        at app//com.intergral.fusionreactor.agent.filter.FusionReactorStaticFilter.doFilter(FusionReactorStaticFilter.java:54)
        at app//com.intergral.fusionreactor.agent.pointcuts.NewFilterChainPointCut$1.invoke(NewFilterChainPointCut.java:50)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:377)
        at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:463)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        - locked org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@695828df
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base@11.0.12/java.lang.Thread.run(Thread.java:834)

   Locked ownable synchronizers: 
        - org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker <0x5c452afb>
Windows
p
Is the box maxed out on mem usage or cpu?
a
no, not even a little
b
I would guess your max requests is kicking in
Look in FR and see how many requests are running
p
yea thats what I was about to say as well
a
b
It would appear CF is throttling it
Look in the CF admin at the max requests setting
a
it's set to 150
it usually has much more traffic - thus he enormous memory.
b
For which type?
Isn't there something like threads, web requests, CFC requests, etc
a
b
I've never understood using a 3rd party service to share an image when you can just paste a screenshot directly into slack 🙂
😂 1
☝️ 1
Now, if you look at the running requests in FR, are they web requests or CFC requests?
Also, FR will show you the queued requests in CF as well
Under 
Metrics
> 
CF Metrics
Most people don't even know about this page
but it will show you how many of each type of request is running, queued, and timed out
Start there to confirm if CF is queuing requests
a
hmmm. it's completely blank
image.png
b
interesting-- and there's defo traffic hitting the site?
Where exactly did you see that stack trace you originally posted in fusionreactor?
a
yes
and yes
b
Also, did you check the types of requests running on the server in the activity page?
It's possible that cf metrics page is broken in CF or something. too bad
p
Almost seems to be something else on the machine possibly?
2 Views
Open in Slack
PreviousNext
Powered by