Hello, our application seems to be loosing data in the application scope. The server has a large number of the same application each in it's own app scope. Some are fine, others are not. I'm at a loss as to the problem. Could this be something CF is doing to free up memory because of insufficient memory or leaks?

Losing application variables or application variables are getting changed to unexpected values? Make sure each application has a unique application name and make sure locks are used when making application scope changes outside of the onApplicationStart method in Application.cfc. The most common problem I usually see is duplicate, or unspecified, application names.

No. As in suddenly going away. No Duplicate application names. Typical error: Element ENABLE_SSO is undefined in a CFML structure referenced as part of an expression.

Nothing tries to expire that data if it ages out, whatever that means? Any chance the process(es) that load it into each application failed to completely do that? Can you log that data immediately after loading, to verify, and ideally , check every so often to see if it's still there? Do you have FusionReactor, or some other way to see if the server is running out of memory?

Did the application time out? Are you calling applicationStop() anywhere in your code? Is this Adobe or Lucee? Is it possible the dynamic application name changed from one request to the other (like, perhaps the user switched domains and you use the domain the app namr).

No, nothing expires out the data. I can execute a reinit and it reloads the data into the app scope. We don't get any errors when the system loads the data into the scope. Do not have FusionReactor on the server at this time. This is Adobe, and we do not do applicationStop(). Application should be around 2 hours and 60 or 70 seconds, and is setup to reinit the data if it's expired or empty. This app has been in use since 2004 and upgraded through the years, we recently upgraded it from CF11 to CF23 about 8 months ago.

Did the issue appear with the upgrade?

Seems like it.. But, that's hard to say. May just be more frequent.

When some data is missing from an application, is the application scope completely empty? You said you don't see any errors during loading, but can you actually confirm that the data is actually there? Is it always particular applications?

To Dave's point, some logging may help. Add a try catch that logs the time this happens and some info like a dump of the app scope. Then you can see if there is a trend for when it happens and what keys are missing.

Dave, No, the app scope isn't completely empty. That's the strange part. The application.system_config (a struct under the application scope) will only contain partial data. Several of the keys are missing when this issue happens. This is a single code base, running multiple application instances, so, it'll hit one or two applications seemingly randomly. I'm tempted to set a key count threshold and if the app detects that the keys in that structure are below a specific number, it assumes it's corrupt, and reloads it from database..

Is it possible there is code doing a partial update to the application settings but setting them as a full scope replacement instead of individual keys.

application = {
    bar = "setting",
    foo = "another setting"
}

instead of

application.bar = "setting";
application.foo = "another setting";

Or maybe compare the keys that exist with what you expect, if not the same, log what IS there and reload.

Were any keys being loaded via an implicit scope lookup?

If you do that each time something updates that info, you may be able to suss out the cause.

When you dump data, are all the values there?

Normally yes. But when this issue happens. No.

If data is blank, the information is never getting to the function to be put into application scope. Where is data coming from? API? Db call? Hunt backwards.

Let me say this again, more specifically. Right after you set each of those keys, compare the keys that exist in that application with what you expect, if not the same, log the keys that ARE there and reload, either just that app, or everything. So that's the first question: Was the data right when it was originally set in the application? If you don't discover anything after doing that for a while, do that same integrity check every day, hour, 5 minutes, etc. Once you see WHEN it happens, you can dig in to discover more about WHAT happened to actually cause it.

I know this is repeated many times, but things I would look for is 1. Make sure you aren't reusing an Application Name that is how your applications are unique 2. Look for potential inheritance and sub applications, maybe they stopped an Application from loading. 3. Look for Areas like APplication = StructNew() or Application = {}

@Mark Takata (Adobe) Data is being loaded from database, using code similar to what I posted a bit ago. It loops through and sets the applications scope. I don't see how it's possible that it can load only some items in the scope and miss other items. Because when I re-init, it works perfectly fine. @Dave Merrill You are correct, I'm not verifying the data after the initial load. I have to believe that was, because if it wasn't the app would never work. I'll see about interrogating the data, it's just hard to do so as this is on a production environment and I'm pressured to get the application back up and running ASAP. @Michael Schmidt 100% grantee not reusing an Application Name, each and every one is unique. It doesn't appear to be the entire Application structure, just part of it.. which is the strange part. Just like half of it missing.. This is a app that was written in 2004, and has been in production ever since, started off with CF6MX and updated all the way up through CF2023.