I keep running into issues with Lucee's handling of isNull() and it's handling of scope precedence. In a nutshell, if a local variable ends up being null and you use isNull() to check the variable, it will use scope precedence to check for another variable with the same name, which means parameters passed via the URL/FORM scope can change the behavior. Here's a simple example of the behavior: https://trycf.com/gist/926837625eb97f5cd4c79258cffc2ed7/lucee5?theme=monokai In Lucee 4.x and all versions of ACF, the dump returns

null

, but in Lucee 5 (and Railo) it returns

test

. Since the

modelId

is declared within the

local

scope, I would expect the

isNull(modelId)

to return true if

var modelId

exists in the function. However, you have to explicitly specify the

local

scope to get it to work (i.e.

isNull(local.modelId)

). I opened up a Lucee issue about this in May 2021, but there's been not traction: https://luceeserver.atlassian.net/browse/LDEV-3523 Am I wrong? Is this the behavior everyone else would expect? I see it as a real problem because any UDF in which you're using

isNull()

to check a local variable could end up returning drastically different results than what you're expected.

My intuition says that the variable would be 'resolved' by looking through the scopes before passing to isNull, so this would be expected to me.

it's an interesting side effect of cf's 'check all scopes in the following order to resolve the variable before saying it's undefined'. however, i would argue the side effect you're seeing is due in no small part to how your function is written, specifically it has no arguments and is setting itself up for scope cascade resolution by referencing an unscoped variable reference. i think you'd have to write your own udf to wrap the isNull check and explicitly check local and any other scopes you may want to consider

Just looking through some popular CFML open source libraries, I see places where this issue would be troublesome: https://github.com/framework-one/di1/blob/46662b51b8798a67bd2b42bc2e339393e28e187b/ioc.cfc#L885 https://github.com/framework-one/fw1/blob/7508da93998c427f51bcb00116d5cb521c73e795/framework/one.cfc#L1879 https://github.com/ColdBox/coldbox-platform/blob/fbb1bd689a90e1ed01f4f2bd9eb26069d40a0d43/system/async/tasks/FutureTask.cfc#L73 https://github.com/ColdBox/coldbox-platform/blob/fbb1bd689a90e1ed01f4f2bd9eb26069d40a0d43/system/RestHandler.cfc#L110 https://github.com/ColdBox/coldbox-platform/blob/17cb4247848e701660ff0d61d842c142464ecf2a/system/web/context/Response.cfc#L218

@dswitzer To me, this is just a reason why you should always be putting the scope along with your isNull() check.

isNull(local.modelId)

and your problem is solved.

would have to agree. in exchange for terse code you open yourself up to the cascade, i don't see how it could be otherwise

It would be interesting to hear one of their engineers justify why they don't perform the standard CFML scope lookup, but it's probably dur to some crappy old code inside the ACF engine making assumptions about what you wanted.

the examples you posted just illustrate how common that tradeoff is made

I would suspect not

personally, i guess i'd just assume some scope cascade will occur if it's an unscoped reference, so i don't see it as a bug per se. but i can get behind a change that if it is an unscoped reference within a function that isNull should just check local and stop (unsure if arguments should also be checked in this case, but that's an argument for the cascade behavior to be left as is)

I just checked the wording of the modern scope setting real quick to confirm if it applies only to setting a variable or also to accessing a variable

Defines how the local scope of a function is invoked when a variable with no scope definition is used.
-- Modern - the local scope is always invoked
-- Classic (CFML Default) - the local scope is only invoked when the key already exists in it

@bdw429s I will say in a very cursory glance, I found some Coldbox code that is affected by this issue. While scoping the check does solve the issue, based upon what you quoted in the docs, it's not the behavior I was expecting. That could because it's now how ACF works and that's what I've been working in for 25+ years. simple smile My expectation is that if a

local

variable is declared, then scope cascading should stop. I suspect there's a lot of CFML code that would be affected by this issue based on some searches on Github.

But you're forgetting how CFML treats nulls. A null values doesn't exist. So it makes perfect sense that it would be ignored.

The behavior doesn't match ACF and it appears a lot of devs aren't expecting the behavior.

ACF is wrong ¯\_(ツ)_/¯

I guess I disagree. var x = nullValue(); isNull(x); I would expect that to be

true

. The variable

x

was scoped and is

null

. It will return

true

unless

x

exists in a different scope. I think at a bare minimum the Lucee docs for isNull() should state you should explicitly declare the

local

scope if you do not want cascading scope check to occur. https://docs.lucee.org/reference/functions/isnull.html

The fact that Adobe has a bug where it doesn't keep looking is undocumented at best, and a lucky break.

The fact that ColdBox, FW/1, DI/1, etc all could be bitten by this issue is concerning.

The variable 

x

 was scoped

No it isn't scoped. I'm unclear why you would say that.

isNull(x);

☝️ That is not a scoped variable. The connection you have to the variable of the same name in the

local

scope is coincidental

var x

scopes to the

local

scope.

The fact that ColdBox, FW/1, DI/1, etc all could be bitten by this issue is concerning.

Programming is hard. We have to be careful. Just because people have run afoul of a language design, doesn't necessarily mean it's bad. This only happened a couple times in ColdBox, and we fixed it. Case close. We learned our lesson.

Unless you fixed it within the last few minutes, there are still issues in the Coldbox Github (see my links above)

x

is not an absolute canonical reference to a specific variable. it's a guess for the CF engine to go track down for you

Are argument comes down to the definition of

defined

. When I see

var x = nullValue();

the variable

x

is defined, but the value is

null

. The

isNull()

function isn't supposed to be checking if a variable has value, but if the value is

null

. So the fact that

var x

exists and is

null

it should return

true

.

I think at a bare minimum the Lucee docs for isNull() should state you should explicitly declare the 

local

 scope if you do not want cascading scope check to occur.

Scope hunting is a core part of CFML and Lucee is the most consistent engine. Why should Lucee tell us, "Oh, but the way, we're still following the spec here"? That should be the expectation, not the exception. Instead, I think the Adobe docs should be the ones with the note saying, "oh by the way, we're throwing the spec under the bus here and ignoring it in some cases"

And that's my fear. I fixed the issues now, but I can see them cropping up again.

This does not seem to be the expected behavior by many.

Dunno what to tell you. It's expected to me and it makes perfect since given the rules of CFML.

But this isn't scope hunting. This is check a variable to see if it's

null

. As soon as it find a variable which is defined and is

null

, then it should return

true

. It should check

local.x

first, see it exists and is

null

and then return

true

.

This is the nature of CF's loose scoping mechanics and questionable decisions about nulls not existing. I don't always care for the monster if created, but Lucee is behaving quite consistently from my perspective.

But alas,

null

does exist

A null variable still doesn't exist unless you've enabled full null support

The function isn't

isNotNull()

, it's

isNull()

. That should return

true

for a variable which contains a Java null value.

I think it would be wrong for isNUll() to return true yet still be able to dump out the variable and get a value! That would make no sense

My example code wouldn't do that. It would be printing

null

.

you can pass it around

How? A null variable cannot be passed directly or you will receive an error as shown here https://trycf.com/gist/7a1198e6e5fddfcc885e9a081e2c96e5/lucee5?theme=monokai

My example code wouldn't do that. It would be printing 

null

.

Lol, take this example for instance. You want CFML to work this way, which would be bonkers

if( isNull(modelId) ) {
	    writeDump( modelId )
	}

☝️ In a well-designed language in which nulls don't exist, that should never be able to run without erroring.

There should be a difference between isNull() and isDefined()

Lucee resolves

modelID

in the same manner regardless of what it's going to do with the value afterward. The fact that it's being passed to

isNUll()

does not change how the engine resolves it.

And that's where I see the problem. Because the purpose of the function is to check if a variable's value is

null

. If there is a variable in the scope chain which is

null

, then it should return

true

. Not just ignore it. That's what ACF does.

There should be a difference between isNull() and isDefined()

This is perhaps another discussion entirely... The main difference here being that the incoming variable is encased in a string so the indefined method is allowed to resolve it internally, whereas the arg to isNull is evaluated before isNull is ever called

Would you agree that the Lucee docs for

isNull()

should talk about explicitly scoping variables in a UDF?

No, not really

Thanks for taking the time to discuss your opinion on the matter. I appreciate it.

Here's a bit simpler example that shows the basic issue Adobe CF has

url.modelId = "test";    
variables.modelID = javaCast('null','');

// If modelID can be found having a value
writeDump( modelId )
// Then this must return false
writeDump( isNull(modelId) )

My recollection was that

isNull()

was created just for this purpose, because Java libs often return

null

values and they wanted a way to check if a local function contained a null to avoid scope bleeds.

Sort of. isNUll was added in CF9 with the addition of Hibernate since Hibernate returned null values for not found entities and CFML needed a way to detect if a variable was null.

But the purpose of isDefined() is different. For example if you take your code example and add the isDefined() check, it return

yes

: url.modelId = "test"; variables.modelID = javaCast('null',''); // If modelID has a value writeDump( modelId ); // Then this must return false writeDump( isNull(modelId) ); writeDump( isDefined("modelId") );

It's worth noting, Lucee has always treated

isNull()

and

isDefined()

nearly the same, which I thought made sense.

@bdw429s, that's my point about isNull() w/the ORM. It's supposed to explicitly check if a value is

null

.

But the purpose of isDefined() is different

How? Why? By definition in CFML, null means not defined. That literally ties the definition of defined-ness to null-ness. Why would they be different?

We're just going round and round. We disagree. That's okay. I could be wrong. That's okay too. That's why I asked the question, because the ticket has no traction.

When you call a BIF, any BIF.

myBIF( myVar )

the

myVar

is first found using the CFML rules of scope hunting and what makes a variable defined. Then, and only then, is the resulting value passed to

myBIF()

I'm seeing what you're saying. I just disagree. As soon as the scope checking finds that

local.x

is

null

it should stop and return

true

.

As soon as the scope checking finds that 

local.x

 is 

null

 it should stop and return 

true

.

According to whom? Null mean not-defined in CFML and has since I can remember. Why should that change now?

Anyway, thanks for taking the time to respond!

You're just making up new rules for CFML, lol

I guess I've just been playing by the ACF rules too long. 😄

it should stop and return 

true

.

☝️ And again, you're conflating the action of the isNUll method with the resolution of the variable

cfml is quirky with structs, a struct value which is null, structKeyExists says yes, but then it errors

I've always hated that one.

hmmm, so i figured that should be documented on lucee docs, but now I'm confused https://trycf.com/gist/9fc260fc781d3361b623fe51682cadc2/lucee5?theme=monokai

I guess it was only

structKeyList()

that showed the null keys.

but not b4 echo() and dump() and updating their loader revision (lol)

And just to circle back... https://ortussolutions.atlassian.net/browse/COLDBOX-1097

@bdw429s I'm glad something good came out (re: DBOX-1097) of this thread (for those willing to keep up with the it!)

Adobe also moved my ticket to "bug verified" and "to fix" 🙂

My take on this: Historically, before "null" was a concept that CFML really acknowledged, it conflated "define`d-ness" with "null-ness". It considered that a variable was not even defined if it didn't have a value. This was never correct, but it it what it is.

myVariableWithNullValue = javaCast("null", "")

myVariableWithNullValue

very much is defined there. You can see it. Being defined. It just has "no value" (in the context of CFML not understanding null). This was always two different things, and CFML messed it up. --- Once CFML acknowledged null is actually a thing, part of that ought to have been to revise the behaviour of the distinction between "define`d-ness" and "null-ness". This would necessarily be a breaking change. So be it: it's the price the language pays for having previously poor design. Using pseudocode (not intended to be pseudoCFML, don't read anything into the similarity between some CFML functions):

// myProgram.notCfml

isDefined(notEverMentionedPreviously) // false
isNull(notEverMentionedPreviously) // error


var definedButNoValue
isDefined(definedButNoValue) // true
isNull(definedButNoValue) // true


var definedWithExplicitNullValue = null
isDefined(definedWithExplicitNullValue) // true
isNull(definedWithExplicitNullValue) // true


var definedWithExplicitNonNullValue = "some value"
isDefined(definedWithExplicitNonNullValue) // true
isNull(definedWithExplicitNonNullValue) // false

To that end, I'm with @dswitzer: scope-look-up should not come in to play here. In his example

modelId

is defined in that function, so there's no need to go looking elsewhere for it. It is irrelevant what its value is; that should not come into it. I cannot test on Lucee5 ATM cos it seems to be crocked on trycf, but this also seems like a regression in Lucee5, if the Lucee4 behaviour was a) different; b) in-keeping with CF's behaviour; c) which is behaviour that makes more sense anyhow. Why did the behaviour change (sorry if this was explained, it's a long thread and I skimmed)?

If Adobe were to "go back to the drawing" board and re-think how they treated nulls and scope lookups, I'd have no issues with your proposals above. Your suggested behavior however, would not be consistent with the current design of CFML. As much fun as it is to dream about what CF could have become in an alternate universe, I'd just be happy getting Adobe to minimally be consistent with their own existing design decisions 🙂

Eeek, @bdw429s... I am having a horrible nightmare to the effect of "ColdFusion hasn't actually implemented full null support yet, has it?". I overlooked that.

Yeah, they added it though I've never really played with it. Not on ACF anyway.

It's quite difficult for most people to try out the setting if they rely on any 3rd part libs at all, most of which don't support it. And without the option to enable it at a CFC level, it's still mostly all-or-nothing even at the application level.

Ah maybe @abram has it switched off.

Yeah, trycf doesn't have it on for either Lucee or Adobe

fuck sake. both vendors should have just pissed or got off the pot with this. "Breaking change in [insert next major version number here]: null handling has been improved but it is a breaking change" This switch it on / switch it off thing is just ballocks. It basically means it's unusable in a shared-code situation.

Correct

What do you guys say "we don't support it switched on, so just don't"? Or have you wasted time and effort making it work in all cases?

Who is "you guys" in that sentence?

*Box

We support it in ColdBox, to the best we can (we don't tend to use it ourselves so rely on bug reports if we've broken it).

I think you should just post in the readme.md "not testing with null support switched on. Caveat emptor", and move on.

We don't use the

null

keyword anyhere since that's not backwards compat, but we just don't rely on things like

strcutKeyExists()

in our code, opting to use

!isnull()

since that works in both cases

Not such a hardship then.