Shouldn't the TESTBOX runner only load CFCs that e...
# testing
a
Shouldn't the TESTBOX runner only load CFCs that extend testbox.system.BaseSpec? It seems to be loading all CFCs in the specified dir structure, and looking for tests in them. It's a trivial problem, but we have a a fixtures class adjacent to the test class that uses it for the sake of convenience, and that fixtures class is being listed in the test output as having no tests. Well... yes... but it's not a test spec class so it shouldn't even be looked at, should it? I had a look through Jira and didn't find anything touching on this.
a
I've seen this. I believe the only thing that gets filtered out are interfaces. I moved our fixtures into another package.
Back in the MXUnit days only files that ended in "Test.cfc" were run so you could mix and match.
a
the only thing that gets filtered out are interfaces
That seems like the wrong approach to me. Tests are intrinsically only in classes that extend BaseSpec, so that's what one should be looking for. Anything else be it interface, fixture etc are not tests, so should not be included.
I moved our fixtures into another package.
Yeah we normally do this. But in this case it's only expectation data that relates solely to the test spec in the same dir.
only files that ended in "Test.cfc" were run
That was actually my expectation here, and we have been diligently naming our test specs thus. This would be a better approach than just filtering interfaces. But not as good as only including classes that actually are test specs. Is the behaviour documented anywhere? I had a quick squizz but didn't find anything. But it I only spent a minute or so on it.
z
I have a heap of filtering rules I use for running the lucee test cases, including this one https://github.com/lucee/Lucee/blob/6.0/test/_testFilter.cfc
👍 1
j
@Adam Cameron I would suggest submitting a ticket. I think it would be a good feature for the
run
command to have a glob pattern for the CFC name ( e.g.
match="*Spec.cfc"
). It would only be a one-line change to use that variable: https://github.com/Ortus-Solutions/TestBox/blob/development/system/TestBox.cfc#L648
a
Sure. Was hoping to understand the decision behind it before I formalise "I think you're wrong" ;-)
j
Typically, the directory of test bundles just contains runnable test specs. Sometimes we scaffold those out during app development and
x
the tests out until we implement - so having the ability to show bundle files that don’t have any tests present can be helpful/clarifying during development ( e.g. - “Oops! I need to enable/write those tests!” ). In our case, our convention for CFC’s that support the specs but aren’t runnable is to put them in the
tests/resources
directory - this includes abstract specs that extend the base spec, and which are extended by child tests. I think if we implemented inheritance type checking it might cause problems down the road for any custom usage of TestBox. @lmajano would have to weigh in on that. Having a glob pattern that can be used in the directory list to find the specs seems like a more workable solution.
l
Technically you don't need to extend the BaseCase for it to be a testbale bundle @Adam Cameron. So technically I can't just look for the inheritance, because, any CFC can be treated as a test bundle. However, I do see your point about it. I guess since nobody has brought it up, we went with the least path of resistance: get it to work.
So whatever ideas come to mind to improve it, I am all ears
a
Supporting looking for something as the file suffix like 'Test.cfc' or 'Spec.cfc' would do for me.
a
@lmajano but, for argument's sake... if one doesn't extend the BaseSpec, one doesn't get any of the TestBox methods (
it
,
describe
, etc). So it can hardly be considered a TestBox TestCase. It's just "a file". I mean even if looked for the implementation of an interface that said "gotta have a
run
method" I could understand. But just globbing the dir and then rejecting stuff that doesn't seem to match isn't... as good as it could be, I think. But sure. Will raise a ticket. If however it breaks backwards compat, I don't really see the point?
l
But you do get them
It does virtual inheritance on the target objet
👍 1
Just like ColdBox handlers
they don't have to inherit from the core
Now, I do agree it's over eager right now
How about adding a globbing pattern like @jclausen suggested. Right now in the
init()
we have
bundles
we can include one called
bundlesPattern
👍 1
default to
*Test.cfc, *Spec.cfc
a
Sees legit. Will raise a ticket. Thanks for the discussion everyone.