Ok Testing Folks! We have had this feature in the books for many years and it's now in alpha in the

be.

Welcome to the world of TestBox Modules!! Hierarchical Modules for Testing Pleasures: https://ortussolutions.atlassian.net/browse/TESTBOX-377 Please try it out and let me know how we can improve it!

New TestBox Release: https://www.ortussolutions.com/blog/testbox-v53-release

Bit of a cross post from the box slack so I apologise for anyone seeing this twice... I use a beanfactory (originally DI1 now Wirebox) to wire up dependancies in my unit tests. This is not a ColdBox app. So I setup the beanfactory with mocks for each of the dependancies and then get the wired up subject under test from the beanfactory. This has worked great for years (I like it as it's also testing the syntax to wire in the dependancies is correct), however with Wirebox 7 it no longer works. So how do people wire up dependancies? Do you just manually call the setters (with mocks)?

For some strange reason, all my tests that existing in a subdirectory of

/tests

now respond with...

@tony has left the channel

@Dirk has left the channel

Hi, When running tests, in testbox, is it possible out of the box to get the test runner to fail fast / early rather than having to wait for all the tests to run ? Maybe a URL argument you can pass to the test runner ?

👋 Hello, team!

Question.. I am trying to write a testbox testing suite that will be based on CFLint results. Has anyone done this already? I hate to reinvent the wheel here.. I am not very familiar with testbox specs, but from what I understand I would need at least one per file that has issues raised by CFLint. I can build all this out be again really don't like doing this the hard way 🙂 Does anyone have some input on this idea?

I'm getting weird results in one of my tests. I'm passing a value into a function, it gets processed and wrapped up in some XML, and in this particular case, I'm trying to verify that it came through the processing step unedited. So I have

var testValue = '{abc:"def"}';
var actualValue = myComponent.getFormattedText(testValue);

expect(actualValue).toInclude(EncodeForXML(testValue));

And that's "failing" because

The needle [&#x7b;abc&#x3a;&#x22;def&#x22;&#x7d;] was not found in [<?xml version="1.0" encoding="UTF-8"?><record><datum key="id">{abc:"def"}</datum></record>

I think it's worth noting that getFormattedText also runs testValue through EncodeForXML under the hood. So it's especially a mystery to me why it comes out nice on one side, but not on the other. Why is EncodeForXML feeling a need to encode the braces and quotes in one string, but not the other, causing them to not match?

@Adam Cameron et al, I'm totally new to TestBox, haven't even downloaded it, just thinking about how it would fit in our dev workflow, so excuse the Fred Flintstone questions here. I get the general pattern for spec'ing out tests, but what actually runs them? We don't have a build process per se as of now. Code gets committed to source control, pushed up the hierarchy to the production stream and workspace, then that workspace gets updated, so it gets the current state of that code. On the way, it passes through the stage stream and workspace, where user testing happens. TestBox gets talked about as automated testing, so I'm thinking some sort of scheduled process would run our suite of tests, and complain somehow if any tests failed. How is that usually orchestrated?

Hey all,pretty new to CFML and just started working on an application using Lucee (no frameworks or coldbox)/Command box/MySQL containerized with docker. I'd like to attempt to add tests with TestBox, but am struggling getting a clear idea of not only how to add the testing setup, but how to write some of the tests. The docker containers are: • Command box v4 to start Lucee server • Httpd v2 to serve the web app • MySQL v8 How can I setup testing? The app currently has zero tests or any infra for it. Do I just add a run command to install TestBox in the command box Docker file? Any thing I need to watch out for setting up config for container environments (I assume I need to expose a port for the test UI, etc..)? Just trying to get my bearings on this and would appreciate pointers if someone else was in a similar scenario. Also is there up to date resources or video tutorials for writing tests for business/query logic in apps that don't use coldbox?

Some of our tests for a self written validation framework sometimes pass and sometimes fail (approx. 5% of all times), So I thought I'd run the tests multiple times in a loop and react to fails (e.g. display "2 out of 100 test iterations failed", log the results, etc) I thought listeners are the way to go, so I created a listener cfc that dumps some output to a file in

onSpecEnd()

, I tested the method by calling it directly and the file was created as expected. I then went on and passed the listener CFC to testbox:

testbox.run( callbacks=myLstener )

, but, it seems in this case

onSpecEnd()

is not executed. So maybe I am misunderstanding the concept and purpose of listeners? Unfortunately the Testbox docs don't say much about it, can somebody please tell me where to find more information or how to tackle my initial problem (tests sometimes failing, sometimes passing)? Thank you!

I'm taking the plunge and just installed TestBox into a legacy CFML app (no coldbox or frameworks used), based on very helpful information from people here a while ago, but I'm a little lost still. I gitignored the TestBox directory created and created a new tests folder at the top level of my project where I want my tests split into nice organized files and folders in. I'm reading about a runner.cfm and creating mappings to tell TestBox where to find and run tests, but it looks like there is already a runner.cfm and a tests folder in the gitignored TestBox directory, so I'm confused - do I move those out of the gitignored testbox folder and modify the runner somehow? Or do I create a new runner file that I add custom mappings to ..or? Not sure how to get everything setup, but I got as far as 'box install testbox' 🤘

Pretty new to TestBox, but I'm trying to use it from a BoxLang class. I'm getting a

Function 'getComponentMetadata' not found

error for a lot of things I'm doing so this might be a generic error.

private function runTests() {
    var javaSystem = createObject( "java", "java.lang.System" );
    var exerciseRoot = javaSystem.getProperty( "user.dir" );
    var testCFC = ArrayFirst( DirectoryList( path = exerciseRoot, filter = "*Test.cfc" ) );
    var testRunner = new testbox.system.TestBox( testCFC );
    var testData = testRunner.runRaw().getMemento();
    println( testData );
    // Set proper exit code
    if( testData.totalFail || testData.totalError ) {
        setExitCode( 1 );
    }
}

I'm setting up a test to test a component and need to initialize some application wide variables for creating an instance of the .cfc component. I am using

beforeAll()

to instantiate the component. My question is what is recommended for initializing an environment (populating and setting up needed global settings. i.e. found in Application.cfc) in the tests? Example:

component extends="testbox.system.BaseSpec" {
	function beforeAll(){

		// I could setup things like this, but is there a better way?
        application.dsn = "myDsn";

		// Is there a way to reference what is set in Application.cfc - Can I just reuse what is already set in Application.cfc? (there are a lot of global vars set...)		
        // should I replicate Application.cfc var settings in the tests/Application.cfc (this feels wrong and maintenance problem prone)


		// instantiate my component with necessary dependencies injected
		myComponent = createObject('component', 'some.cool.Component').init(
			dsn=application.dsn,
			someKey=application.someKey,
			otherInjectedComponent=application.instantiatedOtherComponent,
			//etc....
		)

	}

	function afterAll(){
		// Do I need to do anything here?
	}

	describe("component tests", function() {
		it("can compile", function() {
            // referencing the component instance I created in beforeAll
            expect(myComponent).toBeComponent();
		});
	});
}

I'm working on integration tests using the Database with TestBox and have a working test (even wrapped in a transaction so changes are rolled back automatically), but I keep seeing references to calling

super.beforeAll()

or

super.afterAll()

in test bundles referenced in documentation and examples I'm reading that inherit from a Base Test Class (that inherits from the Testbox Base Spec). Can someone explain to me what this means: "This also helps parent classes enforce their setup methods are called by annotating the methods with

@beforeAll

. No more forgetting to call

super.beforeAll()

!" - from this page: https://testbox.ortusbooks.com/in-depth/life-cycle-methods/annotations I am not doing this (not calling super.beforeAll etc, anywhere), but my tests are working fine. Am I missing something or should I be calling that somewhere? When and where do you need to call the

super.[LifeCycleMethods]()

? My class inheritance: testbox.system.BaseSpec -> myBaseTestClass (initializes application variables/test setup and does not call super.beforeEach etc.)

Using testbox to test a component, is there a way to assert that a component has a method? For example to make sure that a model component has a

setSomeProp()

method that exists on it?

Writing an integration test with TestBox, I want to completely mock an AWS S3 service (.cfc file in the app that reaches out to S3). Is this the right way to do it?

mockBox.createStub(extends="path.to.my.s3service");

That will create a fake S3 service that mirrors the interface etc of my service but not actually do anything, right? Do I need to call init() on that as well to instantiate it?

I have been testing .cfc components with TestBox but now would like to test a view (.cfm file). I know I can use an include statement to bring in a view file, but how do I reference elements and assert they exist or not on the page?

Tried to upgrade to TestBox 5.4 (we're on 5.0). Found a bug in it. Goes to show how good TestBox is at finding bugs... 😉

Is there a way to mock a private method in TestBox? I am testing a public method that calls a private method doing stuff I don't want it to do during a test. I just want to mock the private method being called by the sut to return a simple string. My understanding is that you can't mock private methods in the way I'm doing currently with prepare mock and

.$("myprivatemethod", "return this")

@seancorfield has left the channel

I have great news. TestBox@BE is now fully certified on BoxLang all 335 tests are green and ready to go and have been added to CI The docs also are being completely rewritten for TestBox: testbox.ortusbooks.com to support all the new features and BoxLang support You can also write all your tests and specs in BoxLang now Here is a sneek peek of what's in the next release of TestBox v6!

New Feature
TESTBOX-391 MockBox converted to script

TESTBOX-392 BoxLang classes support

TESTBOX-393 New environment helpers to do skip detections or anything you see fit: isAdobe, isLucee, isBoxLang, isWindows, isMac, isLinux

TESTBOX-394 new `test(), xtest(), ftest()` alias for more natuarl testing

TESTBOX-397 debug() get's two new arguments: label and showUDFs

TESTBOX-398 DisplayName on a bundle now shows up in the reports

TESTBOX-399 xUnit new annotation for @DisplayName so it can show instead of the function name

Improvement
TESTBOX-289 showUDFs = false option with debug()

TESTBOX-331 TextReporter doesn't correctly support testBundles URL param

TESTBOX-395 adding missing focused argument to spec methods

TESTBOX-396 Generating a repeatable id for specs to track them better in future UIs

Bug
TESTBOX-123 If test spec descriptor contains a comma, it can not be drilled down to run that one spec directly

TESTBOX-338 describe handler in non-called test classes being executed

I have great news to report. The new TestBox BE has a new BoxLang ONLY runner now. Which allows you to run your tests from CLI mode ONLY. This is fast and great, so we can do VSCode integration like any other language. Coming soon. YOu can even use the

--verbose

flag and it will stream back to you the progress of the executions of your tests, bundles and suites. Here are the docs: https://testbox.ortusbooks.com/getting-started/running-tests/boxlang-runner I would love some testing of this new feature. Just

install testbox@be

and use the

./testbox/bin/run

script

Having a weird issue running install testbox with latest version of commandBox I am missing system files, only getting the modules directory.

TestBox v6 Released! https://www.ortussolutions.com/blog/testbox-v600-release TestBox 6.x is a major update to the testing framework, introducing key improvements like full support for BoxLang 💻. This release lets users write tests, specs, and harnesses directly in BoxLang ✍️. A new BoxLang CLI runner simplifies running specs and bundles without needing a web server 🖥️. Additionally, the CLI now detects BoxLang projects, adds headless web server testing with the bx-web-support module 🌐, and includes dynamic assertion methods for cleaner test writing ✨. Significant new features include environment helpers 🌍, display name annotations 🏷️, and improved debugging 🛠️ and matcher functions 🧩.

We have also revamped the ENTIRE documentation for TestBox. We have completely changed the menu, flow and revamped the entire book for this v6 release. Enjoy!

🚀 TestBox 6.1.0 is here! 📢 Read all about it: https://www.ortussolutions.com/blog/testbox-v610-release We’re thrilled to announce native support for BoxLang, now fully functional without compatibility mode—bringing dynamic testing on the JVM to the next level! 🌟 What’s New? ✅ Brand-new website: www.testbox.run ✅ Revamped docs—easier to navigate, feedback welcome! ✅ cbMockData 4.0.0: A major evolution of MockDataCFC with native BoxLang support. 🔧 Release Highlights • Seamless BoxLang support without compat mode • Bug fixes and enhanced struct-like object testing ✨ Coming Soon: bx-testbox, a native TestBox for BoxLang & VSCode integration!

TestBox 6.3.0 is LIVE! 🎉 Exciting news! We've released TestBox 6.3.0 with: • 🌈 Colorized BoxLang Reporter! • ⚡️ Direct CLI BoxLang execution! • 🛠️ Bug fixes & improvements! Update now for a better testing experience! 🧪 Details: https://www.ortussolutions.com/blog/testbox-v630-release