This message was deleted.

Looking at other Test task such as C++ tests, it does not look as if it would be supported easily. But I hope someone with deeper insight in Gradle internal have some ideas for me. @Daz DeBoer perhaps you have any thoughts on this?

Hello. We've been given access to a beta with a way of importing JUnit XML formatted test results into Gradle Enterprise and we implemented it with great success on C# NUnit or VSTest files, along with Jest and Jasmine on the nodeJS side. All you have to do is to transform the results from the runner you are using into JUnit, and then add the appropriate declarations.

Thanks, that sounds interesting, a concern is that all of this is legacy code and a very old Gradle version. Results are already there since many years: "We run a command line tool that generates JUnit compatible test result files" so that should be an easy task. I was just digging into the TestResultProcessor which I think may be a key here in case I want to go the long route. I saw @melix has some history in that file.

@Jocce Nilsson

TestResultProcessor

is an internal API hence the lack of Javadoc. 😉

5.6.5

Would TestResultProcessor calls be enough to register the tests in G.E.?

They should be as long as you’re “doing it right”. However, we can’t provide official support for that, i.e. ensure it doesn’t break in future versions.

I see, using an internal API may not be the best. Luckily it is legacy code and no major steps are planned for it. The new tests are all running with JUnit5 and some internal extensions

You’re welcome!

I am still struggling a bit on which calls to actually make in order to propagate details to G.E. My first attempt was to make calls to only:

testResultProcessor.started() //test case
testResultProcessor.completed() //test case

But that failed to be registered even if the code ran fine. I also tried adding

testResultProcessor.started() //test suite
testResultProcessor.started() //test case
testResultProcessor.completed() //test case
testResultProcessor.completed() //test suite

but that failed on the final 'completed' with

java.lang.IllegalStateException: This operation (org.gradle.api.internal.tasks.testing.logging.TestCountLogger - Run tests) has already been completed.

@Jocce Nilsson As Emmanuel mentioned earlier in this thread, Gradle has developed a (as yet undocumented) beta feature that allows test results from JUnit XML files to be "uploaded" to a Build Scan.

ticket is up, 4101

I just replied with the docs for the Beta.

Too bad... this code that I am trying is a hack to get our legacy integration tests registered. All new tests are migrated to a new framework based on JUnit5 already. In legacy versions we are at most on gradle 5.6.5 with "com.gradle.build-scan" version "3.8.1". GE Current version: 2021.4.2 so that one is safe

It should work with Gradle 5.6.5 and

com.gradle.build-scan

3.8.1. Is that unclear from the document?

Alright, it states Gradle enterprise plugin, not build scan plugin. But if you think it works then I will give it a try. Thanks!

I’ll update our docs to clarify that. Thanks for the feedback!

Glad it's working for you @Jocce Nilsson

Thanks! Almost at least... we are using a custom/local version or JUnit XML, so there are some compatibility issues. Jenkins was able to read it but seems we are missing the stacktrace node so the importer fails. It is enough to get us started as the scans are now showing the result at least, then we can improve it later to include the error and the logs.

Yes, there's no real schema or standard for JUnit XML files, and we added the concept of a "dialect" to handle different variations. Are you able to share a sample of your JUnit XML file? (DM is fine) We try to make our

GENERIC

dialect work with as many external tools as possible.

sure