This message was deleted.
# community-support
s
This message was deleted.
j
Can't remember where I found this but I have this in my setup:
test.maxParallelForks = System.getProperty("maxTestParallelForks")?.toInt() ?: ((Runtime.getRuntime().availableProcessors() / 2).takeIf { it > 0 } ?: 1)
v
That only works in single project build though, not in multi-project. It heavily depends on which parallelism we talk about. There is at least • parallel execution built into JUnit that can even run test methods in parallel • parallel execution using multiple test workers for one test task • parallel execution of test tasks in different projects of a multi-project build • parallel execution of test tasks within the same project
d
(gradle worker parallelism in a multi-project + junit test class parallelism) coordination is what I refer to above.
v
What do you mean exactly when you say "JUnit test class parallelism", because all mentioned points run test classes in parallel
d
Copy code
systemProperty("junit.jupiter.execution.parallel.enabled", "true")
		systemProperty("junit.jupiter.execution.parallel.mode.default", "same_thread")
		systemProperty("junit.jupiter.execution.parallel.mode.classes.default", "concurrent")
for all tests, and
org.gradle.parallel=true
Which I expect overtaxes the CPU with workers=$cpucount, and within each worker, $cpucount number of concurrent tests (all tests using class lifecycle)
So I would like to have $cpucount number of tests running at any given time, no matter if one subproject has one single test class, or it has 200.
v
Unfortunately I don't think there is some easy way to do that, if any way at all.
d
Sounds lilke a very useful usecase with forever growing number of cores. Is it planned? Haven't found any issue for it.
v
I heavily doubt it, but I have no idea. It would need a coordination between multiple different VMs.
d
Yep it would.
No core deserves to be idle or context switched to death.
v
I guess if something like that will be supported maybe it has to be on JUnit side, not Gradle side, so that multiple JUnit instances running in different VMs can coordinate the thread count among them.
d
Thanks, checking with the junit ppl.
šŸ‘Œ 1
c
It seems a very interesting issue, keep us posted please
šŸ‘ 1
d
One option would be to define an "aggregator" Gradle subproject that defines a Gradle
Test
task and contains a
@Suite
class that runs all tests. That subproject would then have dependencies on the test source sets of all other subprojects.
Sounds like something that could work even if it's not entirely clear how I would do that. So I guess instead of just running
./gradlew test
in CI, you would then run
./gradlew :ci:test
for example which would fork once and let junit engine schedule the tests across the number of cores available. The
@Suite
part looks trivial, but it would be cool if it was possible to automatically add all test source sets (with their corresponding
testImplementation
-deps) dependencies to the hypothetical
:ci:test
task. Sounds doable, but my gradle-fu is too weak to simply envision it. Any suggestions? Otherwise a hardcoded list of dependencies would be fairly low maintenance in my use case.
Having the other subprojects test source sets as deps wouldn't cause some weird rebuild if already built right?
v
You should probably define a feature variant for the test source sets and then you can just depend on those feature variants. Much like the
java-test-fixtures
plugin does it, just not with an extra source set. Or you actually use the
java-test-fixtures
plugin and move all your tests into that source set, but I think that is the less clean solution.
d
I am using java-test-fixtures for test fixtures shared among tests. Don't want to pollute the testfixtures with tests as that would increase compile time as I have a lot of dependencies between test projects and other projects testfixtures. It sounds like the solution suggested by junit would require me to refer to other projects test configurations as implementation dependencies, and possibly also add the superset of all subprojects
test*
-dependencies in the whole project to that
:ci:test
project?
v
I just told you how to do it. The test fixtures plugin was just an alternative.
Or you can probably also use the simplified method also detailed at https://docs.gradle.org/current/userguide/cross_project_publications.html#header
d
Hehe, trying to decipher this archwizard tongue you speak in šŸ™‚ Now that I think of it, it's how this jacoco convention collects its inputs, https://github.com/triplem/gradle-by-example/tree/58c6d92a824e3d0eb52248e2f1e47dfa2cbe371b/buildSrc/src/main/kotlin .. and I happen to have my build convention already imported in all bazillion subprojects already so just need to read and understand how that stuff works. I adapted the jacoco-stuff linked to my build without taking the time to get a deeper understanding of it a couple of months back, should have taken the time, but now is also a great time šŸ™‚
If I understand correctly I need to produce a jar for the other subprojects. I thus declared a "exportedTests" configuration that extends testImplementation, testRuntimeOnly etc and added a jar task to its artifacts. Not sure how to refer to it though. I usually use the project accessors like
testRuntimeOnly(projects.some.subprojectFoo)
but then I can't seem to declare a specific configuration, and
testRuntimeOnly(project(":some:subprojectFoo", configuration = "exportedTests"))
leads to
Project with path ':some:subprojectFoo' could not be found in project ':some:thetests'.
what would be the correct way to refer to the other projects tests? Adding the jars to the exportedTests like:
Copy code
val testJar = register("testJar", Jar::class.java) {
		archiveClassifier.set("test")
		from(sourceSets.test.map { it.output })
	}

	artifacts {
		add("exportedTests", testJar)
	}
v
Did you read the documentation about feature variants? https://docs.gradle.org/current/userguide/feature_variants.html As I said, I think the easiest would be if you just define a feature variant for the test source set and then depend on that feature variant. If you want to go the custom consumable configuration route and use it with project accessors, you probably need something like
testImplementation(projects.some.subprojectFoo) { targetConfiguration = "exportedTests" }
d
No I checked the "Sharing outputs between projects" link you posted. So src/test/kotlin/* can be part of two feature variants and still only be compiled once? Thinking that when running under IDEA the use case would be plain old simple run tests for a subproject, then perhaps run allthe:tests before push which would use whatever solution to get the junit suit to reference all tests in all subprojects and run within the same fork.
Using the
testRuntimeOnly(projects.foo.somesubproject) { targetConfiguration = "exportedTests" }
strategy for referencing the other artifacts had the same result as
project("...", configuration = "...")
... is that because they are sibling directories or that doesn't matter?
Is there any issue with how I register the exportedTests?
Copy code
val exportedTests by configurations.creating {
	isCanBeConsumed = true
	isCanBeResolved = false
	// If you want this configuration to share the same dependencies, otherwise omit this line
	extendsFrom(configurations["testImplementation"], configurations["testRuntimeOnly"])
}
Does the names have any significance here? https://docs.gradle.org/current/userguide/cross_project_publications.html#header ... The name picked is instrumented*Jars* and then later on, before being declared anywhere instrumented*Classpath* is used. Are those two magically related? It's first after the use of instrumentedClasspath that there comes a declaration in the documentation, is that wrong, or is instrumentedClasspath actually automatically declared by the former steps?
would be sweet if it was as simple as
testRuntimeOnly(projects.foo.subprojectA.tests)
or
.integrationTests
and poff the @Suite tagged test class would know of all the tests and all their deps etc without any need of creating jars or whatnot.
v
So src/test/kotlin/* can be part of two feature variants and still only be compiled once?
By default it is not part of a feature variant, otherwise you could simply use that one. But yes, you can have multiple feature variants for the same code and for example just have different dependencies, or you can have different feature variants that provide different code, many possibilities.
Using the
testRuntimeOnly(projects.foo.somesubproject) { targetConfiguration = "exportedTests" }
strategy for referencing the other artifacts had the same result as
project("...", configuration = "...")
Of course, it is just the project accessors syntax, but effectively the same. Didn't realize there was a problem with the latter.
is that because they are sibling directories or that doesn't matter?
That shouldn't matter, the other project should be found. Hard to say what is wrong there without an MCVE.
Is there any issue with how I register the exportedTests?
Seems fine from a first look. MCVE might help.
Does the names have any significance here?
No, they are freely choosable
Are those two magically related?
No, just what is configured.
It's first after the use of instrumentedClasspath that there comes a declaration in the documentation, is that wrong, or is instrumentedClasspath actually automatically declared by the former steps?
No, no automatism there. It just first shows how to declare the dpendency, but in the build script you actually should have the configuration created first.
without any need of creating jars or whatnot.
That's why I said you should use a feature variant. That makes the jar task declaring and so on for you implicitly.
d
Ok, thanks for your immense patience. Next attempt will be with feature variant.
šŸ‘Œ 1
If I declare as follows:
registerFeature("allTheTests") { usingSourceSet(sourceSets["test"]) }
no
allTheTests*
will be registered. If I change to
sourceSets["main"]
I will get
allTheTestsRuntimeOnly
and friends - but wrong sourceSet. Kind of tricky to google on, nobody does things like this.
yay, works
šŸ‘Œ 1
CI effect of this => old PR = 5m54s test time, new PR 2m47s test time, and should scale linearly with the amount of cores when no longer thrashing the runner.
šŸ‘Œ 1
....and the full suite, 8m32s => 3m22s šŸ”„
šŸ‘Œ 1
I think it would be valuable if the gradle documentation had some details on this kind of setup. Using junit concurrency and variants like this. Pretty sure there are quite a few either wasted cores, or overworked cores out there.
v
Feel free to open a feature request ticket that suggests it, there were no Gradle guys in this discussion and even if, it would probably need a feature request or PR. :-)
šŸ‘ 1