This message was deleted.

lowering

--max-workers

could help

we are using 8.6 and test tasks have set a high Xmx maxHeapSize

I can only share that I am facing similar issues when going full parallel with the config ache. The tricky thing is that you want to limit the tests, but not other tasks. So

--max-workers

is not hepful IMO. If you already parallelize the tests through JUnit, using an empty build service with

maxParallelUsages = 1

to limit how many test tasks run in parallel sounds like a good solution (we do that). And also keep

maxParallelForks

at 1 so that there is only one worker per test task.

I used the fake service approach to limit it. I talked with some Gradle engs yesterday, it's under the radar but not started yet. The 8.1 memory management improvements are only about not keeping idle workers when there are not enough memory

probably depends on project structure in practice (e.g. with 2 modules, test forking is definitely more parallelizable)

I believe it will become very painful when Gradle enables CC by default

I think that will mostly happen to folks who did not activate "parallel" before as well. But for those there might be other surprises as well.

probably depends on project structure in practice

Yes. having seen a bunch of different projects in the past months, this is my experience. It depends a lot how you are testing and what tests are doing. I think this is to a large degree also a documentation and usability issue. I think like 90% of the users do not understand the different layers of parallelization: multiple tasks, one task with multiple worker VMs, multiple threads in a VM by JUnit5 parallelization, tests creating multiple threads themselves, tests starting additional processes themselves And this is also too complex to "just work". You can maybe improve some things in Gradle but it will never be able to automatically figure out what the best parallelization for your setup ist. Plus there is the issue of different contexts: Sometimes you want to use as much machine power as possible (CI agents). And sometimes you want to run it in the background and only use half of your machine (run all the tests in the background without making my laptop feel slow while I am in a Zoom call)

This build had

--no-parallel

precisely to handle the memory issue, but you are right. Multiple layers of parallelization + not well documented (tracking memory usage is not good either from gradle side). I agree it's complex, the question is how to help reducing the friction

this relates to https://github.com/gradle/gradle/issues/24253

Isn't the build service exactly for shared resources?

it is what I'm using, for example in native build tools: https://github.com/graalvm/native-build-tools/blob/565302248856ba3e140f8b562a7d5e7e78b16a82/native-gradle-plugin/src/main/java/org/graalvm/buildtools/gradle/NativeImageService.java#L57-L65

I'm happy that my case is just about my build, not a plugin