Is anyone else doing a massive Gradle 8 migration and getting wrecked by the new Task Dependency Validation being non-deterministic? We have ~4000 repos using Gradle and ~100 different plugins and are getting wrecked this every day, now that we have a big chunk of repos on Gradle 8. A new change in a plugin or repo Gradle code that introduces implicit dependencies can easily slip through plugin upgrade PRs, as CI happens to pass on the PR, then starts failing non-deterministically on the main branch/releases depending on the order tasks are scheduled in Gradle builds. Where as we’d hope we would just end up with PRs that can’t merge due to deterministically failing CI rather than breaking team’s repos/releases.

Looks like you should rework how you define those tasks.

yes, but that is somewhat orthogonal - we’re seeing this even with newly developed tasks. The result of writing code that introduces these implicit dependencies (which is going to happen - the interactions between plugin code and 1000s of repos with their own custom Gradle written over the last 10 years is hugely complex) should not result in these problems slipping through PR CI builds and failing later CI builds on the main branch/releases. The task dep validation in Gradle needs to be deterministic, not non-deterministic, for us to even have a hope of working at this scale.

But the problem is the task graph is different depending on what you are executing, maybe that is what causes you see the issue only on some circumstances

yes, that is root of the problem. If there is going to be a check for this, it needs to find the problem whichever order the tasks run

That mean spending time configuring things that are not going to be executed

there’s currently a perverse situation where the check only fails if the tasks ran in the correct order, and does not fail when they run in the wrong order - which is the problem the check is trying to find in the first place!

Yeah, I think a flag to allow easier migration is a valid use case

The problem is that this check is done just before a task is executed. That means that one time task1 can be faster than task2 and the other time task2 can be faster than task1 and you get a different result. A fix would probably be to move the check when tasks are scheduled (I think). I wonder if in the meantime one could write a check on their own, e.g. something like:

gradle.taskGraph.whenReady {
    allTasks.forEach {
        // println("${it.name}: ${it.outputs.files.files}: ${it.taskDependencies.getDependencies(it)}")
        // build task dependencies and check output conflicts
    }
}

and then you would run some task that depends on all tasks as:

./gradlew taskDependsOnAll --dry-run

But yeah, I am not sure if that is enough.

Probably not a full fix, but at least an improvement. You will never cover the case that task2 is depending on outputs of task1 but task1 is not executing at all, without breaking task-configuration avoidance. The current logic is like • if tasks are running in the correct order, there is the failure as it just works by accident • if tasks are running in the wrong order, there is no failure, but hopefully somehow else it is recognized that the result is wrong With the issue fixed / improved it will be • if tasks are running in the correct order, there is the failure as it just works by accident • if tasks are running in the wrong order, there is the failure as it does produce wrong result • if only the depending task is running, there is no failure, but hopefully somehow else it is recognized that the result is wrong This would at least make sure builds are deterministic and not failing due to task ordering. Do you really think moving the check to the real scheduling will work? I guess it might be like that as a task's outputs might only be determinable after it was executed. So maybe instead the check has to be done later, not earlier, as last action of a successful build where all task inputs and outputs are finally determined.

> Do you really think moving the check to the real scheduling will work? Basically once inputs/outputs are finalized we should be able to check it (so once configuration phase is done). > I guess it might be like that as a task's outputs might only be determinable after it was executed. Locations should be known before the execution, or is there any case that it is not true? You can techically still change output location at execution time if you are using setters, but then you might also have problems with caching. We should be more strict here, and not allow that (if you use Property types it's already forbidden). I mean, checking that after execution seems a bit weird to me, since you allow to poison possible valid outputs created before. But yeah, in the end that solution would also be an option. It is also possible that the ordering of tasks is not an issue here, but just that existence of outputs influence the check.