I have a project where we are downloading differen...
# community-support
r
I have a project where we are downloading different jdks to build distribution packages. I am currently trying to make the download and unpacking of the jdks into individual tasks (as opposed to single tasks that operate on all of them) to make them run in parallel as well as avoid needing to download jdks for all OSs when only building for one In my loop over the different jdk configurations (a property map), I register the download, verify, unzip, and package tasks. Download and Verify task type is from the download plugin. I get this error when trying to run a task that builds all packages (so needs to download all jdks):
Copy code
FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
A problem was found with the configuration of task ':verifyJavaRuntimeForMacos_arm_64Bit_Java23' (type 'Verify').
  - Gradle detected a problem with the following location: 'D:\Programming\aij\jres\zulu23.30.13-ca-crac-jdk23.0.1-macosx_aarch64.zip'.
    
    Reason: Task ':verifyJavaRuntimeForMacos_arm_64Bit_Java23' uses this output of task ':unzipJavaRuntimeForWindows_x86_64Bit_Java23' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed.
    
    Possible solutions:
      1. Declare task ':unzipJavaRuntimeForWindows_x86_64Bit_Java23' as an input of ':verifyJavaRuntimeForMacos_arm_64Bit_Java23'.
      2. Declare an explicit dependency on ':unzipJavaRuntimeForWindows_x86_64Bit_Java23' from ':verifyJavaRuntimeForMacos_arm_64Bit_Java23' using Task#dependsOn.
      3. Declare an explicit dependency on ':unzipJavaRuntimeForWindows_x86_64Bit_Java23' from ':verifyJavaRuntimeForMacos_arm_64Bit_Java23' using Task#mustRunAfter.
These are the unzip and verify tasks:
Copy code
tasks.register(verifyTaskName, Verify) {
	dependsOn downloadTaskName

	it.inputs.file(it.project.layout.projectDirectory.dir("jres").file(sysInfo.name))

	src it.project.layout.projectDirectory.dir("jres").file(sysInfo.name)
	algorithm 'MD5'
	checksum sysInfo.md5
}

tasks.register(unzipTaskName, Copy) {
	dependsOn verifyTaskName, deleteTaskName

	it.inputs.file(it.project.layout.projectDirectory.dir("jres").file(sysInfo.name))
	it.outputs.dir(it.project.layout.projectDirectory.dir("jres/${(sysInfo.name as String).replace("." +sysInfo.ext as String, "")}"))

	if (sysInfo.ext == "tar.gz") {
		from(tarTree(resources.gzip(it.project.layout.projectDirectory.dir("jres").file(sysInfo.name)))) {
			into ''
		}
	} else if (sysInfo.ext == "zip") {
		from(zipTree(it.project.layout.projectDirectory.dir("jres").file(sysInfo.name))) {
			into ''
		}
	} else {
		logger.error("Did not know how to handle " + sysInfo.ext + " for " + sys as String)
	}

	into 'jres'
}
The file names and task names are unique
v
Some general notes first: • I strongly recommend switching to Kotlin DSL. By now it is the default DSL, you immediately get type-safe build scripts, actually helpful error messages if you mess up the syntax, and amazingly better IDE support if you use a good IDE like IntelliJ IDEA or Android Studio. • Like in most cases you also here probably want a
Sync
task, not a
Copy
task, then you also do not need to depend on the
deleteTaskName
, assuming that it wipes away the old copy target. •
into ''
should be superfluous I think, you should be able to just remove it if I'm not wrong •
logger.error
will not make the task fail, so maybe you should throw an exception there • And explicit
dependsOn
where on the left-hand side is not a lifecycle task is a code smell and a sign that something is not done properly, most often it means that task outputs are not properly wired to task inputs which would bring necessary task dependencies automatically Regarding the error, the cause is one of the most typical problems you have when this error appears. You have tasks with overlapping outputs which is a very bad idea and should be avoided at practically any cost. (which is pretty low usually) Your download task downloads a file to
jres/<sysInfo.name>
, your verify task verifies the file at
jres/<sysInfo.name>
, your unzip task has
jres
as destination directory and thus all files in it as output files, including
jres/<sysInfo.name>
. So now the unzip task has
jres/<sysInfo.name>
as ouptut file (which is wrong of course) and the verify task has the same file as input, but there is no dependency or ordering constraint ensuring that the verify task runs after the unzip task (actually you have the opposite). Overlapping task outputs or polluting the outputs of another task is always a bad idea and can lead to serious problems. Each task should have dedicated outputs that are not shared with any other task. The solutions in that error message are unfortunately almost never helpful, as most often the problem is something different and that would only be symptom treatment if possible at all.
r
If I gave each task a unqiue subfolder in
jres/
would that resolve the issue?
dependsOn
is old, you're right in that I could probably remove it
deleteTask
is actually to work around a bug on mac where the JDK needs to be deleted before packaging with it a second time, though I should still make it a
Sync
task to get rid of the old files, though I'm not entirely sure how that would work when I want to keep the current packed files
Unique folders does seems to work, great
v
Yeah, if then each task has unique outputs and no task pollutes the outputs of another task, it should be fine in that regard.
r
I'm not sure how I would remove the dependsOns for verify and download, verify I don't think has an output
v
Maybe unzip should not depend on verify. Maybe verify should finalize download and unzip should use the download output as input. If unzip should really depend on verify I guess that's the famous exception from the rule as there it is then indeed more a semantical dependency than a file-dependency. (as long as the download output is still used as input for the file)
r
finalaizedBy may work, I'll give it a try, thanks I only want unzip to proceed if the download has been verified
v
Then you would still need a depends on the verify probably
r
Alright, thanks.
p
Any reason why you don’t want to use the builtin toolchain feature to download different jdks?
v
How would you do that to get jdks for different os and architectures? ;-)
p
Oh, missed that part :(
🙂 1