https://gradle.com/ logo
Join Slack
Powered by
hello, I'm trying to use the `Sync` task within a ...
# community-support
s
hello, I'm trying to use the 
Sync
task within a custom plugin to copy files into an intermediate folder for assembling a package's content. I want to be able to configure this directory, as in another task/subproject I need to first generate the content of this directory and in other subprojects I can just use a default. The problem I'm facing is that Gradle detects a non-declared dependency between the output of the preparation task and the input of the 
Sync
task. I don't want to directly depend on the task as it's an internal task to the plugin. Is there a way to specify the source of a 
Sync
task as an input property? Some example code to better explain:
Copy code
// subprojectA (build.gradle.kts)
val pkgSources = layout.buildDirectory.dir("generated/pkgSource")

myPlugin {
  sourceDir = pkgSources
}

tasks.register("foo", Sync::class.java) {
  from("someFolder")
  into(pkgSources)
}

// custom plugin
// has extension with sourceDir DirectoryProperty

tasks.register("prepare", Copy::class.java) {
  from(ext.sourceDir) // points to pkgSources
  into("somewhere/else")
}

tasks.register("buildPackage") {
  dependsOn("prepare")
}
If I call 
buildPackage
I get the non-explicit dependency error, and I assume it's because the call to 
from()
doesn't set an actual input for the 
Copy
task? Or am I missing something?
a
So, do you want subprojects to be able to provide files to 
myPlugin
, and then your plugin will do some processing on them? It makes sense that you want to hide the internal prepare task.
s
@Adam yes that in short is what I want to do. In some (most) cases suprojects will rely on the convention provided by the plugin, but in some cases I need more control over where the plugin reads from
a
what about adding a ConfigurableFileCollection to the 
myPlugin
extension? Then buildscripts can pass in the files, and then internally the 
prepare
sync task can do it's thing.
s
how is that different from providing the extension with a 
DirectoryProperty
?
v
The problem is not "it's because the call to 
from()
doesn't set an actual input for the 
Copy
task?". If that were the case it would bad and you wouldn't get the error. The problem is, that your 
pkgSources
Provider
does not carry an implicit task dependency to the 
foo
task.
a
Copy code
// my-plugin

abstract class MyPluginExtension {
  abstract val inputFiles: ConfigurableFileCollection
}

// build.gradle.kts

val fooTask by tasks.registering(Sync::class) {
  from("someFolder")
  into(temporaryDir)
}

myPlugin {
  inputFiles.from(fooTask)
}
v
If you set 
sourceDir
to the 
destinationDirectory
of 
foo
it probably works as intended
s
I think the main problem, which makes totally sense, is that I'm "skipping" a dependency in the depedency chain, i.e. I'm telling the build script that the final build task depends on the subproject preparation tasks, but in between there is another task that needs to run, also after the subproject preparation tasks
@Vampire I was thinking about that in abstract but didn't know which field to use, will try that out
a
> how is that different from providing the extension with a DirectoryProperty With a single DirectoryProperty, the tasks will all be competing to publish the files into a single directory. And so that's what Gradle is warning about. But if you used a FileCollection, then the tasks can run independently, and then internally your plugin can aggregate the tasks into a single directory.
s
@Adam the truth though is that somehow I think I need to resolve the fact that the plugin tasks all need to run after the suproject is done preparing the sources, I'm not sure if a 
FileCollection
provides different semantics in that regard
a
maybe I haven't understood the problem fully yet :)
something different about a FileCollection and a DirectoryProperty is that you can pass a task (that produces files) into a ConfigurableFileCollection, and then Gradle is able to automatically determine the task dependencies.
s
ah yeah ok I probably need that then also to do what @Vampire is suggesting, because actually I have 3 tasks running and putting files into the destination directory, which need to run before the plugin's own build task
let me try a couple of things out and if I don't manage I try to explain the problem in a better way
a
Let's say you have 3 producer tasks. They all produce files into separate directories Then a ConfigurableFileCollection is populated with the results of each task. And then a 
process
task (like your plugin's task) uses the ConfigurableFileCollection. Even though 
process
doesn't have explicit dependencies on the producer tasks, Gradle will be able to figure it out correctly.
Copy code
// build.gradle.kts

val producerTask1 by tasks.registering(Sync::class) {
  from("sources1")
  into("result/producer1")
}
val producerTask2 by tasks.registering(Sync::class) {
  from("sources2")
  into("result/producer2")
}
val producerTask3 by tasks.registering(Sync::class) {
  from("sources3")
  into("result/producer3")
}

val allProducedFiles = objects.fileCollection()
  .from(producerTask1)
  .from(producerTask2)
  .from(producerTask3)


val process by tasks.registering(Sync::class) {
  from(allProducedFiles)
  into("result/aggregated")
}
But that might not be what you're asking for :)
v
actually I have 3 tasks running and putting files into the destination directory
This also sounds suspicious. Hopefully all three tasks have no overlapping outputs. That means they have the concrete files as outputs, not the output directory. If all three tasks have the same directory as output directory, this is highly problematic and discouraged. The solution for that would exactly be what Adam just showed.
s
yeah truth is that I have exactly that. The reason for that is that the grpc plugin generates sources in different folders, and I need to put all those different files into a single python package. On top of that, I need to generate a project specification file that also goes in the same output directory. Only once that's done I can run a build command to generate an actual package. Adam's solution looks very close to what I need, though the 3rd task is part of the plugin, so I'd have to pass a file collection to the plugin, and then "enhance" that file collection with another file. I assume that's possible?
3 Views
Open in Slack
PreviousNext
Powered by