I want to download dependencies at execution time ...
# community-support
j
I want to download dependencies at execution time AND use the configuration cache... The performance page on the user guide says I shouldn’t resolve dependencies in the configuration phase. But it also says that dependency resolution results are stored in the configuration cache, and that happens in the configuration phase. I have a task that consumes hundreds of megabytes of dependencies to produce a small output file. This task is cacheable. Ideally most of my teammates’ builds will download its output file from the build cache without also downloading the dependencies. Is there a good way to do this? I have something that works but it’s a bit weird...
Here’s my task:
Copy code
@CacheableTask
abstract class MyTask @Inject constructor() : DefaultTask() {
  @get:Input
  abstract val inputJars: ListProperty<Provider<MinimalExternalModuleDependency>>

  @get:OutputFile
  abstract val output: RegularFileProperty

  @TaskAction
  fun execute() {
    val configurationContainer = services.get<ConfigurationContainer>()
    val configuration = configurationContainer.detachedConfiguration()
      .apply {
        isCanBeResolved = true
        isTransitive = false
        isCanBeConsumed = false
        for (provider in inputJars.get()) {
          dependencies.addLater(provider)
        }
      }
    val files = configuration.files

    // Use files to produce output
  }
}
And here’s my
build.gradle.kts
file:
Copy code
val myTask by tasks.registering(MyTask::class) {
  inputJars.addAll(
    libs.myVeryLargeDependency,
    libs.myOtherVeryLargeDependency,
  )
  output.set(layout.buildDirectory.file("myTask/output.txt"))
}
I am very happy to be able to get a
ConfigurationContainer
at execution time, with which I can create a new
Configuration
It’s also quite rad that
Provider<MinimalExternalModuleDependency>
doesn’t seem to interfere with the configuration cache
I don’t like that I’m using
ListProperty<Provider<MinimalExternalModuleDependency>>
instead of something normal like
Property<FileCollection>
And I don’t like that I can’t use the
dependencies {}
block here
And I also don’t like that there’s no obvious way to mix and match other kinds of dependencies, like project dependencies
All that said, this seems to work!
t
v
You might just want to create a detached configuration in the implementation of your task, that is then just done at execution time as it is an implementation detail. You can also maybe trick the configuration cache, because only the resolutions that do not depend on task execution results are serialized to configuration cache. So if you make sure a property has a task dependency, then its value will not be serialized to the configuration cache entry but calculated at execution time. If it is input files it will of course still be done before the task's up-to-date check as all input files are relevant for that. Btw.
Property<FileCollection>
usually is not normal. 🙂 Normal is
ConfigurableFileCollection
which is already a lazy construct.
j
Love it
I’m gonna try to figure out why I get this warning...
Copy code
Resolution of the configuration :treehouse-build:upstream-protos:detachedConfiguration1 was attempted from a context different than the project context. Have a look at the documentation to understand why this is a problem and how it can be resolved. This behavior has been deprecated. This will fail with an error in Gradle 9.0. For more information, please refer to <https://docs.gradle.org/8.12.1/userguide/viewing_debugging_dependencies.html#sub:resolving-unsafe-configuration-resolution-errors> in the Gradle documentation.
v
Oh, somehow I missed, that you actually do use a detached configuration already. Btw. using
services
is not a good idea, it is in an internal package as well as the
get
you use, and it (the
getServices()
) returns a class in an internal package. Just get it injected using
Copy code
@get:Inject
abstract val configurations: ConfigurationContainer
Why you get that resolution error I have no idea from what you showed 😕
Usually you get that when you resolve configurations from other projects and similar
Maybe it does not work to use detached configurations at execution time with configuration cache. 😞
j
Oooh the
@get:Inject
is just what I needed. Thank you!
👌 1
I filed a feature request to dismiss the warning, or offer a new API to resolve a configuration during the execution phase. https://github.com/gradle/gradle/issues/32269
👌 1
v
You should replace the Slack link with the according Linen link, as those persist while the Slack link will become invalid in 90 days.
1