https://gradle.com/ logo
Join Slack
Powered by
Can anyone cite an example of a deferred model pro...
# community-support
d
Can anyone cite an example of a deferred model provider ? What I mean by this is I have the following sequence: • Run a large amount of stuff (lets call it 
toplevel:assemble
) • Now run some Gradle code that will evaluate what exists at this deferred point in time, there maybe multiple data items of different types, such as String, ListString, FileCollection, etc.. each is given a label like a class property is a label. Maybe I call this a class Model that acts as Provider for multiple things. • Now run a standard Gradle task, that 
dependsOn('toplevel:assemble')
and consumes the model provider.
Copy code
// pseudo code that is not real but that I think I need
abstract class MyModelProvider : ProviderBaseSomething() {
   @get:Provider
   abstract val listOfStrings: ListProvider<String>

   @get:Provider
   abstract val fileCollection: FileCollection

   @get:Provider
   abstract val oneString: Provider<String>

   @ProviderRunner
   fun generate() {
      listOfStrings = listOf("what I found")
      fileCollection = FileCollection(File("first.dat"), File("second.dat"))
      oneString = "My String"
   }
}

val myTask = tasks.register("myTask", Type::class) {
   dependsOn(":assemble")

   // Happy to checkpoint state here, meaning re-evaluation can be allowed from the position of this checkpoint state
   // So the provider is free to re-evaluate the situation and produce a new set of data,
   //  that is allowed to differ from a previous evaluation, allowing future tasks to act upon its findings

   val myModelProvider = someApiHereToCauseDeferredExecutionOf(MyModelProvider::class)
   // I can not put the code here as it execute too early
   // doLast { /* This also did not get the deferred view of the project */ }
   info("foobar") {
      from(myModelProvider.get().fileCollection)
   }
   something.set(myModelProvider.get().oneString)
   options.addAll(myModelProvider.get().listOfStrings)
}
v
The `Property`s and `Provider`s and 
FileCollection
themselves are the lazy types. If you set their value from a 
Provider
or use 
map
, then those things are evaluated on getting the 
Provider
. You should not need such a deferred model provider. But you could probably just wrap your model inside a `Property`/`Provider` yourself and then 
flatMap
to the contained providers for example if you really think you need something like that.
d
Thanks for the reply, trying to decode what it being said there. Currently I am trying with:
Copy code
data class OutputData(val oneString: String, val fileCollection: FileCollection, val listOfStrings: List<String>)

// Then trying to find a way to get use of
Provider<OutputData>
// as one conceptually important thing, the whole set of properties is consistent with itself, a `dataset` of multiple pieces of data that should always be together and only make sense together
Then I can look at how to force evaluation to produce a new set of data at the point in time that should occur. I guess this might be called a composed (composite) Provider
s
I believe providers are (supposed to be) evaluated at most once. Perhaps you want a different abstraction? What's your specific use case?
v
Providers are not supposed to be evaluated only once. You can evaluate them as often as you want.
d
I am finding it will evaluate the provider callable everytime 
get()
is called. This is ok for me, as I can control that, and do
Copy code
val m = myModel.get()
val thing = m.propertyOne
val another = m.propertyTwo
v
And unless you do something like 
finalizeValue
or 
finalizeValueOnRead
, the configuration chain of the provider is re-evaluated each time it is requested
👍 1
d
Am using
Copy code
data class OutputData(val oneString: String, val fileCollection: FileCollection, val listOfStrings: List<String>)

val myModel = project.provider {
  OutputData("value", fileCollection, listOf("one", "two"))
}

// However I am not seeing it get executed in the time frame I expect, it evaluates too early
// it needs to evaluate after it has reached checkpoint state `toplevel:assemble`

// I am also struggling with using standard AbstractCopyTask due to the NO-SOURCES situation,
//  because it needs to always force run this task, if it is scheduled to run, it should not
//  try to skip it.
// It also need to only evaluate what the sources are after checkpoint 'toplevel:assemble'
//  has been reached.
The use case use is standard release-engneering situation that it trying to incorporate the releng into the same project hierarchy as the target being released.
of course I can use another external project and have dependency headache having to name all versions and all output artifacts and input artifacts. But it is easier to switch to a BASH shell script and some other tool at that point just to package up in a specific way.
v
If your provider is evaluated too early, then probably because you do so somewhere. If you for example 
get()
it for configuring the copy task. That is then done at configuration time, so before 
assemble
could execute.
.map
or 
.flatMap
your provider instead
You should always avoid `get`ting any provider at configuration time
d
How do I defer a JarTask/ZipTask/AbstractCopyTask to defer ALL configuraition until after checkpoint 
toplevel:assemble
v
The sense of the whole 
Provider
/ 
Property
API is, that you 
get
it as late as possible, optimally only at execution time, so that further configuration changes can be done without the chance to evaluating too early and thus again introducing ordering problems
How do I defer a JarTask/ZipTask/AbstractCopyTask to defer ALL configuraition until after checkpoint 
toplevel:assemble
I already said, you cannot.
All configuration has to happen in configuration phase, execution happens in execution phase.
d
"I can not" ... without using another external gradle project
v
So 
.map
or 
.flatMap
your provider instead of `.get`ting it
As I already said
> "I can not" ... without using another external gradle project Or a composite build, yes We are only talking about withing one build
d
obviously you can, I am claiming using another external gradle project, show only purpose is to releng
v
If another build or a composite build solves your problem, just do it.
d
obviously you can, I am claiming, using a BASH script to manage the execution of gradle target, then to access the situation and manage the releng (but this enters non-gradle solution terrority)
v
If you want to do it in the same build, I already told you twice how to do it
d
Copy code
val m = myModel.map { it } // does not work
s
Instead of 
myModel.get().foo
you use 
myModel.map { it.foo }
d
I want 
it
is of type 
data class OutputData
it must not use 3 instances of 
it
to provide each property at each of the 3 call sites
it MUST use the same instance of 
it
(it is a "dataset" of multiple things, they are all only consist with itself in that moment)
v
Make 
myModel
finalizeValueOnRead
could maybe help?
Might be necessery to make it a 
Property
for that
d
Thanks trying something like this...
Copy code
data class OutputData(
 val oneString: String,
 val fileCollection: FileCollection,
 val listOfStrings: List<String>
) {
  fun finalizeOnRead() = true
}
v
I don't think that will bring you anywhere 🙂
Have a look at the cached providers in the issue I linked. They do exactly what you need.
d
Indeed the presence of the finalizeOnRead() in the form above makes no difference to anything. The use of
Copy code
val m = myModel.map { it }.get()
val m = myModel.get()  // also makes no difference to use map, might as well keep it simpler
The issue I have is one of deferring the created/configuration of a task until after a specific checkpoint state 
toplevel:assemble
I don't think the problem is one of a caching of the data, because I can get a single reference to the 
data class
object at the call site and use that same reference for accessing each property. The issue I face is deferring when it invokes the producer callable. So caching the Provider data is no the problem I have. Deffering the first/only call to the provider is the issue. Gradle won't let you programmatically create a task on the fly and execute it immediately. It wants to have all the tasks worked out at the start. So looking at how to invoke Gradle from inside Gradle, maybe that ends up really being, some kind of serialized state (lets call it JSON) being passed to a process, that knows how to run an isolated gradle task (it builds a skeleton gradle project, in a temporary directory, copies all the data into it it needs, provides configuration, and then runs it, extracts output data, then deletes the temporary gradle project). The key thing is Gradle during the main project build, has all the information to hand that such a process needs to perform the task, so its just case of conveying it to the process and having it execute upon the input data. But that data can only be known at the time after checkpoint 
toplevel:assemble
s
What you want is 'configure toplevel:assemble', 'execute toplevel:assemble', 'configure the rest', 'execute the rest', but Gradle doesn't work like that. Gradle does 'configure everything', 'execute everything'.
d
Yeah I want to define what I am calling a 
checkpoint state
that would allow another round of 
configure
then 
execute
etc.. to happen
v
Indeed the presence of the finalizeOnRead() in the form above makes no difference to anything.
Yeah, why should it? I said you need to call it, not provide a random method in a random class. I even gave you concrete examples on how to make a cached provider so that the 
it
is the same in all three 
.map
calls.
also makes no difference to use map, might as well keep it simpler
Of course it makes not difference, you still 
.get
the provider which I said multiple times you must not do. You should instead only use 
map
.
The issue I have is one of deferring the created/configuration of a task until after a specific checkpoint state 
toplevel:assemble
And I already said multiple times that this is not possible within one build. First all configuration is done, then all execution is done, that's by architecture and you cannot change that without building a custom Gradle distribution that you majorly refactor to fit your needs. There are surely other ways, like your bash script, a separate build, a composite build, just like you said yourself. I just don't understand why you seek experts help here for getting it done in a better way but then ignore the recommendations you get and say "no I don't want to do that" when it clearly is what you should want to do. But ok, I'm out of the conversation until you indicate that you are interested in getting it solved.
d
Thanks for your help, as I keep saying!
v
I don't doubt that you are helpful for me answering, but you more or less ignore what I say, and I'm not really up to that right now. 🙂
d
I have tried the things you stated, you seem to think I didn't, you are mistaken about something, but it matters not.
v
You did not try what I say. You majorly changed what I say and combined it with the stuff I said you should not use, resulting in what I said not working.
You could also consider providing a proper MCVE, then it would also be easier to talk about stuff
d
So the solution for me appears to be:
Copy code
data class OutputData(val first: String, val second: String)
val myModel = project.provider {
  OutputData("111", "222")
}
// Inside a DefaultTask
val myTask = tasks.register("myTask", DefaultTask::class) {
dependsOn(":assemble")
doLast {
  // Can call project.provider {} Callable from here, and get deferred evaluation as needed
  val m = myModel.get()
  / I can access m.first and m.second and they will related to each other and provide consistent model
  // Generate a file containing serialized data to perform task
  // I went for: build\data\releng.properties
  exec {
    val pathToJava = "C:\somewhere\bin\java.exe"
    standardOutput = System.out
    errorOutput = System.err
    commandLine(pathToJava, "-jar", pathToUberJarOutputArtifact, "--do-releng-thing", "--output-file", absolutePathToBuildDirFilenameOutput, "--data-file", absolutePathToRelengDotProperties)
  }
}
This appears to satisfy the goals: * of having access to the entire state of gradle after the checkpoint situation. • Allows execution of a task, written for JVM to occur using the data known. • Allows execution of a single Gradle run on larger project to get out expected custom releng packaging • My project already had an uberjar (shade/fatjar) which is designed to facilitate any command line tooling requirements such as this. So allows using Java/Kotlin code to create tasks, even if they are not Gradle tasks. Thanks for the suggestions everybody in getting there.
🤷‍♂️ 1
7 Views
Open in Slack
PreviousNext
Powered by