https://gradle.com/ logo
Join Slack
Powered by
I have a task with an input that is a `RegularFile...
# community-support
i
I have a task with an input that is a 
RegularFileProperty
which is generated by another task (in a plugin I don't control) but isn't an output. Without configuration cache, everything works well. With configuration cache, Gradle tries to configure the input before running any tasks, and that breaks because it's not possible to know what the file will be. Is there a way to set a 
Property
with a flag that it must be lazily-generated?
Essentially, I'd like to mark a specific input as "don't try to compute this value until all dependencies have run" and "do compute it each time, even if we loaded the configuration cache"
m
The path of the output property should be set at configuration time. Its content at execution time.
So probably an issue with the other plugin?
i
I don't know what the path is going to be until the other task runs
So probably an issue with the other plugin?
Yeah but it's the KGP so…
m
don't know what the path is going to be until the other task runs
That cannot work I think? Gradle needs to know the path to put in CC for an example.
KGP
interesting. What property is this?
i
File(compilation.npmProject.require("vite"))
allows to task KGP to scan the node_modules files to find the binary. That only works after the 
:kotlinNpmInstall
task has run
(and I need the Vite path to be an input for another task)
m
Why is it always 
kotlinNpmInstall
😅 . That part of KGP feels a bit alien.
You could write a file to a know location that contains the path to the computed location
😅 1
i
…because they decided to wrap NPM instead of mapping its behavior to Gradle concepts of dependencies. I guess it was much easier to do, but it's very hard to write plugins that play well with it
☝️ 1
m
Just write the relative path so as not to break caching...
"Just"
i
For now, I hardcoded the path, but that will probably break in the future 😕
😞 1
m
Actually, thinking about this more, could you do something with ValueSource?
Not sure this can execute tasks though...
i
I've never used it, so I don't really know what it's capable or not. The overall idea seems similar though
m
I've used it very rarely but my understanding is that it allows to model changing "CC-inputs"
Like if an env variable changes, it won't invalidate your CC
But getting a task to compute a ValueSource sounds very unlikely (because chicken-egg problem)
i
Can the value source change within a single build?
m
Mmm probably not
It's captured once and put in the CC ideally
v
I've used it very rarely but my understanding is that it allows to model changing "CC-inputs"
[...]
Like if an env variable changes, it won't invalidate your CC
Exactly the other way around. If the value source result changes, the CC entry is discarded and configuration is done freshly. They are always evaluated once to determine whether the value changed. And if the CC entry is discarded and recalculated it is computed a second time. The value of the value source cannot change within one build and I don't think it can execute a task unless you start a separate build in a new process which you most probably should not do. With CC, values of `Provider`s (and thus also `Property`s) are evaluated after configuration was done and before CC entry is serialized, serializing the value of the 
Provider
and reusing that value if CC is reused. The only exception is, if the 
Provider
has a task dependency in which case the value is of course only evaluated on the first actual 
get()
so that the task was already executed. Without the task dependency it most probably also just appears to work properly, because you miss the task dependency, unless you added an explicit task dependency which is bad practice. What you want is most probably something like 
myRegularFileProperty = tasks.theTaskProducingTheFile.map { calculateTheFileAndReturnItHere }
so that the property has the necessary task dependency and thus also is not serialized into CC entry.
i
@Vampire Oh, that looks promising. I tried this:
Copy code
vitePath.set {
			project.rootProject.tasks.named("kotlinNpmInstall")
				.map { File(compilation.npmProject.require("vite")) }
				.get()
		}
but it doesn't fix the problem, the value is still evaluated before the 
:kotlinNpmInstall
task runs.
v
Of course
You 
get()
it, so it is evaluated immediately. You should never 
get()
a provider at configuration time optimally but always only at execution time.
Also 
set { ... }
is probably a bad idea
And reaching into the root project and getting a task from there is a super bad idea
Querying information from other project's models is almost as bad as doing cross-project configuration
You need to create an outgoing variant in the root project that you then request from the subproject to do it properly.
i
And reaching into the root project and getting a task from there is a super bad idea
That I do know, but I don't have a choice. That task isn't mine, and it only exists in the root project 😕
> Also 
set { ... }
is probably a bad idea What should it be? I'm in a plugin, I don't have access to the 
=
syntax. 
vitePath
is a 
RegularFileProperty
.
Doesn't work either:
Copy code
vitePath.set(
			project.layout.file(
				project.rootProject.tasks.named("kotlinNpmInstall")
					.map { File(compilation.npmProject.require("vite")) }
			)
		)
m
If the value source result changes, the CC entry is discarded and configuration is done freshly.
Sorry feels like I should know that but what's the benefit of using a 
ValueSource
compared to "just" letting the bytecode instrumentation intercept 
System.getenv()
then?
v
That I do know, but I don't have a choice. That task isn't mine, and it only exists in the root project
Sure you have. That it is not your task does not mean that you cannot register an outgoing artifact using it.
I'm in a plugin, I don't have access to the 
=
syntax.
You could have, you just have to apply the according compiler plugin, either manually, or by applying the 
koltin-dsl
plugin. (
kotlin-dsl-base
would also be enough if you do not want the other effects of 
kotlin-dsl
, even 
org.gradle.kotlin.kotlin-dsl.compiler-settings
would be enough if 
kotlin-dsl-base
still does more than you want, the latter is the one applying the sam-with-receiver and assignment plugins). But even without the 
=
, it is just an alias for calling 
.set(...)
, (not 
.set { ... }
). You call 
.set(...)
with a 
Provider<RegularFile>
, so yes, the 
vitePath.set(layout.file(taskProvider.map { File(...) })
should work API-wise. What error do you get?
Sorry feels like I should know that but what's the benefit of using a 
ValueSource
compared to "just" letting the bytecode instrumentation intercept 
System.getenv()
then?
None in that case, but in a value source you can do lots of things, including calling external processes or doing computation to determine the value of the value source.
❤️ 1
👍 1
i
the 
vitePath.set(layout.file(taskProvider.map { File(...) })
should work API-wise. What error do you get?
The value is evaluated before the 
:kotlinNpmInstall
task, so it crashes:
Copy code
* What went wrong:
Configuration cache state could not be cached: field `__vitePath__` of task `:viteBuild` of type `opensavvy.gradle.vite.kotlin.tasks.KotlinViteExec`: error writing value of type 'org.gradle.api.internal.file.DefaultFilePropertyFactory$DefaultRegularFileVar'
> Cannot find node module "vite" in "/…/kotlin-vite/examples/simple/build/js/packages/example-simple"
That it is not your task does not mean that you cannot register an outgoing artifact using it.
That would require asking all of my users to apply a new plugin to their root project, which I want to avoid.
v
which I want to avoid
Well, do whatever you like, but I as user of that plugin would report a bug. Latest with project isolation this will be a hard error as you must not access mutable state of other projects. So latest then you have to do it that way.
The value is evaluated before the 
:kotlinNpmInstall
task, so it crashes:
The code should work and have the necessary task dependency implicitly. Of course 
vitePath
must be declared as input property, if you do not have that yet.
It it's not that, please show an MCVE.
i
Latest with project isolation this will be a hard error as you must not access mutable state of other projects.
So latest then you have to do it that way.
Can you clarify, depending on a task counts as depending on mutable state? It's not like I'm reading any of its configuration, it's purely about its existence. Also, if that's the case, KGP will be broken anyway so I'll have to wait until they redesign it 😕
Of course 
vitePath
must be declared as input property, if you do not have that yet.
Copy code
@get:InputFile
@get:PathSensitive(PathSensitivity.RELATIVE)
abstract val vitePath: RegularFileProperty
It's not really an MCVE, but the plugin is quite small; • 
git clone <https://gitlab.com/opensavvy/automation/kotlin-vite.git>
• Branch 
upgrades-2.2
• File 
vite-kotlin/src/main/kotlin/tasks/ViteExecTask.kt:50
• Reproduction command: 
./gradlew -p examples/simple --include-build ../.. clean; ./gradlew -p examples/simple --include-build ../.. build --rerun-tasks --configuration-cache
v
Can you clarify, depending on a task counts as depending on mutable state?
Of course, whether the task exists or not is mutable state of the project model. It only exists if it was registered already or a plugin registering it was applied already. Immutable state would be the project path or project directory, i.e. thing that you configure in the settings script.
Also, if that's the case, KGP will be broken anyway so I'll have to wait until they redesign it
Why?
Reproduction command
compilation.npmProject
is not found
i
compilation.npmProject
is not found
Are you in the correct branch? It's in the same file, line 34
v
compilation is there, npmProject not
image.png
i
> Also, if that's the case, KGP will be broken anyway so I'll have to wait until they redesign it
Why?
When KGP is applied to a project with the JS platform, it adds a bunch of tasks in the root project to install stuff, even though KGP isn't applied to the root project. If Gradle ever becomes stricter about cross-project configuration, they will have to redesign all of that anyway, and I'll have to rewrite my plugin to work with whatever they replace that with
Can you force IntelliJ to reindex to ensure it's not drunk? It works on my side and in CI, so I don't think the branch is incorrect
v
It is the Gradle call that also complains, not just the IDE
i
🤔
v
Copy code
> Task :kotlin-vite:vite-kotlin:compileKotlin FAILED
7 actionable tasks: 1 executed, 6 up-to-date
Configuration cache entry stored.
e: file:///.../kotlin-vite/vite-kotlin/src/main/kotlin/tasks/ViteExecTask.kt:58:5 Type mismatch: inferred type is Unit! but File! was expected
e: file:///.../kotlin-vite/vite-kotlin/src/main/kotlin/tasks/ViteExecTask.kt:59:11 Type mismatch: inferred type is (Task) -> Unit but (Task) -> File! was expected
e: file:///.../kotlin-vite/vite-kotlin/src/main/kotlin/tasks/ViteExecTask.kt:59:30 Unresolved reference: npmProject

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':kotlin-vite:vite-kotlin:compileKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Compilation error. See log for more details
Ah, needs an import
i
Just pushed a new branch with my exact changes: 
remove-workaround-vite-path
with the 
named().map
v
Yeah, found the import already, see last comment 🙂
👍 1
i
I searched a bit, can't find an equivalent of 
.buildBy
on 
Provider
, do you know if that exists with another name?
v
You shouldn't need builtBy, as the provider itself should carry the information if used properly
And ripping the task apart so that there is no dependency on 
kotlinNpmInstall
with the 
upgrades-2.2
branch and then using that snippet with 
project.layout
adds the dependency on 
kotlinNpmInstall
, so that seems to be as expected.
i
Sorry I don't understand. Can you copy/paste your section where 
vitePath
is set?
v
Just copy from your own comment above.
i
This one?
v
Yes. Well, I actually used 
File("""compilation.npmProject.require("vite")""")
to just verify the task dependency is there.
Without the quotes it indeed fails with "cannot find node module". 😕
i
Or maybe Gradle does understand that the dependency exists, but still evaluates the value right away?
v
If Gradle sees there is a task dependency, it should not evaluate the property for serialization into configuration cache, as it expects that it needs the task output to calculate the provider result so should delay it to later.
Only providers without task dependencies should be calculated right away and their result serialized to the CC entry
i
That makes sense to me, but it doesn't seem to be the behavior here
v
Yeah, the question is why and whether it is a usage error or Gradle bug
Hm, sorry, my brain is probably still on vacation, also this is not doing what I expect:
Copy code
abstract class MyTask : DefaultTask() {
    @get:Input
    abstract val foo: Property<String>
}
val bar by tasks.registering
val baz by tasks.registering(MyTask::class) {
    foo = bar.map {
        println("Calculating foo")
        ""
    }
}
Something I forgot on Madeira. 😕
i
Yet, the 
Provider.map
documentation confirms your explanation
It doesn't mention configuration-cache initialization order
v
i
thanks, following
Coming back here;
Copy code
vitePath.set(
			project.layout.file(
				project.rootProject.tasks.named("kotlinNpmInstall")
					.flatMap { it.outputs.files.elements }
					.map { File(compilation.npmProject.require("vite")) }
			)
		)
The initialization order is now correct! However now it complains that I'm using 
project
at execution-time 😅 But that's something I can fix for myself
Thanks a lot @Vampire!
v
Yeah, probably something like
Copy code
val npmProject = compilation.npmProject
vitePath.set(
	project.layout.file(
		project.rootProject.tasks.named("kotlinNpmInstall")
			.flatMap { it.outputs.files.elements }
			.map { File(npmProject.require("vite")) }
	)
)
or something like that
But you should really consider not querying the tasks of the root project
i
yep, that's what I did, it seems to work
👌 1
> But you should really consider not querying the tasks of the root project I wish I could, but again, it's KGP that creates this task, and it doesn't create an equivalent task in the current project 😕
v
You could, you just don't want to as your users would need to apply an additional plugin to root project or settings 🙂
i
Fair enough. I will create a bug report in KGP though
👌 1
a
Also, if that's the case, KGP will be broken anyway so I'll have to wait until they redesign it 😕
I'll be working on KGP JS soon, to make it compatible with project isolation https://youtrack.jetbrains.com/issue/KT-75899/Support-Gradle-Project-Isolation-in-KGP-JS-Wasm
👀 1
❤️ 1
i
Nice, I'll follow that issue. I don't mind helping beta-test it if that helps
a
because they decided to wrap NPM instead of mapping its behavior to Gradle concepts of dependencies. I guess it was much easier to do, but it's very hard to write plugins that play well with it
For some context: With KGP JS the reasoning behind the use of the root project is the node_modules dir. The JS target has a lot of default JS dependencies. Long story short: When there are lots of subprojects a node_modules dir per-subproject would use a lot of disk space, even if the subprojects has no explicit dependencies. When KGP JS was first implemented there weren't as many options as there are now to help out. I think this is really interesting topic in KMP. When integrating with another language, should KMP wrap the native tools, or should the tools be re-implemented to make them work more like Kotlin already does (i.e: make it work like JVM, since that's Kotlin's origin)? Either option has trade-offs. Making Gradle-first wrappers for the JS tools would be more maintenance and documentation. Trying to delegate to the existing tooling is quite hard when Gradle doesn't have an appropriate mechanism to support it.
i
100% agree, it's a difficult question
Personally I would prefer if everything was integrated into a single way of doing things, but I know that it's very important for developers from other ecosystems that they can find the hidden actual NPM folder and try stuff out inside there before trying to write it for Gradle
10 Views
Open in Slack
PreviousNext
Powered by