From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

You could just use `tasks.configureEach { ... }` and action will be executed when task is realized. If you collect realized task names from the action and compare them to `tasks.names` you should have actual list at the action execution time I think

Differently asked, what do you actually want to achieve, what is your use-case?
It sounds like you chose a solution for a use-case and wonder how to fix it, but there might be a better solution if you take a step back.

My use case is working fine, I'm basically wondering why there is no lazy equivalent of `#maybeCreate`

I guess because that method is supporting odering constraints which is bad.
If plugin A is applied first, task x is variant X, if plugin B is applied first, task x is variant Y, for example.

I think you are facing an order problem. Consider this
```// plugin.kt

    class MyPlugin {
(2)   fun apply() { ... }
    }```
and then…
```// build.gradle.kts

    plugin {
(1)  id("my-plugin")
    }

    myPlugin {
(3)  myProperty.set(...)
    }

(4) ... // other code

    tasks.register("aTask") { 
(5)   ... // lazy task configuration
    }```
The order of execution is 1,2,3,4 &amp; 5. All run in the configuration phase except 5 which runs in execution phase (if set up properly).

However, _“configuration phase”_ *to your plugin* is execution number (2). Theres no way code that runs in 2 will “know about” what happens in 3 &amp; 4 later.

Now you can set up your value source using lazy properties so that in (2) you have
```val myPluginExt = ...
val myValueSource = providers.value(MyValueSource::class) {
  myValueProperty.set(muPluginExt.myProperty)
}```
but if you try to resolve this value source in this same step 2, then it will be the default/empty since 3 &amp; 4 have not run yet.

So your options are to delay the resolution of the value source to the execution phase (within some task) or use afterEvaluate.

exactly, that is the problem

```plugins {
    id("some.plugin")
}

somePlugin {
    someProp.set("v")
}

println(project.version) // this is happening before reading the the someProp```

I did a workaround by using a Gradle project property 

probably <https://github.com/gradle/gradle/blob/master/subprojects/plugin-development/src/main/java/org/gradle/plugin/devel/plugins/MavenPluginPublishPlugin.java|https://github.com/gradle/gradle/blob/master/subprojects/plugin-development/src/main/java/org/gradle/plugin/devel/plugins/MavenPluginPublishPlugin.java>

No, I’m talking about the com.gradle.plugin-publish plugin that communicates with the plugin portal. I’m starting to think it’s closed source, given the com in its id…

Thanks Benedikt! I wanted to look at it mostly to understand why it exists :smile: AFAIU it hides away all the complexity of communicating with the plugin portal. But I’m curious about why it is complex in the first place, what makes it need more than a `credentials` block.

It’s indeed closed source, but mostly due to legacy/accidental reasons. We do want to open-source it, but unfortunately it’s pretty low on our priority list… it does hide all the complexity of communicating with the portal, but it also does a lot of wiring together (forcing source and javadoc artifacts, signing, …) and checking (versions, vulnerabilities, …); even the communication part is not as simple as it seems, because it’s the plugin that uploads artefacts directly to S3, not the portal; we have simplified it a lot lately, but it’s still an ugly beast… I hope we can make OS soon, it would definitely improve it in the long term

Couldn't `ConfigurableFileCollection` solve the same? I'm not sure if it's resettable but it's `#from(Any)` method is god-send

&gt; I'm not sure if it's resettable
You can call `#clear`. The problem is that the user _has_ to call `files.clear()` in that case. The `Property#convention` solution is way more elegant in that respect

I would still use a `ConfigurableFileCollection` . Mark your configurable property `@Internal` and add another `@InputFiles` property that checks whether your configurable property is empty and then returns a default value otherwise.

```@Internal
public abstract ConfigurableFileCollection getMyProperty();

@InputFiles
/* does not need to be public */ FileCollection getActualFiles() {
    var myProperty = getMyProperty();
    if (myProperty.isEmpty()) {
      return getProject().files(…);
    }
    return myProperty;
}

// use getActualFiles() in the task action```


Adding a convention to `ConfigurableFileCollection` would be a good feature request.
In the meantime, +1 to Thomas solution

Nitpick: I'm not sure that that solution can distinguish between `myProperty` being empty and `myProperty` being unset (another thing properties/conventions are good at)

<https://github.com/gradle/gradle/issues/21752>

you’d typically want a provider, not a resolved property.  e.g. `myInstallTask.whateverProperty.map { … }`, which will be deferred until execution time, avoiding that error message.  Also - if you follow this route - the dependsOn is no longer required, as the provider propagates the task dependency,.

Well, moving this all inside a `project.afterEvaluate` really didn't have any effect, either. I think I'm just misusing/abusing the `DirectoryProperty` . Not sure how else to go about this. I don't want to `resolvedDir.get()` too early.

`project.afterEvaluate` is a code smell - causes far more problems than it solves.

Something like this:
```project.pluginManager.withPlugin(OTHER_PLUGIN) {
            val install = project.findProperty("myInstall") as InstallPluginExtension
            val myInstallTask = tasks.named("myInstall")
            cmakeDebug.configure {
                // not necessary
                // dependsOn(myInstallTask)
                includeDirs.from(myInstallTask.resolvedDir.map { it.dir("include") } )
                linkFiles.from(myInstall.resolvedDir.map { it.dir("debug/lib") })
            }```

Ahh. Yes, that seems much better. I was overthinking this trying to use the extension. Should just use the `OutputDirectory`. Much saner approach.

Definitely, someone could have configured the task directly instead of the extension so by taking the extension value you maybe take the wrong one.

Besides getting the proper implicit task dependency of course.

&gt; Does this all just need to be moved to a `project.afterEvaluate`?
Almost all usages of it are a smell and should be avoided.
&gt; And, really feels a little dirty to put a `#withPlugin`  closure there.
`pluginManager.withPlugin` is never dirty, you want to configure an extension a plugin adds or use a task a plugin registers, then react to the plugin being applied (or if you always need it, then explicitly apply it), no need to feel dirty.

Explicit `dependsOn` on the other hand are also almost always a code smell (unless there is a lifecycle task on the left-hand side of it)

It seems like I am still misusing `DirectoryProperty`. I have a much cleaner task wiring thanks to the suggestions, yesterday. But, my error has just propagated to the `task`.
```&gt; Cannot query the value of this provider because it has no value available.
  The value of this provider is derived from:
    - task ':myInstall' property 'dir'```
So, why is this trying to resolve the property so early, rather than mapping to it?
```         project.pluginManager.withPlugin(INSTALL_PLUGIN) {
            val myInstallTask = tasks.findByName("myInstall") as Install
            cmakeDebug.configure {
                includeDirs.from(myInstallTask.dir.map { it.dir("include") })
                linkFiles.from(myInstallTask.dir.map { it.dir("debug/lib") })
            }```
I must be struggling to understand the proper usage of `#from` and `#map` in this context.

&gt; public abstract ConfigurableFileCollection from(     Object... paths )
&gt; Adds a set of source paths to this collection. The given paths are evaluated as per org.gradle.api.Project.files(Object...).
&gt; ConfigurableFileCollection files(     Object... paths )
&gt; Returns a ConfigurableFileCollection containing the given files. You can pass any of the following types to this method:
&gt; ...
&gt; A Provider of any supported type. The provider's value is recursively converted to files. If the provider represents an output of a task, that task is executed if the file collection is used as an input to another task.
&gt; public abstract &lt;S&gt; Provider&lt;S&gt; map(     org.gradle.api.Transformer&lt;? extends S, ? super T&gt; transformer )
&gt; Returns a new Provider whose value is the value of this provider transformed using the given function.
&gt; The new provider will be live, so that each time it is queried, it queries this provider and applies the transformation to the result. Whenever this provider has no value, the new provider will also have no value and the transformation will not be called.
Hmm. It says that it is live, but doesn't say it will throw an error if it has no value. So, `#from` requires an immediate value at invocation?

Live means that it is not resolved once when you create it, but that it always shows the state that is valid at the time you query it.
From a quick look I don't see an error, but it is probably in code you did not show. Can you provide an MCVE?

BTW, you don't need the `map` here, `myInstallTask.dir.dir("include")` should do the same AFAICT

And better to use something like `val myInstallTaskProvider = tasks.named&lt;Install&gt;("myInstall")` with `myInstallTaskProvider.flatMap { it.dir.dir("include") }` to avoid realizing all tasks on `tasks.findByName`

why would you want to do that? Chances are very high that the sourceSet is used somewhere else, given that it was added before. So removing it again seems very dangerous.

So I used `sourceSets.all{ .remove(it) }` instead and that seems to work :shrug:.

That said, I understand the message above but these source sets are added in response to the agp variants and for the modules in question these agp variants are disabled and no tasks generated that use the variants or sourceSets (in theory :sweat_smile:).

Still, I take responsibility for whatever consequence removing the sourcesets bring though I imagine it to be none after looking through the kotlin plugin source :thinking_face:.

There’s `.whenObjectRemoved {  }` , so the clients have APIs to track removals and act accordingly.

Not trying to track removals. I need to remove because an object is added  erroneously

How do you configure your generation task?

I write sources in the output classe dir of the main / test source set, using a DirectoryProperty would fix the implicit dependency ?

I think it would be better to have these sources generated in their own output directory and just add it to the output of the sourceset instead (you can use `builtBy` to manage the task dependency if necessary).
e.g.
```sourceSets {
    main {
        output.dir(generatedResources, "builtBy" to "generationTask")
    }
}```
In general I believe it's not advisable to reuse the same output dir for multiple tasks.

Are you generating `java`  files or `class` files? For `class` files something like :point_up: should work

For `java`  files it should be something like this:
```sourceSets.main {
    java.srcDir(myGenerationTask)
}```

An example (not pretty, but still illustrates this ; note that the modification of classpaths is specific to GWT, which needs the source files as resources too): <https://github.com/gwtproject/gwt-places/blob/master/processor/gwtTest.gradle.kts>

Thanks for all those answer, I'm generating java file, I will try that 

changing `implementation` to `compileOnly` maybe fix your problem

but to be honest I never add the `jvm`  one as `implementation`, I add the generic one `org.jetbrains.kotlin:kotlin-gradle-plugin`

`compileOnly` only sort of works, ran into the expected challenges.  As this distribution contains multiple plugins (only a few of which are Kotlin related), applying a base plugin on the root project of a multi-project build fails with NoClassDefFoundError for Kotlin classes, even though the root project isn’t applying any Kotlin plugins.  Adding the Kotlin plugin to the root project resolves this, but isnt’ desirable (the root project isn’t a Kotlin project and should not have/need that plugin).

So your initial error is probably due to that too: root project has the transitive embeddedKotlinVersion in its classpath, and subproject tries to add another version.
As a workaround (for both approaches), you can probably add the Kotlin plugin to the root project with `apply(false)` to only add it to the buildscript classpath without applying the plugin.
I'd look at why the root project throws a NoClassDefFoundError though: if it's supposed to work on projects which aren't Kotlin projects, then it should be able to detect whether the project is a Kotlin project and only access those classes in those cases.

thanks.  it does work with `apply(false)`.  It isn’t the root project that fails with that error, it’s subprojects when the root project is missing the Kotlin plugin (even though subprojects have it applied, before my plugin).

Ah, the dreaded plugin classloader hierarchy… subproject plugin classloader _inherits_ parent project classloader, so your plugin is from the parent classloader and cannot see Kotlin plugin's classes in child classloader.
<https://dev.to/autonomousapps/abusing-gradle-s-class-loader-hierarchy-for-fun-and-profit-oca>

KGP wants to be loaded in the root project anyway though

Specifically we're using the gradle docker plugin and trying to create a plugin that loads a test container.

I'm not sure if it's the best approach, but I think using a javaexec might be the best solution.

Inject a `FileFactory` if you're in a task
```    @get:Inject
    abstract val fileFactory: FileFactory

@TaskAction
fun task() {
    fileFactory.file(project.file("filename.txt"))```

Alternatively if you can use a `RegularFileProperty`, then
```project.objects.fileProperty()```
should work.

I don't think `FileFactory` is a thing...at least not in the javadoc.

Oups, it's internal. Ya probably best not to rely on it.

The file is actually from a resolved configuration, and I'm setting a RegularFileProperty value to it. This works :face_vomiting:
```Provider&lt;File&gt; ep = project.getProviders().provider(epConf::getSingleFile);
t.getTemplateFile().set(project.getLayout().file(ep));```

Use RegularFileProperty#fileValue or fileProvider instead of project.getLayout().file() ?

Thanks Thomas, I missed the existence of the `file*` methods.

Just in case you wonder, the issue I run into is that my Gradle plugin is split into several ones (base plugin + another that builds on top of base).

`io.qameta.allure` =&gt; dependsOn =&gt; `io.qameta.allure-adapter-base` =&gt; dependsOn =&gt; `io.qameta.allure-base`
Historically, I used `implementation(project(..))` dependencies across those modules, and it worked, however, it turns out precompiled script plugins do not generate accessors when `-base` is `implementation`  dependency.

Here’s the issue: <https://github.com/allure-framework/allure-gradle/issues/84>

`GradleRunner#getPluginClasspath` does not really help since it includes all the runtime dependencies :confused:

I usually publish my plugin to maven local, add that as a repo, and depend on it like it's any external dependency.

I ended up with adding the main project as `includeBuild` to a sandbox that builds the precompiled.

The gotcha there is that `implementation("$id:$id.gradle.plugin")` does not work when using `includeBuild`, and dependencySubstitution does not help either (which is yet another issue :joy:): <https://github.com/allure-framework/allure-gradle/pull/97/files#diff-8526501554aa4e1e5fab26527313bd433dab89164723f7fa2bfe0867e2f27eb7R10-R13>

I would say that you can't (and that you have a design problem rather than a technical one). I suppose that you can actually read the current value (eagerly!) and then set it (same with any kind of Property btw, not just ListProperty)

I know I have a design problem, but it is too much to change it right now :sweat_smile:

Seems usual workaround by using `afterEvaluate { .. }` works here

WDYT of adding another list property that builds upon the original one?
In other words, ListProperty.map { … } which creates the “updated list property”

actually your idea with additional input is better then using `afterEvaluate { ... }` . Thank you!

As long as your plugin doesn't register the task in another project, then `Task::getProject` should be the same.

But I'd say the task shouldn't access the project by itself, either the plugin should configure task's properties with values computed from the project: or the task should be injected services rather than accessing them on the project (e.g. inject `ObjectFactory` or `ExecOperations` rather than using `Project#getObjects` resp. `Project#exec`/`Project#javaExec`); see <https://docs.gradle.org/current/userguide/custom_gradle_types.html#service_injection>

it is not inside the task class, in the registering block

```register&lt;Foo&gt;("someTask") {
   this.project
   target
}```

I would use the `project` passed to the plugin then, but that shouldn't really matter (even with configuration cache enabled afaict)

I tried running `./gradlew.bat --stop` in-between the tests, however, it stops all the daemons, not just idle ones, so the build terminates

`--no-daemon`  is not supported by `GradleRunner`, and `org.gradle.daemon=false` makes no difference either

Adding `--no-daemon` to runner args along with tasks doesn't help?

`--no-daemon` is explicitly unsupported in `GradleRunner` , and it just throws an error

I suppose you're testing for a list of Gradle versions? (or otherwise use some arguments that prevent reusing the same daemon) Maybe you could test one at once, with several runs of Gradle so you could `--stop` in between tests (I personally use a job matrix so those tests can run in parallel)

I’m using parameterized test over gradle verisons

maybe a stupid idea, but have you tried executing `System.exit(0)` in the context of a build under test?

&gt; The TestKit uses dedicated daemon processes that are automatically shut down after test execution.
&gt; — <https://docs.gradle.org/current/userguide/test_kit.html#sec:controlling_the_build_environment>
I stopped using TestKit and instead use Java `ProcessBuilder` to start an actual Gradle build. There simply are too many things that do not work properly in TestKit and one ends up doing more debugging of it, instead of testing. I recommend to do the same here, as it is going to unlock what you need.

You could call in `tearDown()` - `ConnectorServices.reset()`. But this will stop *all* currently running Gradle daemons, so may not work for you if you are running tests in parallel

I am puzzled a this point.

It looks like the machine had enough RAM (~2G of 7G was reported as “free” just before the failure)
I was able to customize testkit-based daemons via `((DefaultGradleRunner) runner).withJvmArguments("-Xmx1024m")`, however it did not help either.

So I just stopped, and I excluded “windows + java8” from CI matrix.

<@U01A244DYUF>, do you think you could suggest pointers to “no-testkit” style of writing tests? I can imagine several ways, however, I don’t want to reinvent the wheel :)

GradleDistributionTest.kt

I do have plans on releasing a library, currently I'm only using it in closed source projects. Let me dump what I have into a public repository so that you can copy it until I find the time to package it up. (Gimme a few minutes for that.)

For me the problem started while testing custom Gradle distributions that are auto-configuring repositories. This fails miserably with the `GradleRunner` because repositories cannot be changed at all with it. Another huge issue is that Gradle keeps locking things all over and relies on the advisory locking of POSIX systems, which obviously fails miserably too on systems with mandatory locking (e.g. Windows). The testing kit that is based on Java's `ProcessBuilder` solves all issues. Builds work exactly as you think they work. Concurrent/parallel execution is no problem at all, since each build is hermetic. The Gradle process either finishes successfully, fails, or gets killed by a timeout. Piping input is no problem, so, testing of interactive things is no problem either.

The only _downside_ (if it can be considered one) is that you cannot access the outcome of individual tasks. This would require some output parser, which is doable, but probably going to be fragile over time. I use <https://strikt.io/> which has many great assertions for working with text. I either check the output with `contains` and `matches` or massage the output so that the build only prints what I actually need, then I can use `isEqualTo`. Attached is an example test for a closed source custom Gradle distribution (there's nothing special in this, so posting it won't disclose anything interesting other than the company). Note that I'm running all tests in parallel on my AMD Ryzen 9 3900X (24 tests at the same time) on Windows (mandatory locking) without any hick-up.

Here you go :blush: <https://github.com/Fleshgrinder/gradle-test-kit>

The important part of the whole thing is <https://github.com/Fleshgrinder/gradle-test-kit/blob/d8ddea6b20874552fd8db8647be0c2dcda92fa39/src/main/kotlin/com/fleshgrinder/gradle/test/kit/GradleBuildResult.kt#L23-L54|the process building here>. All the rest is just convenience stuff (and I haven't fully figured out what a good API looks like, as stated before, this is just the current state of affairs).

On Windows everything works perfect, but on Linux I'm running into all kinds of timeout issues. I also found an endless amount of issues in regards to running multiple Gradle build processes in parallel. Seems like parallel execution is not a good idea with Gradle, no matter what. :disappointed:

• <https://github.com/gradle/gradle/issues/10906>
• <https://github.com/gradle/gradle/issues/20055>
• ...

Untitled.txt

This is a build with an isolated `GRADLE_USER_HOME` and `--no-daemon`. :shrug:

Using `junit.jupiter.execution.parallel.config.dynamic.factor=0.5` seems to resolve the issue on Linux. I'll continue working on this. :wink:

Not running tests in parallel always works, no matter what.

Basically <https://github.com/bmuschko/gradle-docker-plugin> this plugin doesn't seem to like gradle 6, and it's only a few classes that end up complaining.

<https://bmuschko.github.io/gradle-docker-plugin/current/user-guide/> says “This plugin requires Gradle &gt;= 7.4.0 to work properly.”

Yeah it's the `requires Gradle &gt;= 7.4.0` that I'm trying to upstream a few fixes for :wink:

generally you should use the worker API for parallel task actions (but it doesn't support programmatic cancellation either)

Have you tried using `Property` for the properties?
It works great for both task properties and extension properties: <https://docs.gradle.org/current/userguide/lazy_configuration.html#lazy_properties>

It does resolve many issues with “value is set too late”

At the same time, there’s `MapProperty` , so you could replace your own `mutableMapOf` with `MapProperty`

I personally want to avoid properties since `property = value` is nicer to work with compared to `property.set(value)`, but I could try using them I guess :confused:

`.set` is not that harmful, however, it makes the use of plugin WAY better. It is a real pain to integrate several plugins that use primitive fields instead of `Property`

Then, it looks like the upcoming Gradle (8?) would include a Kotlin plugin that would enable use of `Property&lt;..&gt;` via regular `prop = value` syntax: <https://github.com/JetBrains/kotlin/pull/4933>

How would I set defaults on the properties of this extension?

For instance, via

```init {
   prop.convention(...)
}```
See <https://github.com/sigstore/sigstore-java/blob/8bf92a22ffd326834b5d61acff00abe17817e71a/sigstore-gradle/sigstore-gradle-sign-base-plugin/src/main/kotlin/dev/sigstore/sign/SigstoreSignExtension.kt#L39-L42|https://github.com/sigstore/sigstore-java/blob/8bf92a22ffd326834b5d61acff00abe1781[…]ugin/src/main/kotlin/dev/sigstore/sign/SigstoreSignExtension.kt>

I switched to properties, but the map entries still don't get passed to the task

The one that shows "properties not passing to the task"

I have the plugin applied in a local project with properties added, but running `gradle setupMetadata` doesn't list the properties, that's the test

What do you mean by "doesn't list the properties"? Where are the properties configured?

on the extension using metadata() in my build.gradle.kts, as documented in the project's readme

Would you please show a sample project that configures the extension and that "doesn't list the properties"?

Here it is as part of a larger project
```import com.martmists.kpy.plugin.PythonVersion
import org.jetbrains.kotlin.gradle.dsl.KotlinCommonOptions
import org.jetbrains.kotlin.gradle.dsl.KotlinCompile
import org.jetbrains.kotlin.gradle.targets.js.webpack.KotlinWebpack
import org.jetbrains.kotlin.gradle.tasks.Kotlin2JsCompile

plugins {
    kotlin("multiplatform") version "1.7.0"
    id("com.martmists.kpy.kpy-plugin") version "0.4.5"
}

group = "com.martmists"
version = "1.0.0"
val production: String? by project
val isDevelopment = production != "true"

repositories {
    mavenCentral()
}

kotlin {
    js("frontend", IR) {
        browser {
            commonWebpackConfig {
                outputFileName = "index.js"
            }
        }
        binaries.executable()
    }

    linuxX64("python") {
        binaries {
            staticLib(listOf(if (isDevelopment) DEBUG else RELEASE)) {
                binaryOptions["memoryModel"] = "experimental"
            }
        }
    }

    linuxX64("backend") {
        binaries {
            executable(listOf(if (isDevelopment) DEBUG else RELEASE)){
                binaryOptions["memoryModel"] = "experimental"
            }
        }
    }

    sourceSets {
        val frontendMain by getting {
            dependencies {
                compileOnly(npm("decky-frontend-lib", "3.0.0"))

                // Kotlin Wrappers
                for ((module, version) in listOf(
                    "react" to "18.2.0-pre.383",
                    "react-dom" to "18.2.0-pre.383",
                )) {
                    implementation("org.jetbrains.kotlin-wrappers:kotlin-$module:$version")
                }
            }
        }
    }
}

val plugin by sourceSets.creating {
    resources {
        srcDir("src/plugin")
    }
}
val python by sourceSets.creating {
    resources {
        srcDir("src/pythonMain/python")
    }
}

tasks {
    withType&lt;Kotlin2JsCompile&gt; {
        kotlinOptions {
            freeCompilerArgs += listOfNotNull(
                if (isDevelopment) null else "-Xir-minimized-member-names",
                if (isDevelopment) null else "-Xir-dce",
                if (isDevelopment) "-source-map" else null,
                if (isDevelopment) "-source-map-embed-sources=always" else null,
                "-Xir-build-cache",
                "-opt-in=kotlin.js.ExperimentalJsExport",
            )
        }
    }

    withType&lt;KotlinCompile&lt;KotlinCommonOptions&gt;&gt; {
        kotlinOptions {
            freeCompilerArgs += listOfNotNull(
                "-Xenable-incremental-compilation",
            )
        }
    }

    val webpackTask by named&lt;KotlinWebpack&gt;("frontendBrowser${if (isDevelopment) "Development" else "Production"}Webpack") {
        outputs.upToDateWhen { false }
    }

    val buildPython by creating(Exec::class) {
        dependsOn("pythonBinaries")

        executable = "python3.10"
        args("src/pythonMain/resources/setup.py", "build", "--build-lib=${buildDir}/python")

        outputs.dir("${buildDir}/python")
    }

    val installPythonDev by creating(Exec::class) {
        dependsOn(buildPython)

        executable = "python3.10"
        args("-m", "pip", "install", "--use-pep517", "-e", "src/pythonMain/resources")
    }

    val bundle by creating(Copy::class) {
        dependsOn(webpackTask, named("backendBinaries"))

        destinationDir = file("${buildDir}/bundle")

        into(project.name) {
            from(
                python.resources,
                buildPython,
                "README.md",
                "LICENSE",
            )
            from(plugin.resources){
                filesMatching("package.json") {
                    expand(mapOf(
                        "project" to project.name.toLowerCase(),
                        "version" to project.version,
                    ))
                }
            }
            into("dist") {
                from(webpackTask.destinationDirectory) {
//                    filter {
//                        it.replace("${project.name} =", "return")
//                    }
                }
            }
            into("bin") {
                from("${buildDir}/bin/backend/${if (isDevelopment) "debug" else "release"}Executable") {
                    rename(".*\\.kexe", "backend")
                }
            }
        }
    }

    val bundleZip by creating(Zip::class) {
        dependsOn(bundle)

        archiveFileName.set("${project.name}-${project.version}.zip")
        destinationDirectory.set(file("${buildDir}/bundled"))
        from(bundle.destinationDir)
    }

    val build by getting {
        dependsOn(bundleZip)
    }
}

kpy {
    target.set("python")
    moduleName.set("plugin")
    metadata("debug", if (isDevelopment) "True" else "False")
}```

Now as you can see on the bottom, `debug` should be set to `True/False` , but it's not present in the setupMetadaa output:
```project_name = "plugin"
project_version = "1.0.0"
build_dir = "/home/mart/git/DeckPlugin/build"
target = "python"
has_stubs = True```

At same time, if there is a diff, that mean that the task output has changed, so it can't be up to date. But not sure if you can print something special based on the task status 

Anyway, I have used extensively jgit with no issues in gradle plugins 

jgit has many limitations (e.g. lack of worktree support, etc, etc), so it would probably make sense to go for `git` commandline

publish your library to a local maven repo and declare that local maven repo in your test projects

the answer to your original question is Yes, you can do that.  you can use `withPluginClasspath()` and pass in the Files that you need to include from your library (such as the files in `build/classes/groovy/main`

Yea, I used to do that and got bitten multiple times. Especially when you start relocating your plugin, etc... The local maven solution is as close as possible to what users are doing so I like this better. But whatever floats your boat!

<@U02162U7WCX> just for me to understand better: I was on that path but I got stuck on fetching the list of paths with the compilation output. How do I do that in a safe way? (ie a programmatic way that's robust to relocations and path customization)

you can customize what gets pulled in by `withPluginClasspath()` with
```tasks.pluginUnderTestMetadata {
    pluginClasspath.from(...)
}
```
(in the plugin project's build script, not the plugin or test itself)

Thanks <@U019L3NM4PK> I'll give it a try :)

ive noticed some tasks, like the maven publish tasks such as "publishMavenJavaPublicationToMavenRepository" don't exist if i try to add a dependency on them, even within a plugins.withType closure

but they do exist if i put that code in an afterEvaluate

What do you mean with "even within a plugins.withType closure" exactly?
What is your code that is not working?

in this case, like
```        plugins.withType&lt;MavenPublishPlugin&gt; {
            
        }```

i dont have an exact snippet b/c i have avoid writing this code for these types of tasks because of this issue, but i am rethinking it and wondering if ther eis a better way

to make sure the code doesnt run until after the plugin is loaded

i can experiment aroudn with tasks.named, but i was wiondering if anyone knew offhand if th at works

`tasks.named` only works if a task is already registered.

ok. yeah i think the issues with these tasks are that they arent registered explictly

If you can restrict by task type (to avoid realizing all tasks) you can do things like `tasks.withType&lt;...&gt;().matching { it.name == "..." }`

its some kind of pattern that is dynamically evaluated

No, they should be registered explicitly, depending on the configuration of the publishing extension

Dynamic tasks handled by task rules are a different topic maybe

ah, well it could be that i unfairly lumped these tasks in then. it does seem they are registered during plugin application

<https://github.com/gradle/gradle/blob/master/subprojects/maven/src/main/java/org/gradle/api/publish/maven/plugins/MavenPublishPlugin.java>

but how is it possible, if additional repositories and publications can be declared int he build script? would those configurations evaluate after the plugin is applied?

Actually all these work here:
```val foo by tasks.registering {
    dependsOn(tasks.withType&lt;PublishToMavenRepository&gt;().matching { it.name == "publishMavenJavaPublicationToMavenRepository" })
}```
```val foo by tasks.registering {
    dependsOn(tasks.named("publishMavenJavaPublicationToMavenRepository"))
}```
```val foo by tasks.registering {
    dependsOn("publishMavenJavaPublicationToMavenRepository")
}```
The first of course realizes all tasks of that type.

The second does only woks here because of task configuration avoidance.
If `tasks.named` does not work, that is a sign that the dependee task is maybe realized a little bit early and thus the dependency is not yet registered by the plugin.
But first and third way even work with `tasks.creating`  and thus eager realization.

And it also depends on order.
The task is not registered on plugin application.
The plugin listens to instances added to the `publications` and `repositories` containers of the `publishing` extension and if either is added, the respective tasks are registered.

So in my tries I had the `foo` configuration before the `publishing` block.
But if I move it after the `publishing` block, again all three variants work fine, even with eager configuration, as then the tasks are registered already.

vote for my CC Hackathon PR  :slightly_smiling_face: (<#CA7UM03V3|>)

To do this, we wrap the Kotlin configuration blocks with:
```public static void whenKotlinSupportPresent(Project p, Consumer&lt;? super Project&gt; action) {
        p.getPluginManager().withPlugin("org.jetbrains.kotlin.jvm", unused -&gt; action.accept(p));
    }```

so the lambda is only executed if the Kotlin plugin has been applied.

```tasks.withType(KotlinCompile.class).configureEach(kotlinCompile -&gt; { ... }```

which should work because the Kotlin plugin _has been applied_, but it fails with: `java.lang.ClassNotFoundException: org.jetbrains.kotlin.gradle.dsl.KotlinCompile` :fearful:

<https://ge.micronaut.io/s/wr3ktmpvxu5wq/failure?focused-exception-line=0-1-2-3-4-5-0>

I think the problem comes from the fact our plugin uses a `compileOnly` dependency on the Kotlin plugin. It does this because we only want to _react_ to the presence of the plugin, but then we need to access the types to be able to do something.

BUT, it seems this only works if the plugins are applied in a particular order...

<https://github.com/gradle/gradle/issues/18848|https://github.com/gradle/gradle/issues/18848>

I don't think this issue is relevant here

because I don't want to execute something after all plugins are appleid

the plugin _is_ applied, but its types are not visible from mine

I created that issue because I had a plugins order issue and being unable to check if there is multiple plugins applied and not only one. My solution in my precompiled plugin was applying my plugin the last one which was not ideal…

in other words, this works:
```plugins {
    alias libs.plugins.kotlin.jvm
    alias libs.plugins.kotlin.kapt
    alias libs.plugins.kotlin.allopen
    id 'io.micronaut.build.internal.oraclecloud-example'
}```
and this doesn't:
```plugins {
    id 'io.micronaut.build.internal.oraclecloud-example'
    alias libs.plugins.kotlin.jvm
    alias libs.plugins.kotlin.kapt
    alias libs.plugins.kotlin.allopen
}```
but because of _classloaders_

I don't want to check if multiple plugins are applied or not, I just want my plugin to not throw a `ClassNotFoundException`

actually re-ordering doesn't seem to work either :confused:

it works if I put all plugins on my script classpath (via `buildSrc`). Annoying.

Hm, very strange. I would have expected this to work. Maybe you found a bug?

I suspect this is caused by the the “root build” classloader thing. The subproject inherits the root loader. Your `io-micronaut...`  is loaded there and operates in this context. The Kotlin stuff if loaded on top only in the subproject loader. That explains for me why it is not seen - which would be what I expect (although I think it’s a shitty behaviour, if I am correct).

probably something like this, yes. The `io.micronaut` plugin is loaded in `buildSrc`

&gt; it works if I put all plugins on my script classpath (via `buildSrc`). Annoying.
That would confirm it. “buildSrc” is always in the root.

`buildSrc` is even one level higher than root iirc

I don’t know how to deal with this nicely in a plugin you share publicly. Other then recommending to users to make sure that all plugins involved are loaded (not necessarily applied) in the root. You could catch the ClassNotFound and print a corresponding advice as error. :confused:

Maybe you could use the thread context class loader if it is set by Gradle.
So that you have some "configure kotlin" class that you load from the thread context class loader in your plugin and that can then see the Kotlin classes?

That's my problem, I can't really know how the plugin is going to be applied :confused:

&gt; So that you have some "configure kotlin" class that you load from the thread context class loader in your plugin and that can then see the Kotlin classes?
That would be extremely cumbersome to write the configuration code, basically using reflection

&gt; loaded in the root.
iirc that wouldn't work if buildSrc is involved as it is the parent of the root project class loader

&gt; That would be extremely cumbersome to write the configuration code, basically using reflection
Nah, not with reflection, by loading your class from the lower class loader

Ah, no, I saying rubbish, it would still load from the parent class loader with the same problem

You would need to inject the class to the respective class loader

So you would need to create a new class loader that has the thread context class loader (if it is set properly) as parent class loader and then load your configure class from that new class loader, then it should properly see the classes without reflection

I won’t try to solve this in my plugin (if it was me). I would see to produce an error with good advice. Might probably be different depending on situation then. Things like:
```If plugin applied in root -&gt; 'apply id("other-plugin") false' in root
When used through buildSrc -&gt; Add dependency 'implementation("other.plugin:coordinates")' to "buildSrc/build.gradle"```

I would try to solve it, but I'm crazy :smile:

This is a general Gradle plugin system design issue that should be addressed in Gradle itself

Maven makes it easier, since all plugins are loaded in isolation, and none can see the classes of another. With great powers come great responsibilities, said Hans.

one could argue that the other way to fix this is to have a plugin for each "cross configure" aspect, that users would have to explicitly apply. But I don't like much the "explicit" here.

and that also greatly multiplies the number of plugins to publish and understand

all in all it's about finding a reasonable tradeoff, but that error is mystifying :smile:

Couldn't you make them separate plugins that you then apply in reaction to other plugins being applied, or was that also not possible? I mean to remember some problem there.

I have experienced a very similar problem with my plugin, and I even asked about it in this slack a long time ago. The only workaround that made sense was telling users to do this in their root build.gradle:
```plugins {
  id 'kotlin' version 'whatever' apply false
}```
because it is indeed a classloader issue where my plugin couldn't see the kotlin classes

when you apply `kotlin` without `jvm`, `multiplatform`  and so on, suffix, it is like applying a general one?

I didn't know that exist a `kotlin` id plugin

Note on this - AFAIK all Kotlin plugins are packaged in one Jar. Therefore, it does not matter which one you use in an “apply false” as Gradle loads the complete Jar onto he plugin classpath.

If you use _buildSrc_, or an included build for plugins, you can instead add a dependency to the plugin Jar (which is more elegant imo):
`implementation("org.jetbrains.kotlin:kotlin-gradle-plugin")`  in _buildSrc/build.gradle_

Not all kotlin plugins are in the same jar IIRC, all-open is another one

(just saying because it seemed that everything was reduced to a single jar)

Let me ask a naive question, can't you get the class loader from the target project?

To be fair in a internal plugin I did just require people to include AGP and kotlin in their buildSrc or root project, but now that I have smarter people around I'm very curious

As I said above, even if you can get the class loader from the target project, for example if it is set as thread context class loader, that's just half the rent.

If you load a class from your plugin through that class loader it wouldn't change anything, as class loaders first ask their parent class loader for the requested class and the highest class loader that finds it returns is, still only able to access classes from its own or higher class loaders.

You need to have a separate class that does the configuration that is not part of your normal plugin class files (due to the parent-first delegation), then you need to create a new class loader that has the target project class loader as parent and adds the configuration class, then you load that configuration class from your new class loader. This class can then access the classes in the higher class loader and if it implements some interface that is actually available in your normal plugin classes, then you can also conveniently control that configuration class and go on without any reflection and without requiring the user to do something special. :slightly_smiling_face:

I guess one of the ways to workaround the behavior is to use `reflection` API to configure target plugins.
Even though it kind of defeats the reason to have `compileOnly`, it might be viable if you do not need to use `KotlinCompile`-like classes a lot.

Of course, the point here was to avoid reflection :-)

My use case is that `kotlin-dsl`  is using `org.jetbrains.kotlin:kotlin-sam-with-receiver` with the version `1.7.10`, so it is not building due being incompatible with the new update of Kotlin `1.7.20`

`kotlin-dsl` is also incompatible with `1.7.10` as it uses an older version. And, no, it's not possible, because it needs to be resolved and included in the plugins classpath. It's like using Groovy 4 in Groovy 3, or Java 18 in Java 8.

But after upgrading everything to 1.7.20, to fix issues, I am forced to add to the buildscript classpath of each consumer of my plugin the artifact `org.jetbrains.kotlin:kotlin-sam-with-receiver` with the version `1.7.20`, which is annoying

So I guess I need to wait until kotlin dsl is updated internally in some way to remove that limitation

Gradle currently ships with Kotlin `1.6.21`, that is `2.3.3` of the `kotlin-dsl`. Including `3.1.0` in your dependencies is not changing that, it just leads to more incompatibility whenever your plugin is included in any Gradle build. Until a new Gradle version is released with the updated `kotlin-dsl` that is.

Then I think the correct approach is dropping Kotlin DSL and use the java gradle plugin to avoid those incompatibilities

Indeed it is, or to use the correct versions, which is actually simple since Gradle provides variables to get them:
`build.gradle.kts`
```println(org.gradle.kotlin.dsl.embeddedKotlinVersion)
println(org.gradle.kotlin.dsl.support.expectedKotlinDslPluginsVersion)```
The caveat is that the plugin you build is now tied to Gradle releases that contain a compatible Kotlin version. This can get hairy very quickly, since Kotlin evolves extremely fast and does not have the stability of Java: <https://kotlinlang.org/docs/kotlin-evolution.html>

That said, if you are willing to simply always require latest of everything, and update your things in a timely manner than you can easily build Kotlin libraries and plugins that are Gradle compatible.
`gradle-compatible-kotlin-library/build.gradle.kts`
```plugins {
  kotlin("jvm") version embeddedKotlinVersion
}```
`gradle-compatible-kotlin-plugin/build.gradle.kts`
```plugins {
  `kotlin-dsl`
}```
`gradle-compatible-plugin-that-applies-the-kotlin-dsl/build.gradle.kts`
```import org.gradle.kotlin.dsl.support.expectedKotlinDslPluginsVersion

plugins {
  `kotlin-dsl`
}

dependencies {
  api("org.gradle.kotlin:gradle-kotlin-dsl-plugins:$expectedKotlinDslPluginsVersion")
}```


Thank you for your detailed answer :slightly_smiling_face:

One last thing, always build your plugins with Java 8 as the target, or your users are going to see incompatibilities too. Gradle expects Java 8 source/target compatibility. If you build against any other Java version and use `inline` for anything it's going to result in an error on the consumer of your plugin.

Fun fact: Gradle itself is still built against Java 6. :stuck_out_tongue:

I think I can't because I am using some libraries and plugins that are using Java 11 (JGit, Spotless...) :disappointed:

That's fine, the incompatibility only has to do with Kotlin generated bytecode for the Gradle plugin itself, not any libraries or anything else. The thing is that Gradle compiles all `kts` scripts (e.g. `build.gradle.kts`, `settings.gradle.kts`) with `jvmTarget = 8` and if a plugin is compiled with e.g. `jvmTarget = 11` certain Kotlin features start to fail since they cannot be compiled. Most prominent (and maybe even the only feature :shrug: ) is `inline`. The error message you get is `cannot inline bytecode 11 into 8` (or similar).

You can still use dependencies and libraries that target greater Java versions, given that all users of your plugin know that they have to run the Gradle build with a newer Java version.

Gradle would actually fail and let the user know that there is no compatible dependency if all dependencies have proper Gradle metadata but JGit and Spotless afaik don't since they use Maven.

I have curious about why the community is still stick to Java 8 and not moving forward to Java 11

Why not Java 19? The biggest problem are enterprises who are super cautious about upgrading anything, and Java is all about enterprise.

Because Java 19 was released "yesterday". Java 11 has a lot of years. If a company is not trying to update its Java version which is core, why they are worried about not being able to use newer versions of a plugin that is working for them on an old version?

I know that Java 19 was released yesterday, what I meant to say by it is why we all are not moving a little faster than just 11. LTS is hoax unless one is willing to pay a fortune to Oracle, so 18 would be the realistic target for the entire community to be at, and 19 is what we would be upgrading to right now.

The situation with Gradle is actually even more interesting. Gradle could easily move to 18 as a min. requirement. Simply because the toolchain required for compiling the code in question is totally decoupled from the Java that is used to run Gradle itself. It's no problem to use Java 18 for Gradle and compile, build, and test a Java 8 library.

tbh, I try to use all latest versions, but I don't work in a larger company with a large legacy application, what I am sure is I am using Java 18 in all my machines and I am going to move to 19 asap (it has improved build times based on some comments).

But in all projects I have participated, the Kotlin last version adoption was delayed less than 1 month as maximum time for example.

I said Java 11 to be reasonable with legacy apps, for me it would be fine to have a newer one as a personal approach (something I am already doing with Kotlin, if renovate's PR is green, it is merged).