From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

`build.gradle.kts`
```plugins {
    `kotlin-dsl`
}
dependencies {
    compileOnly("com.android.tools.build:gradle-api:7.1.3")
    testImplementation("junit:junit:4.13.2")
}```
`src/main/kotlin/example.gradle.kts`
```val android: com.android.build.api.dsl.ApplicationExtension by extensions
println(android)```
`src/test/kotlin/ExamplePluginTest.kt`
```import java.io.File
import org.gradle.testkit.runner.GradleRunner
import org.junit.Rule
import org.junit.Test
import org.junit.rules.TemporaryFolder

class ExamplePluginTest {
    @get:Rule
    val tempDir = TemporaryFolder()

    @Test
    fun test() {
        File(tempDir.root, "settings.gradle").writeText(
            """
            |include 'app'
            |""".trimMargin()
        )
        File(tempDir.root, "build.gradle").writeText(
            """
            |buildscript {
            |    repositories {
            |        google()
            |        mavenCentral()
            |    }
            |    dependencies {
            |        classpath 'com.android.tools.build:gradle:7.1.3'
            |    }
            |}
            |""".trimMargin()
        )
        File(tempDir.newFolder("app"), "build.gradle").writeText(
            """
            |plugins {
            |    id 'com.android.application'
            |    id 'example'
            |}
            |""".trimMargin()
        )
        GradleRunner.create()
            .withProjectDir(tempDir.root)
            .withPluginClasspath()
            .forwardOutput()
            .build()
    }
}```

the test fails with
```FAILURE: Build failed with an exception.

* What went wrong:
* com/android/build/api/dsl/ApplicationExtension
* &gt; com.android.build.api.dsl.ApplicationExtension
*
* * Try:
* &gt; Run with --stacktrace option to get the stack trace.
* &gt; Run with --info or --debug option to get more log output.
* &gt; Run with --scan to get full insights.
*
* * Get more help at <https://help.gradle.org>
*
* BUILD FAILED in 3s
*```

I have a workaround by using
```        File(tempDir.root, "build.gradle").writeText(
            """
            |buildscript {
            |    repositories {
            |        google()
            |        mavenCentral()
            |    }
            |    dependencies {
            |        classpath 'com.android.tools.build:gradle:7.1.3'
            |        classpath files(${
                PluginUnderTestMetadataReading.readImplementationClasspath(javaClass.classLoader)
                    .joinToString(separator = ", ") { "'${it.absolutePath}'" }
            })
            |    }
            |}
            |""".trimMargin()
        )```
instead of `.withPluginClasspath()`, but I'm not sure if it's a good approach since `PluginUnderTestMetadataReading` is in an internal package

why is whatever `GradleRunner.withPluginClasspath()` is doing behaving differently from this?

changing `build.gradle.kts` to
```plugins {
    `kotlin-dsl`
}
dependencies {
    implementation("com.android.tools.build:gradle-api:7.1.3")
    testImplementation("junit:junit:4.13.2")
}```
doesn't fix things, it just changes the error

Implementation doesn't change it? It's weird

About the compileOnly is the usual

<https://github.com/Flank/simple-flank/blob/1bb1ef7e790e34d0a8b9cb721954fc7f6028e473/build.gradle.kts#L30|https://github.com/Flank/simple-flank/blob/1bb1ef7e790e34d0a8b9cb721954fc7f6028e473/build.gradle.kts#L30>

implementation changes the problem to a ClassCastException

In any case what you want is very common, and the fix is always adding a configuration to the PluginUnderTestMetadata.

You may just add your compileOnly configuration if it suits you

I was now looking at a repo from <@UES7RHBDZ> but it seems he found another fix (even if I don't see how)

thanks for the tip. I simplified it down to
```tasks.pluginUnderTestMetadata {
    pluginClasspath.from(configurations.compileOnly)
}```
for my use case

In kts I would add the `.configure {...}`

This is <https://github.com/gradle/gradle/issues/7974>

My personal workaround:
```// See <https://github.com/gradle/gradle/issues/7974>
val additionalPluginClasspath by configurations.creating

dependencies {
    compileOnly("com.android.tools.build:gradle:${Version.ANDROID_GRADLE_PLUGIN_VERSION}")
    additionalPluginClasspath("com.android.tools.build:gradle:${Version.ANDROID_GRADLE_PLUGIN_VERSION}")
}

tasks {
    pluginUnderTestMetadata {
        this.pluginClasspath.from(additionalPluginClasspath)
    }
}```
I prefer declaring the dependency twice, in different configurations, than adding `compileOnly` to the `pluginClasspath`, as that allows me to add other things to `compileOnly` (or `compileOnlyApi`) independently of that testing need.
I could also have made `compileOnly.extendsFrom(additionalPluginClasspath)`  fwiw.

yes, I see that in the general case. in my specific case, I only have this one dependency; I can always extend it in the future if needed

I did duplicate the configuration too because I run the tests with different AGP versions (using a gradle property) in some of my plugins (just not in the one I shared)

What I want to know about <@UES7RHBDZ>  solution is how can he run the tests with different AGP versions...

Fwiw: I don't. I don't even *use* AGP myself as I don't do Android stuff (PWA FTW!) so AGP support is _best effort_ (I haven't migrated to the new variant API, and really wonder whether I should, or just remove AGP support from my plugin and let interested people, if any, _rebuild_ it in a new, Android-specific, plugin –based on mine, or maybe not even)

AGP is just a detail here, is an plugin dependency that you don't want to expose transitively

It's an "optional dependency", plugin behaves differently with `java-base` plugin vs AGP.
This is <https://plugins.gradle.org/plugin/net.ltgt.errorprone> fwiw (plugin extends _annotation processor configurations_ and tries to determine which `JavaCompile` tasks are _test_)

AGP support could easily be split into its own plugin _on top of_ mine.

I don't use withPluginClasspath in my OSS plugin (I do use it for some simple cases at work). Instead I just publish my plugin to a local repo and apply it like you would in a normal build. this makes the tests more realistic imo. it also makes it possible for me to instrument my test builds to run against different versions of AGP in the same test invocation

Basically it is always in daemon mode, the question is just whether it is a one-off daemon or a reused / reusable daemon.

I have task using the worker API, and depending on whether gradle is using a reusable daemon I could choose between classloader and process isolation modes, to save time on starting a new process

If I were you, I wouldn't depend on that, especially if it is a published plugin.
Besides that, just that it is a one-off daemon does not mean a new process needs to be started.
process isolation does *not* mean each work item gets a new separate process.
worker process can be reused for other work items with the same requirements.

Also generally, don't try to micro-optimize, especially with depending on impelmentation details like this, without having a problem.
1. make things work - optimally with tests
2. make things nice by refactoring - here tests from 1. are helpful to see the refactoring does not break the logic
3. optimize things like CPU or RAM if, and only if, you have a measurable performance problem
That's the same for basically any software you write.

Of course some things always make sense to do like using `StringBuilder` instead of `StringBuffer` or percompiling regex `Pattern` objects,
or using the task configuration avoidance APIs, and so on.
But sacrificing cleanness by using internals, or depending on implementation details,
or readability by using ugly but 1 picosecond faster constructs should not be done lightly without a measurable problem.

If it nees process isolation to work properly, use project isolation, if classloader isolation is enough, use classloader isolation.

I used something like `if (tasks.names.contain("bar")) tasks.named("foo") { shouldRunAfter("bar") }` but it will work only if task `bar` is at least registered by that time.

Of course, you could try to do it in `afterEvaluate` callback and hope target plugin doesn't register/create task in `afterEvaluate` block itself

currently resorting to afterEvaluate + `if (!pluginManager.hasPlugin(...)) tasks.register('bar')` (that is, if the optional plugin is not registered by then, register an "empty" task in its place)

```tasks.matching { it.name == "bar" }.configureEach(fooTask::shouldRunAfter)
```
is terrible in that it makes all tasks eagerly configured just to find out their name, but it would work even if the task is registered in `afterEvaluate`.

oh actually, could
```tasks.addRule("foo") { name -&gt;
    barTask.shouldRunAfter(name)
}
```
work?

I completely forgot about rules! that might work

this works!
```project.tasks.addRule("Foo stub") { taskName -&gt;
  if (taskName == "foo1" || taskName == "foo2") {
    project.task(taskName)
  }
}```
thanks <@U019L3NM4PK> for the idea

That was the right answer, thanks <@UD13D3DSR>
```abstract class AddTask @Inject constructor(
    @get:Input val projectLayout: ProjectLayout
) : DefaultTask() {```


Just in case you are curious about the actual error. You tried to access `rootdir` from the script object, so the `AddTask` class got automatically a parameter to receive a reference to the enclosing object. And then Gradle couldn't instantiate it and suggested to annotate it with `@Inject` which actually wouldn't have helped in that situation.

By using the injected project layout you do no longer reference the enclosing object and so no extra parameter is added to the constructor.

But projectLayout has access to to root dir?

`val file = projectLayout.projectDirectory.file("some/path").asFile`

Then <@UCLKS610F> can be having issues if he wants the root dir, so the only way is passing it from outsite the task, right?

In his question he said he wants to access `projectDir` or `file("xxx")` and for that `ProjectLayout` is perfectly fine.
Just the code example used `rootDir` which of course is only the same if you are in the root project.

To really get the root dir, you probably have to pass it in, yes

I have a single-project build, I don't really care

Fwiw, tasks all have a `getProject()` too, so:
```tasks.register&lt;AddTask&gt;("add")

abstract class AddTask : DefaultTask() {
    @TaskAction fun test() = println(this.project.rootDir)
}```


Yup: <https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution>

That's something years ago I did using libevent to have a single threaded server, that most of the time was just waiting for the database to reply

possible to use a non-blocking communication layer, such that it isn’t a thread-per-request?

for example, if the calls are HTTPS, using an async HTTPS library would provide for multiple outstanding requests with a fixed set of threads to process responses as they come in, landing in your callbacks.

it isn’t clear, though, what the blocker is in current implementation - is it exhausting a thread pool, and waiting until outstanding requests finish?  Or something else that is causing the blocking?

The issue is that for example CI has just 4 CPUs, so my tasks for the first modules will make everything else wait

number of CPUs shouldn’t affect the threads - you can have hundreds of threads on any number of CPUs (and may want to do so where the threads are waiting on external resources).   Is the task making external HTTP/API calls?

by default, the number of parallel Gradle tasks will match the number of CPUs.  That doesn’t work for tasks that are light on CPU,  waiting on external resources.  This is configurable via `--max-workers=n` or setting `org.gradle.workers.max=n` in a gradle.properties

As plugin author max-workers is not something I can change, I can suggest users, but they have other tasks that may be CPU intensive

I was looking if Gradle had some way of handling this usecase

to my knowledge, Gradle could help via the Workers API or a shared build service that could manage the concurrency.

the shared build service does support a concurrency limit on a per-service basis.

a brief example of a shared build service.  Note that you can do your own caching within it, for repeated calls with the same inputs (if that’s applicable in your case).
```abstract class CdkToolchainBuildService : BuildService&lt;CdkToolchainBuildService.CdkToolchainParams&gt; {

    interface CdkToolchainParams : BuildServiceParameters {
        val cacheService: Property&lt;CacheBuildService&gt;
    }

    private val versionCache = ConcurrentHashMap&lt;String, CdkToolchain&gt;()

    @get:Inject
    abstract val execOperations: ExecOperations

    companion object {
        internal fun registerIfAbsent(
            gradle: Gradle,
            configureAction: Action&lt;BuildServiceSpec&lt;CdkToolchainParams&gt;&gt;
        ): Provider&lt;CdkToolchainBuildService&gt; {
            return gradle.sharedServices.registerIfAbsent(
                "cloudshift-cdk-toolchain",
                CdkToolchainBuildService::class,
                configureAction
            )
        }
    }

    fun resolve(versionProvider: Provider&lt;String&gt;): CdkToolchain {
        require(versionProvider.isPresent) { "Missing version" }
        val version = versionProvider.get()
        require(version.isNotBlank()) { "Empty version"}

        return versionCache.computeIfAbsent(version) {
            // omitted for brevity
        }
    }
}```

by default, there are no concurrency limits but you can specify a level of parallelism if needed (# of tasks concurrently accessing the service)

Thanks for the example, I'll take a look tomorrow and see if I can use it. So far it's something I didn't know (caching with services), so, thank you very much.

I did read about it but, if I understood correctly, in my case it doesn't help.
At the end there is a task that will be using a gradle thread to wait for the service to finish.

The service can have unlimited threads, but tasks are going to be queued until some of them finish and free a Gradle thread/worker

yea.  that’s accurate (and unfortunate).  Needs some thinking on how tasks can be made async.

I could split each task in the starter (throw and forget) and the watcher (reconnect and wait for completion) but then I would rely on luck, if Gradlew decides to start multiple "starter" jobs (from different projects) before starting the "watchers", everything is fine. If not I'm in the same situation as before.

It happens the same with workers API, because as I'm playing with multiple projects I can't have a single task orchestrating the jobs

So probably there is no solution for my concern

Actually...I wonder how Gradle does remote test execution, it's not that different but I guess that's a private API

only thing that comes to mind is, as you suggested, splitting the task in two - the initiator portion asks the service to start a background request (with name ‘project:task’), and a completion task (that would somehow have to run much later, hmmm…) to ask the service for ‘project:task’ result (via a future, which will block if not available).

The issue is that I can't tell gradle to wait in between those 2 tasks...
mustRunAfter(tasks.withType(starter)) won't work between projects

(if that was the option I would need to figure out how to declare my inputs&amp;outputs so the tasks are actually cacheable, but anyway it would be worth)

If it's compileOnly it wouldn't be available at runtime, resulting in ClassNotFound

yeah I understood that much but I also apply the android gradle plugins at the root build.gradle :thinking_face: It is my understanding that the AGP classes should also get on the script classpath by way of subprojects inheriting from the rootProject classpath?

There's a separation between projects; cleanest for each project to specifically setup its config.

The way I reasoned this is that
• The included build will contribute its classes to the script classpath. It gets on the script classpath by means of the plugin that I apply to my sub projects. The `compileOnly` on its AGP dependency makes it so that *only the IPB classes and not agp* are contributed but…
• Applying the android plugin at the root level build.gradle with apply(false) will put the AGP classes in the root and subprojects script classpaths, and it should all work
Obvs this is wrong but I’ll appreciate some insight into why the above is wrong then maybe how to fix it

For added context: if I switch AGP dependency to `implementation`  it can find the BuildConfigField class but then it raises another error that AGP is already on the classpath with an unknown version so I cannot reapply it in my root build.gradle. My root build.gradle does some common setup and I’m a bit hesitant to move that.

Typically a plug-in would have dependent plugins on its implementation configuration, and can itself apply those plugins (or respond to the consuming script applying them).  The root project can also use the AGP, just without a version.

Seems like the plug-in is depending on Gradle internal classes (directly or indirectly) that have changed (moved or removed) between Gradle versions.

Likely an issue with a transitive dependency getting pulled in that isn't compatible with Java 18 (not surprising as jandex is indexing class files).  Compare the build scan output (dependencies list) and check dep versions.

Jandex does not depend in Gradle. The plugin was compiled with Gradle 6.9.1 IIRC. This the only change in dependencies is the Gradle version (7.5-rc-1) used by the consuming project

I don’t have a clue on how the plugin might depend on internal classes, look at the imports

It's reading class files via jandex, tripping over something on the class path that contains that class.

Could add some debug statements to show which class file is being read when the exception occurs.

I get that. The class at fault must be Gradle specific

Yep.  Something pulling in an incompatible chunk of code.  Knowing what class file is referencing that will help identify the cause.

The issue linked to in the ticket should already be the problem.
This is usually not a Gradle "bug", but an intended behavior that causes problems if the Plugin does a bad action.
The "bad action" is if the plugin uses its runtime class path outside of Gradle.
For example if it determines its own class path and then runs some tool like Jandex with it.

Gradle does byte-code transformation of the plugin class path to find violations of configuration cache compatibility or automatically handle reads of system properties as configuration cache input and so on.

This byte-code transformation is not done on each run but it is done once and saved to disk so that it is not done each time the build is run.

If you then use those manipulated jars outside of the Gradle context, the woven in Gradle internals fail to be found with exactly that message.

As it works with 7.4 and fails with 7.5, there is most probably some additional tranformation done that hits the classpath now and didn't hit it before.

Wrong. The plugin does not use its runtime classpath. It reads the compiled classes from the output of the main sourceSet

I just say how this error usually happens, didn't investigate deeply in your plugin :slightly_smiling_face:

I read the linked issue and saw that the problem for that reporter was indeed reading the plugin’s classpath. I understand how that can cause trouble. However, in this case the plugin reads the compiled output using standard file i/o APIs. Is there now in 7.5 a different way to do this?

In the main classes output there should not be this class, unless you use it yourself which is unlikely I'd guess.

Apparently creating a fresh new Quarkus project (via <http://code.quarkus.io|code.quarkus.io>), with Gradle 7.5, jdk 18, and the latest version of the jandex plugin is enough to trigger this issue

&gt;     private FileCollection resolveClasspath() {
&gt;         project.files(
&gt;             new File(JandexMain.protectionDomain.codeSource.location.toURI()).absoluteFile,
&gt;             classpath)
&gt;     }


There it resolves a jar from the plugin class path and then uses that in a `javaexec { ... }` . One point for the hip. :wink:

So, Jandex + its dependencies (if any) must be loaded to run the javaexec process. But the way it’s done it’s not good with 7.5. Should it be done with a Configuration?

Instead for example use a detached configuration to resolve Jandex outside of Gradle class path context, or if you want the user to be able to control the dependency, you can even create a named configuration with the dependency as default dependency, so that the user could overwrite it.

Please do not unconditionally register a repository to ensure it can be resolved. This would conflict with `FAIL_ON_PROJECT_REPOS`, at most make it optional or just require the user configured a repository where the dependency can be found.

The plugin already offers an option to manage the jandex version. Switching to a detached configuration may be the ticket to fix this issue

oh yes, no implicit repos. I learned that lesson years ago

&gt; Switching to a detached configuration may be the ticket to fix this issue
Having a named configuration has the advantage, that the user can specify a complete different dependeny, like changed coordinates of Jandex in the future, or an own fork and so on.
I think the code quality plugins do it like that.
They provide an extension property to configure the version, this version is used for the default dependency of the configuration, but if a user defines own dependencies for that configuration the default dependency is not considered.

Just as a thought. :slightly_smiling_face:

A named configuration plus default dependency? If so the configuration must be resolvable and visible, right?

`visible` for a configuration is extremely misleading.
It is part of the legacy approach to define artifacts.
The `assemble` task creates all artifacts that are attached to configurations that are `visible`.
So iirc you should practically always have it to `false` nowadays.

But `resolvable` yes, you want to resolve it, so don't forget to it to `consumable` false.
The default is `true` for both due to backward compatibility but that combination is legacy too.

Well know I’m confused. I thought visible meant users could refer to it in their build scripts. Now I must set visible &amp; consumable to false? While keeping resolvable to true?

Yeah, that is confusing, as I said, the name is very unlucky chosen.
I had the same thought for years until I learned differently.
Users can always refer to them in their build scripts.
It means in the legacy world "visible to downstream" or however you call it.
So yes, if I'm not mistaken it should be visible false, consumable false, resolvable true.
You can just leave resolvable at its default or you could set it to true for clarities sake.

Actually as long as you don't attach artifacts to the configuration I think it shouldn't matter what `visible` is set to.
But I just always set it to `false` nowadays unless I have a good reason not to.

Are there any kind of migration instruction from previous versions? Trying it out I get
```Cannot add task 'signPluginMavenPublication' as a task with that name already exists.```


`signPluginMavenPublication` is the name of the signing task defined by the signing plugin when you instruct it to sign a publication named `pluginMaven`; since this new version of the Plugin-Publish plugin does all this automatically, if you have the signing plugin applied, I suspect that your build also does it explicitly; you would need to remove that; pls. provide a reproducer, if this doesn’t help

and indeed, migration guide starts to sound like “why didn’t we think of that”…

The "auto-configuration" might be an issue for me as I share the signing configuration between regular modules and plugin modules

FYI we are thinking about ways to best address this problem (ie. we aren’t ignoring you)

Thanks for the update :slightly_smiling_face: . No problem at all!

Let me know if I can help. The project I was testing the rc-2 with is <https://github.com/GradleUp/gr8/blob/main/build-logic/src/main/kotlin/gr8.build.publishing.gradle.kts#L150|this one>

(uses in-memory GPG keys. I have this setup in a lot of other projects to easily pull the keys from Github Actions)

fyi. fix merged in, will be in the next released version

That just got me thinking. Is the code for the Plugin-Publish open source ?

I can find <https://github.com/gradle/gradle/pull/20797|the commit that bumps the version in Gradle> but looks like the code is somewhere else?

yes, the plugin code is in a private repo, unfortunately

FYI RC3 with the fix is out, please give it a try.

For featureEnabled:
Either afterEvaluate or you have to create the tasks and use the flag to enable/disable them

1. One option is to create all the tasks and then enable or disable them based on `featureEnabled`
2. Make `configFile` a property. The Kotlin DSL syntax will have to change to `configFile.set("...")` but  this will allow you to configure task inputs before the extension is configured

About the configFile you can map the property and read the file content, so the tasks can have the inputs you need.
(Considering it's a property as <@U019L3NM4PK> suggested)

You could make everything a method and create the task from your method.
```myPlugin {
  enableFeature(featureEnabled: Boolean, configFile: File)
}```

True, method is a good "hack" to create tasks

as long as your users call the function exactly once

Credit goes to <@UDC9Q9RD0> for suggesting it in <https://gradle-community.slack.com/archives/CAHSN3LDN/p1652546857822089?thread_ts=1652546550.566479&amp;cid=CAHSN3LDN|that other question> I had about convention plugin

it's an option but not one I think scales well to being configurable by other plugins

another option is to use NamedDomainObjectContainer as the type in the extension, with whenObjectAdded callbacks - when a “feature” is added to that container the callback is invoked.

Looks like that the task being itself which decides if it has to be enabled or disabled is the more intuitive and less problematic way to solve this, right?

there's still potential confusion in that users will still see the disabled tasks (e.g. on the cli, as "skipped" in Gradle build output) but it's the less problematic solution IMO

Thank you all for the insights.

Yeah so seems like that the enabled/disabled task approach is the 'preferred'.

<@U019L3NM4PK> for the `configFile`, that's already a Property, I used the Groovy dsl in the example for simplicity.

The problem is that the configFile is a YAML file in my case, that contains paths which I'd like to use as `@InputFile` /`@InputFiles` for some of my tasks.

I could set `configFile` itself as `@InputFile` for the task, but I haven't found an elegant way to parse the content of the provided file + pass the user provided paths as `@InputFile` for the task I'm creating.

ah that's something that I think Gradle can't do a good job of capturing - a dependency both on the contents of the config file and also the contents of a file determined by its contents.

depends on how safe you want to be. you could create yet another task that reads the config file, outputs the collection of files it references, and is untracked; then use that as the other tasks' input files

I don't like disabling tasks, because if the task is in the graph, even if it's disabled, Gradle will run all its dependencies

&gt;  you could create yet another task that reads the config file, outputs the collection of files it references, and is untracked; then use that as the other tasks' input files
Thanks for the hint <@U019L3NM4PK>. I'm still trying to find a way to make it work correctly. Not sure how the first task should _outputs the collection of files it references_. The problem is that this list of file will be known only at execution time and won't be accounted for the UP-TO-DATE checks of the second task

the first task will declare an OutputDirectory and copy the referenced files to it, which will then be a known location that Gradle can use for subsequent tasks' inputs

if you decide that's too overkill, that's fine, but then all of your other tasks will have to read files outside of what Gradle can track

&gt;  and copy the referenced files to it
But then I'll end up copying over and over the files for every build no?

it will, but subsequent tasks should still be skipped if it matches a previous cached run

Yeah but still really suboptimal no? Then the `project.afterEvaluate` is probably better?

doing it in afterEvaluate is going to end up reading all the files anyway (when gradle evaluates whether those tasks can use cache or not)

…with the added bonus of unpredictable ordering of afterEvaluate callbacks :exploding_head:

fixed a similar problem using this pattern (could use different DomainObjectCollection types, and of course the generic type can be whatever works for your case - a String, complex object, etc).
```// extension class
abstract class ProjectsExtension @Inject constructor(private val objectFactory: ObjectFactory) {

    @get:Input
    abstract val buildFiles: DomainObjectSet&lt;File&gt;
}

// plugin code
val projectsExt = extensions.create("projects", ProjectsExtension::class)
projectsExt.buildFiles.whenObjectAdded {
    // do magic here
}```

might not matter (since you're using `File` anyway) but `whenObjectAdded` is not lazy (like `all`)

understood.  the goal there isn’t the laziness - it’s responding to the extension being configured (in this example, a file being added), allowing the callback to then do &lt;whatever&gt; (have used it to register specific tasks, etc).  There doesn’t appear to be a loss of laziness, as this responds to setting items on the extension and acts on them (so long as those actions use lazy configuration for tasks, etc).

Thanks for the heads up <@U02S31Y7J2K>. It sounds like this could potentially do the trick (it's a bit of a bummer that we have to rely on a `DomainObjectSet` to do this).

The annoying part here is that:
1. My `configFile` is actually a `RegularFileProperty`. Changing it to a `DomainObjectSet&lt;File&gt;` would be a breaking change
2. I don't have multiple files, but just one. So modelling it with a `DomainObjectSet` would not reflect reality. I.e. would give the false sense to the user that they can provide multiple `configFile` which in reality they can't

understood.  You can change to use DomainObjectSet&lt;RegularFileProperty&gt; if need be.  yea, for scalar values it’s unfortunate to need to use DomainObjectSet.  Alternately, a function on your extension that does the setup may be cleaner.

For the _paths_ from the config file as input. I would probably do it like this:

• Parse file at config time, but try to be cheap. Maybe you don’t need to parse the whole yaml but just find a string in it with a regex. Or just parse the first lines until you reach the info you need. I would use `project.providers.fileContents(configFile).asText.get()`  (like I did <https://github.com/jjohannes/java-module-dependencies/blob/21aae5fd0d0d6568d37960977873c397b39761d2/src/main/java/de/jjohannes/gradle/moduledependencies/JavaModuleDependenciesPlugin.java#L145|here>). Then when you use configuration cache, the file will be tracked as “input to configuration” and only parsed again if something changed.
Then you can do all of the lazily and you do not need to worry about order. Something like this:

```tasks.register&lt;MyType&gt;("myTaks") {
    someInputFile.set(myPluginExtension.configFile.map {
        parsePathFromFile(it)
    })
}


fun parsePathFromFile(it: Provider&lt;RegularFile&gt;) =
    project.layout.projectDirectory.file(
        providers.fileContents(it.get()).asText.get().substring(0, 10)) // Instead of 'substring' do the real parsing```

If you have more than one file path to read, I think I would still parse each path individually from the content of the config file. So you do not have to deal with some state yourself. Reading the actual file from disc will only be done one time by Gradle _I think_ when you go through the `providers` API.

the `providers`are eager, resolving their contents each time `get()` is called - assuming that it’ only called once, you’d be good, but have the risk of multiple executions.  Have added a memoize capability to providers for expensive operations (parsing, API calls) to enforce at-most-once execution.

```fun &lt;T&gt; Provider&lt;T&gt;.memoize(): Provider&lt;T&gt; {
    return MemoizingProvider(this as ProviderInternal&lt;T&gt;)
}

internal class MemoizingProvider&lt;T&gt;(private val provider: ProviderInternal&lt;T&gt;) :
    AbstractMinimalProvider&lt;T&gt;() {

    private var memoizedValue: ValueSupplier.Value&lt;out T&gt;? = null

    override fun getType(): Class&lt;T&gt;? {
        return provider.type
    }

    override fun calculateOwnValue(consumer: ValueSupplier.ValueConsumer): ValueSupplier.Value&lt;out T&gt; {
        if (memoizedValue == null) {
            memoizedValue = provider.calculateValue(consumer)
        }
        return memoizedValue as ValueSupplier.Value&lt;out T&gt;
    }

    override fun toString(): String {
        return "memoized($provider)"
    }
}```

Thanks for the hints <@U023HSZL88M> you made my day! (I owe you a beer in Hamburg!)

Sure. Happy it helped. Ping me if you are ever around again. :wink:

I _could_ make a wrapper plugin but that's a lot of hoops for nothing?

_“lot of hoops”_ as in creating a one-liner class that `apply(com.example.Plugin.class)`?

I guess the point I'm trying to make is that I'm ok with this being a warning but I don't see a reason to make it an error?

You are publishing a plugin from your project, but the plugin is not in your project?

I have 3 modules:
• plugin-core: contains the actual code
• plugin-external: depends on plugin-core and packaged as external
• plugin-shadow: depends on plugin-core and packaged with relocated dependencies

I kind of like the idea that both `plugin-external` and `plugin-shadow` are "empty shells"/"redirect" to the actual plugin code

But then you are publishing jars of the plugins and you want to publish the markers from another module

It seems weird. I mean, is you had a fatJar (including the 3 modules) it would be fine. 1 jar - one module publishing markers

But right now you have 3 modules publishing jars and only one publishing all the markers

Both `plugin-external` and `plugin-shadow` should publish the markers

( I said everything "wrong" because I didn't read your reply first)

Yea and actually `plugin-shadow` wouldn't be an "empty shell" at all because it would be very fat (well, it would be an almost empty project but once published, it should publish the fat jar, sorry for the confusion)

So for plugin-shadow you don't have this issue

And maybe you could publish plugin-external marker from plugin-core then

But if the plugin descriptor ends up in `plugin-core.jar` , it would make it possible to pull the `plugin-shadow.jar` in the buildscript classpath and still apply `com.example.external` , which could be missleading

I think your use case is very uncommon, and understand that gradle is making the right thing, that said.. it's annoying for you, it's true

Ideally I would publish a single publication with multiple variants but I haven't found a way to easily choose a plugin variant on the consumer side

Hello <@UJH87V76G>, This is indeed a needed feature and has come up internally often. I will check in on the status to give you a better update,

I have gotten up to speed in the problem. It is a hard issue because there are so many different ways that the plugins are downloaded and used. The metrics to show haven't been decided and more core features are being worked. This is an important feature but no one has picked it up with the passion it needs.
That being said if you know someone that wants to tackle the project Gradle is actively higher, one of the new hires can change the bandwidth we have for such features.
I do want the portal to have this, so I made a reminder to bug people every month so at least it will stay at the top of their minds.

Observable.kt

Maybe I don’t understand example enough, what are the use cases for that?

Setting property on extension, allowing for notifying observers that can then create dynamic tasks, etc.

What is a use-case for that? Do you have an example in one of your plugins maybe?

it’s a recurring challenge for plugin authors to make their plugin configurable, where that configuration results in say dynamic task creation.  The example in the snippet above is from a golang plugin, where consumers of the plugin can specify which target platforms they’re building for (via the extension DSL).  Each specified target platform results in a a golang build task being configured/registered accordingly.  All of which has to happen at configuration time.

So it seems like the target platforms would probably be in a DomainObjectCollection and then Gradle would need to provide some automation that at some point in time (end of configuration/before running tasks) it would realize the container and run the corresponding actions (aka register the tasks).

With the callback it would work only if folks only add target platforms, though as soon as they remove a target platform then you can’t remove the task again.

one of the workarounds for this situations is to use a DomainObjectCollection with `whenObjectAdded` as the callback (which is where this all started); however, often the state (storing the list of platforms, in this example) isn’t required - only notifying some code that a platform was added (if you want to remove it, don’t add it in the first place).  Modelling as an Observable makes it clear what the behaviour is.

the challenge is “at some point in time” - Gradle does not have a clear demarcation of “end of configuration”, which is why hooking together these at configuration time is required.

it all boils down to modelling a simple “when x do y” situation, without over-complicating that with lazy properties (this isn’t a situation where laziness is desired) or temporal breakage (afterEvaluate and it’s jankiness or other workarounds).

something like this may work:
```tasks.withType&lt;Task&gt;().configureEach {
	enabled = !name.contains("foo")
}```

this variation better reflects your use case:
```tasks.withType&lt;Task&gt;().configureEach {
  if( name.contains(specificName)) {
	  enabled = false
  }
}```

Nice, however, I must ask, how much less work does Gradle have to do with `tasks.withType&lt;Task&gt;.configureEach` vs `tasks.matching` ? From outside, it seems like they are doing the same amount of work :sweat_smile:

they are doing the same thing - comparing names, just using different approaches.  In the latter case it maintains laziness.

there’s a slightly improved variant, yields same results:
```tasks.configureEach {
  if( name.contains(specificName)) {
	  enabled = false

  }
}```
…calling configureEach on all tasks isn’t often used, but is appropriate in this case.  Skips matching the type of each task (which will be a Task anyhow).

Turns out rolling this yourself isn’t all that much code:
```fun TaskCollection&lt;Task&gt;.lazyMatch(
    predicate: (Task).() -&gt; Boolean,
    action: Action&lt;Task&gt;
) {
    configureEach {
        if (predicate(this)) {
            action.execute(this)
        }
    }
}```
```tasks.lazyMatch({ name.contains("foo") }) {
            enabled = false
        }```

I think you are doing the exact right thing already

Another common way is to set `Implementation-Version` in relevant jar manifest and read it via `getClass().getPackage().getImplementationVersion()`.

the built-in Gradle javaexec doesn’t return a handle, as it is blocking.  Depending on what control you have over the external process, you could:
a) Launch the javaexec via a shared build service that wraps this in an Executor to make it async;
b) At the end of the build, ask the external process to stop.  If you can have the external process look for “.stopfile” or somesuch and kill itself.
…if that flow isn’t possible, a variation:
a) as above, but launch the process yourself instead of with JavaExec to retain a handle;
b) at the end of the build, ask the shared service to shutdown external processes (that you now have a handle to)
Other variations, such as using the OS to ‘kill’ a PID, may be possible (assuming you can get the PID).

thanks, running in an executor should work :thumbsup:

No.  It’s a special, magical subproject.  buildSrc only works off the root project.

That's probably good, there are no "turtles all the way down" issues I could introduce

You can use an included build instead. Included builds can include other builds. :slightly_smiling_face:

More precisely: buildSrc is not like a “subproject” - but like a separate Gradle “build” (with a bit of magic, yes :slightly_smiling_face: )

Yes you can (and should) do this with included builds as <@URQ797WRW> suggests. And there are use cases for it. I wouldn’t just neglect the idea of doing it because of possible turtles. :wink:

Here for example, I use a separate ‘build’ called ‘settings’ that defines a _Settings Plugin_ which I then use in the ‘main’ build as well as in the ‘plugins’ build (which effectively is buildSrc as included build)

<https://github.com/jjohannes/gradle-project-setup-howto/tree/main/gradle/settings> (separate build, like buildSrc of buildSrc)
<https://github.com/jjohannes/gradle-project-setup-howto/blob/main/gradle/plugins/settings.gradle.kts> (Used in “buildSrc”)
<https://github.com/jjohannes/gradle-project-setup-howto/blob/main/settings.gradle.kts> (used in “main” build)

And btw. the answer of <@U02S31Y7J2K> was actually wrong.
`buildSrc` *can* have its own `buildSrc`, I just tried it as I meant to remember it is possible.
Whether it makes sense is a different topic. :smile:

Ah thanks for trying. I was wondering about it… I also wasn’t sure anymore. :slightly_smiling_face:

The best option would be to look at perhaps a system property or environment variable to control this, as plugins are not parameterized.

Thanks for the quick answer.
We do this already:
```val path = System.getProperty("user.dir")
val versionsPath = if (path.endsWith("test")) "../someFile" else "someFile"```
But this looks a bit strange.. :see_no_evil: :smile:

could do
```System.getProperty("plugin.config.file")```

You can also use a Gradle property, no need for a system property.

can you get a Gradle property in a Settings plugin?