From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

Shared configuration can be accomplished using a build service.

<@U02S31Y7J2K> how would you reference the shared build service from project `:app` ? There's no task that uses it there. Can you get a reference to the build service without any task (so as to set the values from `fooExtension` in the build service that is later used by `feeTask`)?

Yep.  Call registerIfAbsent to get the instance of the service and then call the service method(s) as needed to pass along extension values.

I see. Curious what the reason is it'd work with BuildService but not project extensions? Do different projects live on different classloarders?

Something like this (simplified to illustrate the concept).
```public abstract class SharedService : BuildService&lt;BuildServiceParameters.None&gt; {
    public companion object {
        internal fun registerIfAbsent(
            buildServiceRegistry: BuildServiceRegistry,
            configureAction: Action&lt;BuildServiceSpec&lt;BuildServiceParameters.None&gt;&gt; = Action {}
        ): Provider&lt;SharedService&gt; {
            return buildServiceRegistry.registerIfAbsent(
                "cloudshift:shared-service",
                SharedService::class,
                configureAction
            )
        }

        fun instance(buildServiceRegistry: BuildServiceRegistry): Provider&lt;SharedService&gt; {
            return buildServiceRegistry.findByType("cloudshift:shared-service")
        }
    }
    
    public fun setProp(name : String, value : String) {
        // do something here
    }
}

public abstract class MyExtension @Inject constructor(private val buildServiceRegistry : BuildServiceRegistry) {
    
    public fun setProp1(value : String) {
        SharedService.instance(buildServiceRegistry).get().setProp("prop1", value)
    }
}```

Thanks for this :pray: My question is more what makes the Build Service "special" that cannot be achieved by accessing projects and extensions directly. Feels like both Project and BuildService are regular JVM classes ultimately so if I can access fields in one, I should be able to access fields in the other?

My current mental model is that BuildService is more "configuration cache" friendly in the sense that it will be recreated even if the build is loaded from CC.
But in this specific case, sounds like `setProp1()` will not be called?

Unless there is a way to mark some of the BuildService properties as "inputs" and they will be serialized and reloaded too? (but sounds like this should happen in the constructor :thinking_face: )

Thank you all for the input. I’ve stumbled upon this issue from <@U023HSZL88M> :
<https://github.com/gradle/gradle/issues/14697>
Which basically summarizes my problem.

Sadly, either of the two solutions are not working really well for me due to classloader issues:
(i.e. this problem <https://github.com/gradle/gradle/issues/17559>)

1. For Build Service: I would have to use `Provider&lt;Object&gt;` on the task as trying to use a Build Service with my given type fails with: `Cannot set the value of task ':a' property 'service' of type MyService using a provider of type MyService.` which is cryptic.
2. For Root Project Extension: I can’t cast the rootProject extension to my type as it fails with `Extension of type 'MyExtension' does not exist. Currently registered extension types: [..., MyExtension, ...]` . It seems the runtime type of the extension is `MyExtension_Decorated` which is incompatible with `MyExtension` . Also cryptic.


I’m trying a different approach using a task on the rootProject

&gt; I can’t cast the rootProject extension to my type
I think that _should_ work. FWIW, I'm trying similar things <https://github.com/apollographql/apollo-kotlin/commit/3d2f37f5220aef831e07e65a2a741e75f74d48ab#diff-5357acab7bfc41dbb75b1621bba8cc1f880ac3d5bd9d883b3e8fbb24bc9f5b70R435|here>. I'm not super proud of it but it seems to work so far. Maybe the issue is an ordering one? I.e. you have to wait until the plugin is apply with `pluginManager.withPlugin("com.example.plugin.id")` ?

`MyExtension_Decorated` is generated by Gradle but should definitely extend from `MyExtension`

Also good find of those issues!
```the services seem to be cached by classloader, which is by set of applied plugins```
That sounds a bit terrifying :sweat_smile:

From my experience, the best solution right now, if you want configuration that is shared by *multiple subprojects* is to *register an extension on the root project* and access that where needed through *rootProject.extensions* in the subprojects. As long as you do not also want to share that in multiple builds (included/composite) you won't have any classloading issues *if* you register the extension on the *root* project.

BuildServices are only needed imo if you need some "real" shared work done. Although it's not clear to me, where to draw the line between *RootProject Extension* and *Build Service*. In my experience, the extension is often enough and moving things to a BuildService only makes everything more complex without any gain.

&gt; access that where needed through *rootProject.extensions* in the subprojects
I guess this is not 'project isolation' compatible, right? I'm pretty sure I've seen errors accessing `project.rootProject`
&gt; you won't have any classloading issues *if* you register the extension on the *root* project
Out of curiosity, is this not true for other projects than the root one?  Centralizing everything in the root project feels a bit weird when the information might already be in the subprojects extensions.

&gt; I guess this is not 'project isolation' compatible, right? I'm pretty sure I've seen errors accessing
:thinking_face:  With `configuration-cache` this should be fine.
With this experimental "project isolation flag" I am not sure. Might be that it is not, but I wonder what else one should do then? Build Service for everything? I am not sure what the state of this is right now. Would be interesting to get some insights from someone working on this.

&gt;  Out of curiosity, is this not true for other projects than the root one? Centralizing everything in the root project feels a bit weird when the information might already be in the subprojects extensions.
This is because classloaders can differ if some plugins are *only* on a subproject's buildscript classpath. It's always this complex plugin classloader setup....

If you make sure the classes of the extension are *loaded* in the root project (are on the buildscript classpath of the root project), it  does not matter where you *register* the extension from the classloading perspective.

Although, conceptually, *if* you absolutely have to reach into another project for this, I would use the root project for it. Instead of connecting arbitrary subproject again. (In any case I would do this as little as possible.)

My issue (which is different but somewhat related to Nico's one here) is about adding dependencies to configurations in different projects so tunneling all of that through the root project feels a bit weird but thanks for the insights, I'll investigate all of this!

&gt;  From my experience, the best solution right now, if you want configuration that is shared by *multiple subprojects* is to *register an extension on the root project* and access that where needed through *rootProject.extensions* in the subprojects. As long as you do not also want to share that in multiple builds (included/composite) you won’t have any classloading issues *if* you register the extension on the *root* project.
Agree. I can’t make it work though.

&gt;  I.e. you have to wait until the plugin is apply with `pluginManager.withPlugin("com.example.plugin.id")` ?
<@UMCNY0BFH> I’m “already inside” that plugin. Why should I wait for a `withPlugin` ?

Different projects apply at different times

If `:app` needs to access `:library` , `MyPlugin::apply` will be called twice (once in `:app` and once in `:library` )

&gt;  Agree. I can’t make it work though.
<@U015SNK4ZTJ> What concrete problem do you see when you use the "root project approach"? Can you share some setup code?

For example, in the Gradle build, you can find such an extension defined here:
<https://github.com/gradle/gradle/blob/master/build-logic/root-build/src/main/kotlin/gradlebuild.build-environment.gradle.kts>

And then accessed in subproejcts through this:
<https://github.com/gradle/gradle/blob/master/build-logic/basics/src/main/kotlin/gradlebuild/basics/BuildEnvironment.kt#L39>

&gt;  Can you share some setup code?
I’m trying to create a reproducer, which sadly I cannot. So there is probably something going on with my setup.

The problem seems to be some classloader issue but I still I’m not able to pinpoint

So after bisecting all my build line by line, it turns out someone added a:
```buildscript {
    dependencies {

    }
}```
inside one of the `lib/build.gradle` which was also using the `plugins{}` block.

This was causing all the weird failures related to classloading like:
```An exception occurred applying plugin request [id: 'com.facebook.react']
&gt; Failed to apply plugin 'com.myplugin'.
   &gt; class com.myplugin.MyExtension_Decorated cannot be cast to class com.myplugin.MyExtension ```


So the rootProject extension seems like the way to go then :slightly_smiling_face: Thanks for the support <@U023HSZL88M> <@UMCNY0BFH> <@U02S31Y7J2K>

Late night thoughts but I'm wondering if Gradle could declare bankruptcy on the whole "separate classloaders per projec" idea.

At this point, sounds like every project is putting all plugins on the root project with `apply: false` and every plugin is shadowing all its dependencies

I don't know anyone that expects different versions of different plugins in different projects to cohabit peacefully

Anyways, thanks for the deep dive <@U015SNK4ZTJ>!

&gt;  declare bankruptcy on the whole “separate classloaders per projec” idea.
You have my axe :+1:

The plugins were declared like this
```gradlePlugin {
    plugins {
        register("my-convention") {
            id = "com.example.my-convention"
            implementationClass = "com.example.MyConventionPlugin"
        }
    }
}```


this is expected.  you can only use the short-hand plugin syntax with plugins that are already on the classpath, which is what buildSrc does; included builds work differently - they aren’t on the classpath until you put it there, via a plugin declaration.

Is it possible to place them on the classpath?

they are on the classpath once the plugin is pulled in.  chicken and egg.  This is one of the improvements of included builds, cleaning up the classpath lifecycle to be consistent (buildSrc was special, which introduced other challenges).

<https://github.com/gradle/gradle/issues/16929>

the workaround listed there puts the included build in the classpath before all projects, which is consistent with the answer above

Of course the work-around reintroduces some of the drawbacks of buildSrc, like that all projects are going to be out of date if anything changes in the included build.

Is that true for any settings plugin in your includeBuild?

If you have a settings plugin, it is present in a class loader that is in the parent hierarchy of the project class loader, so if the jar changes, the classpath of tasks changes and so the tasks probably are going to be out of date the same as if it were a change in `buildSrc`. (The `buildSrc` class loader is the child of the settings class loader)

Thanks for the additional details <@URQ797WRW>

My recommendation would be to go with:
```plugins {
    id("com.example.my-convention")
}```
and not introduce a workaround which removes the pros of using an included build.

sample:
```pluginManagement {
    includeBuild("build-logic")
}```
…generally the included builds use a project path syntax (‘:’ separators), not directory paths.

What do you mean with your last sentence <@U02S31Y7J2K>?
`includeBuild` is correct with `/`
`include` would be with `:`

Manual dependency substitutions should normally not be necessary unless you set up your project wrongly or do special things like manipulating Maven coordinates.
But to get a better advice or help, you probably need to provide an <https://stackoverflow.com/help/mcve|MCVE >

perhaps it’s different for `includeBuild` - for those I’ve only ever done root-level `build-logic` includes, had assumed it had the same semantics as `include`.

Nope, not at all. :slightly_smiling_face:

Well, not really.
It's a totally different use-case.
With `include` you include a logical subproject (or multiple) and for that the project path is derived by naming convention.
With `includeBuild` you include a whole build which location you specify, specifying its path, which can for example also be `includeBuild("../../my-cool-sibling-build-I-want-to-test-without-maven-local")` (which also is the original use-case for composite builds.

Always a pleasure :slightly_smiling_face:

Btw. actually there is <https://github.com/gradle/gradle/issues/14561> which might add some consistency. :slightly_smiling_face:

So when I have to do

&gt; ```substitute(module("com.blundell.plugin:cool-plugin"))
&gt;                 .using(project(":plugin"))```
I’m telling it the specific project (:plugin) inside my composed build to use for the plugin.
But if I don’t tell it that, what was it using? :thinking_face:

I’ve clearly just miss typed something somewhere, just a pain to go through all the places things are named and figure out which one it is :sweat_smile:

What does `./gradlew -p my/nested/cool-plugin plugin:outgoingVariants` say?

```&gt; Task :plugin:outgoingVariants
--------------------------------------------------
Variant apiElements
--------------------------------------------------
API elements for main.

Capabilities
    - com.blundell.coolplugin:plugin:1.0.0 (default capability)
Attributes
    - org.gradle.category                = library
    - org.gradle.dependency.bundling     = external
    - org.gradle.jvm.environment         = standard-jvm
    - org.gradle.jvm.version             = 11
    - org.gradle.libraryelements         = jar
    - org.gradle.usage                   = java-api
    - org.jetbrains.kotlin.platform.type = jvm
Artifacts
    - ../build/projects/plugin/libs/plugin-1.0.0.jar (artifactType = jar)```

is it because in this report it says `com.blundell.coolplugin:plugin:1.0.0`

and in the substitution I have / gradle is looking for: `substitute(module("com.blundell.plugin:cool-plugin"))`

need to remember this command :smile: thanks!

doesn’t look like annotations are checked here - these are assumed to be inputs.  What I’ve done is create convenience register() funcs on a companion object (with default values as appropriate).

is the offending file (trace.log) for-sure closed, in all possible scenarios?

Yes, it is closed close to the end of the task’s execution. 

Code snippet or reproducer project available?

<https://github.com/joanhermay/ioexception-in-gradlew-clean-task-due-to-jrelease-folder-cannot-be-deleted|https://github.com/joanhermay/ioexception-in-gradlew-clean-task-due-to-jrelease-folder-cannot-be-deleted>

Invoking `gradlew build` followed by `gradlew clean` should trigger the problem

<https://github.com/jreleaser/jreleaser/actions/runs/3584910353/jobs/6032201521|https://github.com/jreleaser/jreleaser/actions/runs/3584910353/jobs/6032201521> shows the error occurring even with the potential fix in place

With the fix being this one <https://github.com/jreleaser/jreleaser/commit/ced4f06b34abc498e7a98029a94756f79bcf3b96|https://github.com/jreleaser/jreleaser/commit/ced4f06b34abc498e7a98029a94756f79bcf3b96> 

k thx.  pulling that code down now to look closer at it…

Thanks. I may be doing something wrong when closing that file and I can’t see it

Looks like trace.log is opened @ `org.jreleaser.maven.plugin.AbstractJReleaserMojo#createTracer`, correct?

```protected JReleaserLogger getLogger() throws MojoExecutionException {
    return new JReleaserLoggerAdapter(createTracer(), getLog());
}```
…every time `getLogger()` is called it will open a new file.  Is this bounded to be only called once?

The file was opened earlier <https://github.com/jreleaser/jreleaser/blob/038db63be6e9f6c8ce60e30989cae5b90cfabee5/plugins/jreleaser-gradle-plugin/src/main/groovy/org/jreleaser/gradle/plugin/internal/JReleaserProjectConfigurer.groovy#L347|https://github.com/jreleaser/jreleaser/blob/038db63be6e9f6c8ce60e30989cae5b90cfabee5/plugins/jreleaser-gradle-plugin/src/main/groovy/org/jreleaser/gradle/plugin/internal/JReleaserProjectConfigurer.groovy#L347>

But you’re right, every call to `getLogger()` by a MOJO will create a new log. That’s a bug as I think there are a couple of places where’s called more than once

from what I can see there’s significant potential to open that file multiple times, leaking the previous ones (as they are not retained, they aren’t closed).

But only by MOJOs. The Gradle plugin should open the log once

perhaps add an output every time the file is opened for a debug build and run that through the reproducer.

in any event, it’s a fragile construct - if it’s intended to have a single log, maintain that as a singleton to avoid any question.

Right. As the close would be called once even of multiple files were opened

yep.  the other ones have leaked - they’d perhaps be closed by a finalizer if a GC runs, but that’s even more unpredictable and unreliable (and causes lots of other problems).

I wonder if the plugin’s afterEvaluate is being called more than once, that would explain multiple fds open

afterEvaluate’s are called once, in the order they are registered.  Assuming it’s registered only once.

in the event that no jReleaser tasks are executed (e.g. `clean`) the logger looks to be created by the plugin, but not closed.

this would be better implemented as a Gradle build service, which have a  clean lifecycle (implement AutoCloseable to clean things up).

I mean, the fact that the logger is created even if no plugin tasks are invoked

likely.  outside of build services there isn’t a clear lifecycle to open / close resources such as this.

either scope it to a build service, or scope it to a task (open/close within the task).

I think a BuildService might be the way to go as the logger is needed to validate a shared model across all tasks, thus it can’t be task scoped at the moment

And the validation is performed before task registration

Didn't read the whole conversation, but if you are still on the hunt for the file leak, <https://file-leak-detector.kohsuke.org/> might help in finding it

thx.  this one is as straightforward as the stack trace indicates - challenged lifecycle for opening / closing file, resulting in it not being closed in some cases.  Moving that logic to a build service provides predictability (and makes it a singleton to close other holes).

Looks like the BuildService did not do the trick after all. Going to run tests with the file-leak-detector. Thanks <@URQ797WRW>

You probably need to use Java 8 to use it.
From a very quick try with your MCVE, and running with Java 11 it didn't detect anything, when running with Java 8, it detected the opened files.

Hmm. I’m running with JDK11. The reported issue mentioned JDK17 …

Build Service should do the trick via AutoCloseable.  Code snippet available?

<@U02S31Y7J2K> here’s the whole patch <https://github.com/jreleaser/jreleaser/commit/82446b52e8e2c7e9de9ad6fafea75c113086119f>

FWIW I do see the log being created and closed even when no JReleaser task is invoked, which as we thought was cause of the problem

are all instantiations of the logger now going through the build service?

odd.  build service looks to fully bracket the opening and closing.  What does a breakpoint / debug msg in `org.jreleaser.logging.AbstractJReleaserLogger#close` show?  Is this getting called?

I put a println there and it’s indeed called, once.

```$ gm clean
Using gradle at '/Users/aalmiray/tmp/app/gradlew' to run buildFile '/Users/aalmiray/tmp/app/build.gradle':
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.kohsuke.file_leak_detector.Listener (file:/Users/aalmiray/Downloads/file-leak-detector-1.13-jar-with-dependencies.jar) to field java.net.SocketImpl.socket
WARNING: Please consider reporting this to the maintainers of org.kohsuke.file_leak_detector.Listener
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
File leak detector installed
Starting a Gradle Daemon, 1 stopped Daemon could not be reused, use --status for details

&gt; Configure project :
&gt;&gt;&gt; CREATE

&gt; Task :clean
&lt;&lt;&lt; CLOSE```

ok.  best guess is that there are other code paths outside of the build service that are opening that file.

That is what I thought as well but no one does it except for the plugin. The core library expects the file to be already opened

Other code, if called, can still open it:  `org.jreleaser.cli.AbstractLoggingCommand#createTracer`

suggest removing all other places such as that one to ensure there is a single lifecycle for that file.

`org.jreleaser.ant.tasks.AbstractJReleaserTask#createTracer`

`org.jreleaser.ant.tasks.JReleaserAutoConfigReleaseTask#createTracer`

that’s the cli &amp; ant interfaces which are unrelated to the Gradle plugin

any luck with the file leak detector, showing where the file is opened?

aaand just failed, And like <@URQ797WRW> mentioned there were no file leaks detected probably due to JDK11

&gt; Hmm. I’m running with JDK11. The reported issue mentioned JDK17 …
The bug at least was reproducible in the MCVE when running on Java 8 also

Yes, same error as at the beginning. Can’t delete dir because of file lock

let’s try this - in the build service, change ‘trace.log’ to ‘trace-build-service.log’.  Looking to understand if it’s the file opened from there, or from somewhere else (which would still be trace.log).

The full GHA run is here <https://github.com/jreleaser/jreleaser/actions/runs/3593454548/jobs/6050445646> Build. Assemble. Full Releas. Clean. Those are gradle invocations for which you can see the file-leak-detector agent reporting no open files at the end

ah.  chicken and egg.  if we are deleting from within the build, the build service won’t yet have cleaned things up - close happens after the build is complete.

which is why the file leak detector shows no leaks - we’ve now got it closed, just not at the right time.

Ugh. Clean complains because by the time it rns the file is still open but the build sevrice makes sure to close it anyway?

something like this:
```tasks.named("clean") {
    doFirst {
        loggerService.shutdown()
    }
}```

well that’s not going to work for everyone, unless the JReleaser plugin implicitly registers such block

OK, this might work. Will give it another try. Thanks much :sweat_smile:

Issue has been resolved. Thank you so much for your help folks!

I followed the <https://docs.gradle.org/current/userguide/build_services.html#implementing_a_build_service|docs>, but can't get what's going on

Nothin fancy, just the basics. This is my plugin:

```class VpnCheckPlugin @Inject constructor(
    private val providerFactory: ProviderFactory,
) : Plugin&lt;Settings&gt; {
    override fun apply(target: Settings) {
        val vpnService = target.gradle.sharedServices.registerIfAbsent(
            "vpn-build-listener",
            BuildOperationListenerService::class.java
        ) {
            parameters.isVpnConnected().set(providerFactory.of(IsVpnConnectedValueSource::class.java) {})
        }

        val buildEventListenerRegistryInternal: BuildEventListenerRegistryInternal = target.serviceOf()
        buildEventListenerRegistryInternal.onOperationCompletion(vpnService)
    }
}```

And this is my service

```abstract class BuildOperationListenerService : BuildService&lt;BuildOperationListenerService.Parameters&gt;,
    BuildOperationListener {

    interface Parameters : BuildServiceParameters {
        fun isVpnConnected(): Property&lt;Boolean&gt;
    }

    ...

}```

should be as below; Gradle will fill in the abstract property, whereas it cannot do so for an arbitrary function.
```abstract class BuildOperationListenerService : BuildService&lt;BuildOperationListenerService.Parameters&gt;,
    BuildOperationListener {

    interface Parameters : BuildServiceParameters {
        val vpnConnected : Property&lt;Boolean&gt;

        // fun isVpnConnected(): Property&lt;Boolean&gt;
    }

    ...

}```

docs are poor in this area - they show a Java example without the Kotlin equivalent of property with implied getter.

Nice catch, Chris!

Just a fun fact: I tried to change from a function to a property, but it didn't work since they had the same name. Maybe something related to cache? Just gave another name to the property and it worked :thinking_face:

You cannot execute a task anyway.
Well, at least not without using a `GradleBuild` task, or the built-in tooling API.

so if I have a task that has calculated the string “help” and want to run `./gradlew help` from my plugin, is it possible?

I can’t do `dependsOn("help")` because “help” is calculated after the time you are allowed to call dependsOn :slightly_smiling_face: (I think)

```        project.exec {
            it.workingDir = project.buildDir
            it.executable = "./gradlew"
            it.args = "help"
        }```
I also saw this in some old old Stackoverflow threads :smile:

(help is just an example of any task fwiw)

As I said, you cannot run it within the same build.
You can for example use a task of type `GradleBuild`, or you can use the built-in tooling API (if your build is a composite build `GradleBuild` would not work anyway), or you can of course also use `project.exec`, but I'd prefer using the tooling API.

But actually it sounds fishy to do this anyway. :smile:

&gt; Gradle provides a programmatic API called the Tooling API, which *you can use for embedding Gradle into your own software*. This API allows you to execute and monitor builds and to query Gradle about the details of a build.

hmm, maybe I’m barking up the wrong tree.

I’ve got a plugin that runs some logic, comes out with a list of gradle test tasks that it wants to run. And then I want to run them

I thought once I had a list of names of tasks, I could just execute said tasks

perhaps my plugin should make a new task for every task I want to run and each of those depends on the one I want to run?

&gt;&gt; which *you can use for embedding Gradle into your own software*
Yes, but you can also use it from a Gradle plugin without problems. There you even do not need an additional dependency as it is built-in when having a Gradle plugin.
&gt; perhaps my plugin should make a new task for every task I want to run and each of those depends on the one I want to run
Would probably not help if that list still is only determined at task execution time as you also cannot register new tasks then.

```GradleConnector.newConnector()
            .forProjectDirectory(project.projectDir)
            .connect()
            .use {
                it.newBuild().forTasks(*myTasks.toTypedArray()).run()
            }```
That seemed a little toooo easy :laughing:

You need to write a settings plugin for this. Note, however, that injecting repositories in a plugin can be seen as a security problem (at least for public plugins it is, for corporate plugins that's probably ok).

Thanks, <@UAFGSU86M>! Do you have a guide for adding this functionality?

simply implement `Plugin&lt;Settings&gt;` instead of `Plugin&lt;Project&gt;` and apply the plugin in your `settings.gradle(.kts)` file

You could compile against artifacts that Nokee deploys at `dev.gradleplugins:gradle-api` but I believe most of us build with (and against) a more recent version and only test against older versions with TestKit (this allows using new APIs without reflection and only guard code paths by comparing the Gradle version)

What you are after and should thumbs-up and follow is <https://github.com/gradle/gradle/issues/1835> which also mentions the inofficial Nokee-published artifacts in a comment: <https://docs.nokee.dev/manual/gradle-plugin-development.html#sec:gradle-dev-redistributed-gradle-api>

It even provides the `gradleApi('version')` accessor

Thanks. I think I’ll try using the direct artifact first

Is nokee responsible for creating `gradle-api` JARs or are they taken directly from what Gradle creates?

They're taken from what Gradle generates.
<https://github.com/gradle-plugins/gradle-api/blob/main/buildSrc/src/main/groovy/dev/gradleplugins/api/GenerateGradleApiJar.groovy>

Thanks. I’ve found out that these JARs “appear” to be MR-JARs but they truly aren’t

Well, wrapping it in a provider and resolving as late as possible is probably already the correct way. Does it not work as intended?

Context: One of convention plugins applies `java-gradle-plugin` and puts plugin ids and groupId+artifactId to a build service. Another generates version catalog with both `plugins` and `libraries` sections using the build service. It's a best way I found to avoid using `subprojects` to collect all plugin projects in the build since test report aggregation plugin approach with using configuration is cumbersome and requires additionally maintaining dependency list in the version catalog project.

Yeah, it works but pattern `plugins.all { ... plugins.named(name) ... }` is what bothers me

Previous version used `subprojects { pluginManager.withPlugin("java-gradle-plugin") { ... } }`

Why does it bother you? :slightly_smiling_face:

Actually, you can probably just do
```plugins.all { fooBuildService.addProvider(provider { id }) }```

Because it's quite easy to use `this.id` instead of `plugins.named(this.name).map { it.id }` and get NPE when working with older properties (with setters/getters) while using `this.otherProperty` directly in context of `NDOC#all`.

Although I still have to use `afterEvaluate` in the version catalog plugin since `VersionCatalogBuilder` doesn't accept providers(

it certainly look better, thank for the idea

yeah, while I use providers to pass the same plugin ids for the task to generate my own catalog-like thing I have to bring `afterEvaluate`((

I was a bit hasty, it seems to work only at first glance. Neither `project.provider { id }` nor `plugins.named(name).map { it.id }` provide implicit dependency between producer (plugin convention plugin) and consumer (version catalog convention plugin).

So by the time catalog's `afterEvaluate` is called not all plugin projects registered themselves. I wonder if the build service is allowed to hold `Project`s in its state and/or do additional configuration in the catalog project after it's build script was evaluated. If it is than catalog project could register a callback to configure itself. Still a place for a potential race though. And it might still be a cross-project configuration even if decoupled via build service.

Another idea is to use cross-project publication but push plugin project paths via the build service for the version catalog project (consumer). Same potential issues as previous idea.

On `ResolvableArtifact` :

`gav = (ModuleComponentIdentifier) resolvableArtifact.getId().getComponentIdentifier()`

That doesn’t always give the coordinates, and they appear to be missing the version and classifier

Some reason, on some artifacts, the `packaging` is given as a pretty long path, rather than an extension, and I’m not sure why that’s happening

`packaging` in Maven artifacts often is the same as the extension, but does not have to be, if that is the "packaging" you talk about

Yeah, that’s the packaging I was talking about.

Turns out, what I needed to do was to take into account the “<https://github.com/gradle/gradle/blob/master/subprojects/dependency-management/src/main/java/org/gradle/internal/component/external/model/maven/RealisedMavenModuleResolveMetadata.java#L215-L220|alternative variants>” that might exist.

It’s been a learning experience :slightly_smiling_face:

Thank you, <@URQ797WRW> and <@U023HSZL88M> for the support. It’s appreciated

Does your test project (the one in the projectDir) build your plugin‽ Why don't you use `withPluginClasspath`?

It’s there – I didn’t post full setup, which is:
```private fun builder(gradleVersion: String, vararg tasks: String) =
    GradleRunner.create()
        .withProjectDir(dir)
        .withGradleVersion(gradleVersion)
        .forwardOutput()
        .withPluginClasspath()
        .withTestKitDir(File(gradleHome))
        .withArguments(
            *tasks,
            "--stacktrace",
            "--configuration-cache",
            *gradleArguments.toTypedArray()
        )```

Single test looks like:
```@Test
fun `skip publishing if token is missing`() {
    // append extra configuration to the existing, pre-configured buildFile
    buildFile.groovy(
        """
        publishPlugin { }
        verifyPlugin {
            ignoreFailures = true
        }
        """.trimIndent()
    )

    buildAndFail(PUBLISH_PLUGIN_TASK_NAME).let {
        assertContains("token property must be specified for plugin publishing", it.output)
    }
}```


And the base `buildFile` looks like:
```buildFile.groovy(
    """
    plugins {
        id 'java'
        id 'org.jetbrains.intellij'
        id 'org.jetbrains.kotlin.jvm' version '$kotlinPluginVersion'
    }
    sourceCompatibility = 11
    targetCompatibility = 11
    repositories {
        mavenCentral()
    }
    intellij {
        version = '$intellijVersion'
        downloadSources = false
        pluginsRepositories {
            maven('$pluginsRepository')
        }
        instrumentCode = false
    }
    buildSearchableOptions {
        enabled = false
    }
    
    // Define tasks with a minimal set of tasks required to build a source set
    sourceSets.all {
        task(it.getTaskName('build', 'SourceSet'), dependsOn: it.output)
    }
    """.trimIndent()
)```

How does the settings script of your test projects look like?

There’s just
```rootProject.name = 'projectName'```


I wonder if that’d make sense to – before running tests – build Gradle plugin with 7.6, publish it to the Maven Local, and then in tests refer to it instead of pointing to the actual sources.

That's one option too, but I'd recommend not (practically never unless you need to cooperate with a Maven build) to use `mavenLocal` but use any other local directory as repository.

But if you run your build with Gradle 7.6 and then use the Gradle runner to run with a specified Gradle version, it should not be rebuilt using that Gradle version.

Can you maybe provide an MCVE?

That shouldn't be necessary (edit: to publish to a Maven repo). The setup you're showing here (assuming the `org.jetbrains.intellij` plugin here) is what one would use to test the plugin with other Gradle versions. The plugin would then be built by the "current" (wrapper?) Gradle version for the project, then its JARs added to the "plugin classpath" of the GradleRunner-controlled build, which then only uses the plugin.

Oh ok – thank you for the clarification. I probably assumed incorrectly the root issue of the issue I faced.

Looks like the issue is caused by using Kotlin `1.7.22` together with Gradle lower than `7.1`:
```java.lang.NoSuchMethodError: 'org.gradle.api.tasks.SourceSetContainer org.gradle.api.plugins.JavaPluginExtension.getSourceSets()'
	at org.jetbrains.kotlin.gradle.plugin.internal.DefaultJavaSourceSetsAccessor.getSourceSets(JavaSourceSetsAccessor.kt:34)
	at org.jetbrains.kotlin.gradle.plugin.AbstractKotlinPlugin$Companion.setUpJavaSourceSets$kotlin_gradle_plugin_common(KotlinPlugin.kt:501)
	at org.jetbrains.kotlin.gradle.plugin.AbstractKotlinPlugin$Companion.setUpJavaSourceSets$kotlin_gradle_plugin_common$default(KotlinPlugin.kt:492)
	at org.jetbrains.kotlin.gradle.plugin.AbstractKotlinPlugin$Companion.configureTarget(KotlinPlugin.kt:487)
	at org.jetbrains.kotlin.gradle.plugin.AbstractKotlinPlugin.apply(KotlinPlugin.kt:382)
	at org.jetbrains.kotlin.gradle.plugin.KotlinPlugin.apply(KotlinPlugin.kt:621)
	at org.jetbrains.kotlin.gradle.plugin.KotlinPlugin.apply(KotlinPlugin.kt:595)
	at org.jetbrains.kotlin.gradle.plugin.KotlinBasePluginWrapper.apply(KotlinPluginWrapper.kt:183)
	at org.jetbrains.kotlin.gradle.plugin.KotlinPluginWrapper.apply(PluginWrappers.kt:27)```

Yep, seems the Kotlin Gradle Plugin version you use is not compatible with &lt;7.1 as that method was added in 7.1.
So as you apply the Kotlin Gradle Plugin in your test project and use a version that is not compatible with the tested version, you get that exception.

Looks like i’m about to limit lower Gradle version to 7.1 – it’s already 1.5yo

&gt; Yep, seems the Kotlin Gradle Plugin version you use is not compatible with &lt;7.1 as that method was added in 7.1.
That’s not right. It’s caused by this issue: <https://github.com/gradle/gradle/issues/22465>

Yeah, I know, but if I enable them, would they work on the newer Gradle versions?

if you set the Kotlin DSL plugin language level to 1.7 you can use context receivers.

But you can't do that, it forces them to 1.4 for Gradle 7 and 1.5 for Gradle 8: <https://github.com/gradle/gradle/blob/ab1a54bfd867af505682e85f3d27998603e936ff/subprojects/kotlin-dsl-plugins/src/main/kotlin/org/gradle/kotlin/dsl/plugins/dsl/KotlinDslCompilerPlugins.kt#L48-L49|https://github.com/gradle/gradle/blob/ab1a54bfd867af505682e85f3d27998603e936ff/sub[…]n/org/gradle/kotlin/dsl/plugins/dsl/KotlinDslCompilerPlugins.kt>

Overriding that via an afterEvaluate that sets them to 1.7.

```
afterEvaluate {
	tasks.withType&lt;KotlinCompile&gt;().configureEach {
        kotlinOptions {
            apiVersion = "1.7"
            languageVersion = "1.7"
        }
    }
}```

Or just use `embedded-kotlin` instead of `kotlin-dsl`

Not directly about context receivers but one of reasons about using context receivers is having lambdas with multiple receivers, I have a few thoughts (I am on vacation so I can’t try on a computer):

An `Action&lt;T&gt;` where `T` can be this `Baz`:

```context(Foo, Bar)
abstract class Baz { … }```
Probably this shouldn’t work as Gradle will not be able to create it until they officially support context receivers.

The other option is using a Kotlin lambda instead of `Action`

Not using `Action` had problems related to consuming the plugin in Groovy, but if the plugin is consumed only on Kotlin, it can have problems related to cache or it can be irrelevant?

configuration cache adds wrinkles.  At a minimum you’ll need sam-conversions as a compile flag.
```// afterEvaluate required as kotlin dsl plugin sets its defaults in afterEvaluate
        // <https://github.com/gradle/gradle/blob/master/subprojects/kotlin-dsl-plugins/src/main/kotlin/org/gradle/kotlin/dsl/plugins/dsl/KotlinDslCompilerPlugins.kt#L43>
        afterEvaluate {
            tasks.withType&lt;KotlinCompile&gt;().configureEach {
                kotlinOptions {
                    @Workaround(
                        issue = "<https://github.com/gradle/gradle/issues/17052>",
                        message = "workaround limitation",
                        targetVersion = "unknown"
                    )
                    freeCompilerArgs += "-Xsam-conversions=class"
                    apiVersion = kotlinJavaLanguageLevel.kotlinVersion.value
                    languageVersion = kotlinJavaLanguageLevel.kotlinVersion.value
                }
            }
        }```


Kafka is doing this: <https://github.com/apache/kafka/blob/trunk/wrapper.gradle>

so… you need _a_ version of Gradle to bootstrap the Gradle wrapper required by the build. That, kinda defeats the point if setting a wrapper for bootstrapping Gradle in the first place. In contrast the Maven wrapper only needs Java to bootstrap Maven.  

Not really, does it?
You still have to get the start script from somewhere.
You can download it from somewhere, type it from memory, or use some task to generate it.
The Kafka build is using the standard wrapper task to generate the wrapper files and then modifies them to add the download ability for the wrapper jar.
So when you already have the wrapper start script you can use it to generate a new one from a newer Gradle version.
If you do not have it, you somehow have to bootstrap it initially. Whether you use the wrapper of any other project for that, or just download the start script and copy it in place does not make much difference, does it?

Depending on the reason, you could also base64-encode the wrapper jar and add it to the start script so you only have text checked in, but then can generate the binary jar from the text content

The problem is checking the JAR (a binary) into SCM. Maven’s wrapper can bootstrap itself with just a Java source file, which in turn downloads the wrapper JAR and then runs the build. So yes, the script bootstrap must happen. It’s OK to check it into SCM, just like the Maven source file.
The link to the Kafka solution expects _a_ version of Gradle to exist before hand so that the correct version of the wrapper (script and JAR) may be bootstrapped. That is the crux of the problem and the reason why the wrapper was created in the 1st place: provision _a_ given version of Gradle. Even in this time and age now that Gradle is more common it’s not attainable to expect _a_ version of Gradle to be preinstalled on a pristine environment.

checking in a build-supporting, small JAR into SCM isn’t problematic.

For some people it is problematic due to policies or whatever

But the Kafka solution does not need any Gradle version

It is as self-contained as the Maven wrapper solution is

It downloads the wrapper jar and then runs it just like the Maven wrapper solution

Just copy their `gradlew` script into your project and if you usually use the `wrapper` task to update the wrapper, then also their Gradle code so that the modified wrapper stays in place even then

<@URQ797WRW> I see. This is where the secret sauce is <https://github.com/apache/kafka/blob/b2dea17041157ceee741041d23783ff993b88ef1/gradlew#L118-L128>

the previous link showed a `wrapper.gradle` script which requires Gradle before hand, hence why it threw me off

and sadly it only works for unix. They specifically avoid working with the Windows version of the wrapper script. :sweat:

Exactly, the `wrapper.gradle` is just what generates or regenerates that wrapper script as said before.

And yes, they only did it with the Bash script, but you could extend it also to the Batch script, the principle would be the same.

But no, they do not restrict it to Unix, they restrict it to Bash.
Nowadays probably any developer has Git, so he also has Bash, even on Windows.
And even if not, you could still use Cygwin or WSL. :slightly_smiling_face:

True, but can’t be 100% sure that’s the case. There’s always that person running Windows in a weird fashion

There's always that person running anything in a weird fashion. :smile: