From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

the api for dependency graph is `Gradle.taskGraph.requestedTasks`

im looking for either a list of tasks requested or just the raw string command I could parse

`gradle.startParameter.taskNames` should tell what tasks are requested

you could look at `org.gradle.execution.DefaultTaskSelector` to see how gradle resolves which tasks to call even when you ran something like `gradle :sP:as` instead of `gradle :sub-project:assemble`.

The order in which the plugins are applied on the module remain the same. I apply the android plugin before I apply my custom plugin.

Generally you shouldn't depend on plugin application order (though it should be stable currently).
If your plugin requires `com.android.library` just add `project.pluginManager.apply("com.android.library")`.
If you want to do some configuration if that plugin applied to the project use `project.pluginManager.withPlugin("com.android.library") { .. }`, action will be called after android plugin is applied and extension would be available.

Thank you. That makes sense and your proposed solution works out for me!  I wonder why the behaviour is different for plugins that are applied from different sources though.

the normal included builds doesn't add the plugin artifact to the classpath (any dependency on the build.gradle of the included build), buildSrc adds it, if you add agp to the buildscript or add it in the root project it should work 

:point_up: because of this, I would say it was more ‘accidental’ that it worked with buildSrc. Because buildSrc historically has some “quirks”.

Alternatively to ‘withPlugin’, you can also apply the plugin in your convention plugin first. Then you have the guarantee that it was applied before. You can write `plugins { id("com.android.library") }` as many times as you like. Gradle only applies it once. So the first place it is applied controls the order.

Thanks , this explains the behaviour.
IIRC the suggested mechanism for project specific plugins is to use composite builds instead of buildSrc right?  I remember reading that somewhere but cannot find that reference right now.

Both are ok, but my personal preference is never too use `buildSrc` but always included builds, except for special cases like monkey-patching 3rd-party classes

It probably depends which version of the `com.gradle.plugin-publish` plugin you're using, as things changed in 1.0.0, and how you configured the POM/publication for publication to Central.

(ok, in this case, how Jetbrains configured them for Dokka)

Yep, it's using `0.20.0` : <https://github.com/Kotlin/dokka/blob/e1e114b2c8a025e24b605a0753d7ae0301196315/runners/gradle-plugin/build.gradle.kts#L6>

But given that the `plugin-publish` is closed source, it's quite hard to investigate

IIUC, prior to version 1.0.0 you couldn't customize the POM published to the plugin portal.

you can do it, but it requires Shenanigans

Looks like I needed to set the excludes on dependencies themselves, not configurations: <https://github.com/Kotlin/dokka/pull/2570>

In my org, its easily 90+%. We only use a few oss plugins directly, but many under the hood of our custom plugins. In most of our projects, the scripts are just declaring plugins, dependencies, and maybe configuring a couple extensions. We have about 15 custom plugins (not all applied directly, we have convention plugins). Some totally original, others applying oss plugins with our standard config. I'm just wondering how typical this is

Follow up question: if your org has very few custom plugins, but are interested in going deeper on them, what questions or issues do you have? I am giving a conference talk about developing gradle plugins in sept, to talk about what I have learned in my work, but would love to tune the content to be as helpful and relevant to people just getting started as possible

We have around 50 common plugins (most are published in private nexus repo but some are on the plugin portal to simplify bootstrapping). Mostly they are convention plugins and/or wrappers around open source plugins (~35 of total count). Other group is plugins to develop plugins and simplify tasks like repo, plugin, platforms/boms and version catalogs management.

It allows typical project to have something like this in `settings.gradle.kts`:
```plugins { id("ws.gross.private-repo") version "0.15.0" }
privateRepo {
  manifests {
    create("build-logic") { from("com.example.gradle:manifest:x.y.z") }
  }
}```
which configures private nexus repository for plugins, dependency management and publishing, fetches manifest containing list of plugins and version catalogs and applies it allowing `subproject/build.gradle.kts` to look like:
```plugins {
  id("com.example.java-library")
  id("com.example.lombok") 
}
dependencies {
  implementation(commonLibs.guava)
  testImplementation(testingLibs.mockito)
}```
Last category of  plugins we use is adhoc in a project separated into the in-project included build, for example for codegen exposing simple extension to configure it.

Fortunately it's all green field since we lived with Apache Maven (and Ant in some legacy apps) for years. And we are very small org.

So we try to have only project specific code in the `build.gradle.kts` (at most plugins, dependencies, test suites dependencies and config via extensions). Not entirely clean, of course, things like adding `--add-opens` to `quarkusDev` task are not worth extracting them to convention plugin usually. Almost all projects  are that way except abandoned/archived ones but 80-85% projects are still on maven.

As for current issues when writing plugins:
• lack of some convenient methods/public APIs (DetachedResolver, JvmModellingServices, JvmEcosystemUtilities)
• huge pain if you write plugin in Kotlin
• some pain if you want target several Gradle versions and want use new features if available (heil reflection/method handles)
• testing infra (ProjectBuilder is very limited, TestKit is quite slow and doesn't support included builds AFAIK)
• lack of officially supported Gradle API jar/sourcesJar
Just off the top of my head, maybe there were more issues but they didn't leave such an impression on me

&gt; I'm just wondering how typical this is 
&gt; 
I don't know how typical it is, but optimally all bigger builds should look like that. At least that is what is called an "idiomatic" build. :-)
Our main build is far from that as it grew from pre-1.0 in an organic way, but I'm trying to clean up that mess. :-)

&gt; some pain if you want target several Gradle versions and want use new features if available (heil reflection/method handles)
&gt; 
 Alternatively to reflection you could for Gradle &lt;7 also check the version and have the version specific code in separate classes. And just in case you didn't know, since Gradle 7 you can asleep have different variants of the plugin that are selected automatically according to the consuming Gradle version. :-)

As an IT services and consulting company, our code eventually belongs to our customers, so we don't share plugins across projects. Each Gradle project generally has a couple to a handful convention plugins (generally, a `base` plugin setting up Ktlint through Spotless for the `*.gradle.kts`, a `java-base` one configuring a toolchain, JavaCompile options, Error Prone, Forbidden APIs, and Google Java Format through Spotless, and a `dependency-management` plugin for `logging-capabilities` and the like; and that's it). In terms of third-party plugins we almost always use: Spotless, ErrorProne and Nullaway, FlywayDB, logging-capabilities, Forbidden APIs, and CycloneDX, and sometimes Nebula packaging when we need Deb or RPM packages (we're slowly moving to separate debhelper and rpm steps though).
We do mostly zero-framework _magnificent monoliths_ in Java with React or Vue SPAs and, for instance, we never call Node or Docker from Gradle; orchestration is done at the Jenkinsfile level (Node and Gradle in parallel for client and server, then Docker to package it all ; the Gradle build only prepares directories for the subsequent Docker build)

Wow thanks everyone for the insight. I think our environment is closest to <@UJUMPU407> although we use a custom wrapper instead of settings plugin to configure plugin repo/deps. What are the pros/cons there? Do you need to publish that plugin publicly?  For supporting gradle versions, I have had success supporting 2 major versions at a time, giving people time to upgrade to the second version before dropping support for the old one and adding support for the next one. But that does mean we aren't able to leverage the latest and greatest features in our plugins

Interesting point about kotlin. I have found for simple build scripts its fine, complex build scripts, not worth it, but plugin Dev is where it shines IMO. It makes it easier to navigate the lower level gradle APIs with strong typing and code hints.

Imho Kotlin is always worth it for any build script. And it is very much worth it for custom tasks and plugins, but only if the Game versions to be supported is a fixed small defined list. For a publicly published plugin that should be compatible with many Gradle versions, I would always prefer using Java and there the lowest version supported to run Gradle in the lowest version that should be supported. With Groovy or Kotlin there is too much shenanigans with compatibility and runtime versions and so on in that context.

For a context I have several kinds of gradle projects: our internal projects (which need to use private repo), some open source projects (whose repos/plugins I don't want to affect) and some customers' projects with their own repo setup (either repos defined in the projects or via separate init script).

Main pros of a settings plugin is using vanilla gradle infra (wrapper, distribution, wrapper-validation-action) while keeping enough flexibility. I tried to make plugin more or less generic to it be useful not only for our org but to the others as well (it still needs a lot of work). Another approach that we used was custom distribution with init script embedded but it has some pros as well: you need infra to serve these distributions (preferably without auth), have to use custom wrapper task or full distribution url all the time.

Yeah the hosting of the custom wrapper was a problem I had to solve. I ended up putting it in our private artifactory, and using the gradle properties to set auth

I then had to add to our Jenkins dsl implementation to pass that prop, and provide setup docs to show devs how to set their api key to those props in their user home gradle.properties

&gt; With Groovy or Kotlin there is too much shenanigans with compatibility and runtime versions and so on in that context.
Is this true of Kotlin even with the classloader isolation provided by the `plugins` DSL block?

Not sure what classloader isolation you are talking about, but I think the Gradle provided runtime should always be in a higher class loader, even if the plugins would be isolated from each other which would be news to me and would be problematic even because they could then hardly with hand in hand. Besides that plugins are also used as normal dependencies in plugin projects for convention plugins and so on.

I'm referring to <https://docs.gradle.org/current/userguide/plugins.html#plugins_dsl_limitations|this section> of the userguide, where it says the plugins DSL allows "different plugins to use different versions of dependencies"

It says it allows, it does not say it does so. :smile:
I'm not sure why this is written there and whether it has any relevance.
I did a quick try and I don't see any isolation.
I created two separate included builds that use different versions of a dependency and just print their version that is retrievable through API.
If I only apply the plugin with the older version, the older version is printed.
If I apply both plugins, both print the newer version.
So I don't see any isolation there.

And actually even if it were isolated in this specific case,
as I said, there are many more cases where it would not help,
like when using legacy apply method,
or when using convention plugin project with both plugins,
and as I said even if there would be isolation, the runtime would probably still come from a higher class loader.

I'd recommend to post an issue about the doc statement, so that it gets either removed or clarified.

<https://github.com/gradle/gradle/issues/21293>

I have seriously mixed feelings about internal plugins.  One the 'one hand' they are a very compelling abstraction with the potential of solve vast amounts of build pain, especially in complex projects that need to work togher -- the ability to 'fix' the dependancy versions, core configuration values, versioning schemes, publishing -- all the Good Stuff

On the other hand - my success both in individual attempts (developing them) and in larger context (inheriting plugins from prior projects/teams) -- have been a very rocky road with a benifit/problem ration below 1.0 sometimes near 0
Why?
Modifying shared plugins is *painful* -- especially plugins related to version control and dependancies.   Typically I have to push a plugin update through all layers of the CI system (to all dependent builds) and build everything in order to tell if it worked and how bad the damage is.  Example, bumping the kotlin version and dependent libs , has to make it through all dependent builds upfront - you cant just rebuild the part your working on, everything else has to built first and then published and then new versions referenced etc.  This can turn a '1 hour project' into a week or more

Debugging plugins is a real pain.   Debugging gradle itself is painful, but debugging plugins even more so -- everything lifecyle and defered/lazy dependency issue, ordering issues etc multiple going from debugging build.gradle to debugging a plugin.
All the docs say its awsome and easy, and maybe for everyone else it is, but not me or anyone I work with. Pretty much the definiton of hell.

The compelling beauty of turning a 1000 line gradle script into a 10 line one -- is incredible.
But at the cost of visibility -- when 'stuff doesn't work right' -- figuring out why and where is vastly more painful.
In a real-life example, I inherited an android build system that made use of a single internal plugin that did 'all the rights stuff' -- or so the authors belived.  Established naming conventions, publication tweaks, enforced use of release builds vs snapshots at the right place, integrated with the Jenkins file for CI/CD,  even created build tags - awsome.
Except it was horribly flawed in a few places -- one was a mistaken understanding of how git tags worked which would mysteriously result in builds pulled from the wrong branch and published with no evidence that the code was from the wrong branch,  created tages every CI build that make finding real commits hard, enforced a naming convention that didnt follow a pattern anyone could figure out,  and finally
was written at a fixed point in spacetime --  which meant every time we tried to upgrade gradle or kotlin  or android extensions -- the build would break and we couldn't figure it out -- it would be some use of internal android or gradle classes in the build script hat had become deprecated or changed -- it was *extremely obscure and impossible to maintain
In fact it took me several *months* finally LOCATE the damn thing then almost a year to decipher it enough to extract it out of the builds without breaking them for weeks on end --
The Jeweled Garden that was created turned into self-made Cage of Thorns that in the end caused 10 to 100x more damage then it created benifit.   I would not want to put anyone else through what I had to do ..

My takeaway -- if you are going to do internal plugins ,  you need them built and actively maintained by people who are gradle experts, and who *must be considered a critical part of the team for the entire project lifecycle.*
IMHO this is not an area to express ones inner creativity.     Its an area you need to be very conservative, using all of the core software methodology seriously, and do not even think of doing unless *the organization* is committed to supporting its development and maintenance as much as it does the product.   The skills needed to debug a plugin are skills most developers will never acquire and should not be expected to, its a very specialized skill that takes a lot of work and time and requires a personality that enjoys extremely difficult challenges under pressure in a very closed  technology stack.

Yeah setting dependency versions seems not great. I think keeping that in build script land is the right call. One exception I have found: we have an  internal framework that publishes a BOM. We use a plugin to determine the latest "stable" and non-"banned" version of that framework, and apply that version of the BOM as an enforced platform. That way, people get the latest automatically, but without dynamic versions, and if we ban a version (due to a bug being found) everyone will fallback to previous version automatically until we get a fix out.

Debugging is hard, I agree. I try to counteract this with TONs of unit and integration tests in my plugins

My plugins are mostly taking things copy and pasted between our 200+ services and libraries, and encapsulating them to simplify those build scripts

But yeah you still need that weirdo in the corner muttering to themselves, aka the "gradle understander" to maintain them.

But if it frees up 300 Jr engineers across the org from having to learn gradle just to accomplish simple things, its worth it imo

Like with all software, finding the right place to draw the abstraction line is key

Thanks for the issue <@U03JYHHAC5B>. This sentence was added years back and was wishful thinking at the time. Here's a PR to remove it <https://github.com/gradle/gradle/pull/21448>

<@UNZDFG9AQ> you can use composite builds to include the build of a plugin into an another build and test it live while developing without publishing

<@UABCKNV3K> Nice in theory.  In practice -- not really.   It has gotten better over the last few versions, but even today (gradle 7+) in the projects I maintain, included builds are problematic in a chained dependency, and often even in a simple one.  When they work perfectly, its great, but then when they dont -- not so. TO the point that they do not work as a validation that a plugin wont break the builds.   Both false positive and negatives.
Plus all the caching layers --  A-&gt;include B,C,D    B-&gt;include D, F ,  D -L&gt; include C,G,E etc ..
to get those all to 'swap over to included builds'  -- all at the same time and without caches pulling in the wrong stuff without noticing
(and dont even start on the IDE) -- success chances ? near 0.
the range of things where included builds "dont quite work the same" is large -- very large.

This is a sort of self-defining problem.   Strategies like included builds tend to work better on smaller and simpler projects
exactly the same projects where its not to hard to debug them and probably don't need a bunch of plugins to manage.
But on larger project where you really could use a plugin to help manage things is where the includedBuilds tend to get messy.
And the whole idea of not needing to be a gradle expert falls apart -- you have to know quite a lot to construct a build that will always work perfectly as an included build in a large project.  You have to get unique project names, not do any 'tricks' in the publishing, always use the default configrations etc - and the  list of 'WTF' not documented is large but hidden.

All that said -- I *do use* org wide self-made plugins for exactly all the 'Good Reasons'  -- and have -- after a long long time (years) gotten them stable and have memorized the happy path to update them -- if I dont touch them and dont stray off the happy path -- they do wonders.
They do achieve the desired goals.
I set the kotlin version,  hard-fix the versions of key dependencies to force all projects to use the same version of jackson, kotlinx etc,
and the most difficult one,  I recently added integration to AWS CodeCommit including a build plugin (3rd party) that launches an internal local maven repo and maven authenticator inside gradle to enable the bootstrapping of pulling the initial core plugin itself from code commit
without having to put authentication in any project -- and that is the plugin that implements the codecommit integration.
It is *really cool* -- but I would never do it again, and would not recommend it.
What I thought would  be an afternoon of fun coding turned into 2 weeks+  of hell -- and this is a *small project base* - &lt; 10 git repos with only 5 active ones -- so why so hard ?  Getting all the right repos for resolving and publishing across main, feature/snapshot and release branches,
bootstrapping the initial plugin into the CI/CD tool,  managing different sources of auth secrets depending on if its a local/dev vs CI build,
managing caching of SNAPSHOT of the build plugin itself, the difficulty of debugging a remote CI build running inside the build server where its got an entirely different environement, kicking off huge chains of builds and propagating all the built artifacts into the next chain to discover a bug 4 levels down but not knowing if they pulled different cached versions or was it a bug.
And this is for a 1-person team - me.  I can tolerate breaking my own builds -- painful but self inflected.

At my 'day job' -- not so much tolerance for builds breaking,  you cannot test stuff like this in different environments reliably as the core domain of what the build tools tend to do is manage environment specific build configurations -- so you only get so far in a sandbox before you have to commit it to the real thing -- OR -- get IT to allocate you a cloned environment -- good luck, and when/if you get it, how close is it to the real one?  not close enough.    Then when its all working perfectly -- there are the *other developers* who are extremely creative at doing things slightly differently then I did -- and then they start having problems that they don't know where they came from ---
What I did in my 10 project personal build -- would have taken 6 months at my day job -- if they didn't fire me first.
I dont mean to be depressing and 'gradle bashing' -- gradle is at the very top of my list of amazing tools I do not want to live without.
Its also at the top of my list of "Extremely complex and subtle tools you better not touch unless you have the time and patience 100x what you would predict".   Perhaps my experience is not common.  However, I think it is more common then people admit.
Most engineers are embarrassed to admit they cant figure out gradle *at all* -- if it doesnt work out of the box they are totally lost,
or worse , try to fix it by adding more and more code to the build script --&gt; exactly the wrong thing to do.

Note: All the above commentary is from the perspective of a lead *developer* attempting to 'play devops' in teams where there were no other experts to draw from, attempting to improve things 'in my spare time' .  Your Org may be quite different.

Opinion:  If you have good experience in all this and dont share my experience, good ! go for it, ignore the above as a unusual exception to the norm.   If you have NOT attempted to manage a set of active projects in a team with custom (or 3rd party)  common build plugins as described and are considering trying it for the great anticipated reward -- I recommend doing so very carefully and strategically and make a back-out plan to handle the strong possibility that you get far into the whole thing and it just doesn't *quite work*  -- and the end of the tunnel looks ALMOST THERE -- for days and weeks making good  progress but somehow not getting closer to the end.
Many of the most severe pain points are very specific and coupled to the intricacies of integrations (source control, build servers , publication and resolution repositories,  build versioning conventions or lack, distribution of workers by location and skills,  access-or-lack of good connectivity, differences between developers 'desktop environement', company culture, management, tolerance for 'failure' etc.
Its all those 'non gradle' things that really make it go from 'very complicated, but solvable'  to 'never-ending-toruture'

My plugin depends on a lot of plugins too, like Kover, Detekt and so on. If one of those plugins is using `1.7.10`, can it be the problem?

How can I force to avoid that dependency to be propagated so? The idea is that the consumer decides which Kotlin plugin must be used, that is the reason I am using `compileOnly`

I guess I have to exclude kgp for all of those plugin dependencies (or at least find which is use implementation instead of compileOnly to exclude it there)

Gradle classloaders are known to do surprising things. 
But nevertheless, looks like you need to support 2 versions of KGP at the same time? If some symbols changed between 1.6.10 and 1.7.0, your best bet is certainly some reflection

Yeah.. I was trying to avoid that,  so I am migrating everything to 1.7.10

If possible make the created object use the properties/provider api, then wire the fields instead of requiring them to be set already.
I don't think there is an "after configured" hook and `afterEvaluate` of course has the usual drawbacks.

Thanks a lot Vampire. I don't see any way to do it, one of the properties in these objects is the name of a `maven-publish` publication that I want to grab, configure and publish. So I need to resolve this string before task execution, because I need to, for example, write `myTask.dependsOn("publishXXXPublicationToMavenLocal")` somewhere. I guess afterEvaluate is the only way

I am not sure, can be hard to say Gradle where the project is, because each machine can create the test in a different place and so on

but I guess that if this is not easy to get with test kit, I should create a feature request, I think it is a common use case

There was feature request for testkit to support composite builds iirc.
For now you can publish all into a repo local to a project and add it to the project under test

yeah, that is what I was doing but I was trying to avoid it

I will keep doing that until there is a new way to get it working without publishing

If metadata for plugin under test is simple classpath maybe generating it would work? Never tried myself

I have no idea about that tbh, so I don't know how to start to try that hahah

hm, <https://github.com/gradle/gradle/blob/master/subprojects/plugin-development/src/main/java/org/gradle/plugin/devel/tasks/PluginUnderTestMetadata.java|https://github.com/gradle/gradle/blob/master/subprojects/plugin-development/src/main/java/org/gradle/plugin/devel/tasks/PluginUnderTestMetadata.java> tells that your plugin runtime classpath should already be on the testkit classpath

Mb I could have missed a major point. I reread original post and if I understand correctly it's not your plugin that use core but project that use the plugin? Afaik plugin under test metadata defines classpath for the build script

i tried to find that classpath to extract all dependencies, but that classpath doesn't exist, and I tried to find all classpath dependencies, but that dependency wasn't there

Method documentation is missing this part

I'd say it has to, otherwise it cannot know, can it?

value provider could be wrapped in some container like Java `Optional` that may indicate no value provider is available or some is set.

But from the Gradle code I see it is not the case here and code triggers value calculation

I'm not sure what you mean.
If the provider calculates to `null`,  `isPresent()` returns `false`,
so you cannot say whether it is present without calculating.

```providers.provider { null }.isPresent()```
will evaluate to `false`.

I'm also curious about this: compatibility of _what_, exactly?

i know that gradle ships a different version of the stdlib depending on the version, and that the kotlin dsl is written targeting a different kotlin version depending on the gradle version. but i think if you target the same kotlin version that the minimum gradle you want to support ships with, you should generally be OK

Is it like Scala, where the ABI changes from language version to language version?

Gradle provides a runtime in which plugins run.
This runtime contains a hard-coded version of Groovy and Kotlin that comes higher in the class loader hierarchy so is used.
Your plugin and all transitive dependencies you use have to be compatible with the runtime provided by that Gradle version it is currently used in.

re: like scala, no kotlin is not like that

So how is it different from Java, where you're generally fine as long as you target an old enough JDK?

i think its the same, as long as you target an old enough kotlin version.

Kotlin metadata has some forward compatibility limitations

for example, i target 1.4.20 in my plugins, and kotlin dsl 1.4.9 since i target gradle 5+

Except that Kotlin is faster moving, also in terms of deprecation and removal.
Something that works in Java 8 most likely runs fine with Java 18.
With Kotlin this is not the case afair.

They support like 2 versions back and 1 version forward officially or something like that.

This makes a more dynamic development of the language possible, but limits the backwards compatibility.

gradle will throw warnings if you pull in a version of kotlin that is newer than the version the kotlin DSL is targeting. they are annoying, but i have not seen any issues from that yet. i just downgrade tot he kotlin version the DSL wants to make the warnings go away

Then you still have the problem, that Gradle 7.5 suddenly includes Kotlin 1.6.10 instead of 1.5.x and your plugin might become incompatible because of that.
And such updates happen in minor Gradle versions.

The backward compatibility of `kotlin-stdlib` is pretty good. I don't think the n-2 rule applies there? Or at least I haven't heard a case where that was an issue

Well the point is that it *can* break even in a minor Gradle version change.
And you just have less problems and concerns if you write the plugins in Java.

Of course this is just a personal opinion and everyone can do whatever he likes. :-)

i have also seen (like <@UMCNY0BFH>) that kotlin backwards compatibility since 1.x has been very good, and they do a good job with their deprecation lifecycle and compiler warnings. I think it is totally something to be aware of though, and something to keep in mind, especially if you are targeting many major versions of gradle. thanks for the discussion.

Always a pleasure :slightly_smiling_face:

&gt; I don't think the n-2 rule applies there
Yup, the "2 backward, 1 forward" rule is for the binary format: <https://kotlinlang.org/docs/kotlin-evolution.html#evolving-the-binary-format>

The stability and dependability of the Java language and ABI is something you can't get anywhere else

Ah, so that only affects the compiler <@UMCNY0BFH>?

It's about the `@Metadata` annotations that the compiler reads

Hm, but on the linked page it says it can always read all older versions, not only 2 backwards

Now I forgot where the "2 backward" came from :thinking_face:

Anyway, for this discussion it seems to be irrelevant.

But even in the stdlib there are incompatible changes.
I've seen methods that were deprecated, then removed and then even readded with a different signature.
Which still makes me feel like Kotlin is too dynamic for a reliable, as compatible as possible, public plugin.

Stdlib is case by case. I'm scrolling to find binary breaking changes in <https://kotlinlang.org/docs/compatibility-guide-15.html#forbid-spread-operator-in-signature-polymorphic-calls> but there aren't many

A bunch of source breaking changes. A few behaviour changes.

The method I talk about was stdlib afair.
But I cannot say now which it was.

yeah, my use case is internal plugins where we are only committed to supporting 2 major gradle versions at a time, so in that use case, I think the compatibility is not an issue. but a public plugin that is supporting many more gradle versions is a different assessment

And another thing that makes me uneasy are those warnings about different versions of Kotlin libraries in the class path

Yes, definitely <@U02162U7WCX>.
For local or internal plugins with a very narrow scope of supported Gradle version, I do the same.
I always only said for plugins that should be compatible with a wide variety of Gradle versions and also future versions it is imho better to use Java.

Searching the binary incompatible changes since 1.3, I found only one and I think it only applies to JS: <https://kotlinlang.org/docs/compatibility-guide-16.html#hide-implementation-and-interop-specific-functions-from-the-public-api-of-classes-in-kotlin-js|https://kotlinlang.org/docs/compatibility-guide-16.html#hide-implementation-and-interop-specific-fun[…]s-from-the-public-api-of-classes-in-kotlin-js>

Well, seems not all changes were doumented then, another point to be uneasy :-)

Maybe if I remember, as I said, I don't remember which methods it was about right now.

Twe weeks vacation and mind is cleared :slightly_smiling_face:

Is it possible that it was a source change but they kept the symbol in binary form somehow?

Re: warnings, they usually go away by upgrading Gradle to the latest Kotlin embedded version

<https://github.com/gradle/gradle/issues/16781>

kotlin-stdlib changes like the min/notNull change are binary compatible, source-incompatible. changes like kotlin.time.Duration representation are binary-incompatible, but also marked experimental

Ah yeah, what I had in mind was something with minByOrNull or something like that.

there are no issues with binary incompatibility there, and the source transition was gradual over like 4 or 5 releases iirc

overriding `toString`  so it uses `myProp.get().toString()`

I have difficulty predicting what Gradle can serialize into the configuration cache and what it can't

If you want to force that value to be changed, use `ValueSource` API

Is the basic concept of `ValueSource` that the `obtain()` method is called on each build, and the `ValueSource` implementation deals with things like caching? For example, I can shell out to an external process, cache what it returns, and invalidate the cache when an input changes that I know about (but I have to handle all this myself instead of Gradle tracking it)

Whether `project.version` is serializable should be irrelevant for configuration cache.
Iirc the config cache stores the tasks and their configuration / state.
and a task also is not allowed to access the project at execution time.

Regarding `ValueSource`, if you know the external processes delivers the same output for the same input, you can of course add some caching. The `ValueSource` is indeed evaluated on each build and if the value changes, the cache entry is discarded. In that case, the value source is even calculated a second time during normal configuration phase.

&gt; In that case, the value source is even calculated a second time during normal configuration phase.
Interesting; is that deliberate or just a bug?

I guess it's intended, but actually I don't find this in the docs. Maybe just someone mentioned it. So now I'm not sure anymore whether it is the case and whether it is intended.

Ah, stumbled upon it again.
<https://docs.gradle.org/7.5/userguide/configuration_cache.html>
&gt; In both approaches, if the value of the provider is used at configuration time then it will become a build configuration input. The external process will be executed for every build to determine if the configuration cache is up-to-date, so it is recommended to only call fast-running processes at configuration time. If the value changes then the cache is invalidated and the process will be run again during this build as part of the configuration phase.
This is about running processes, but the second of the "both approaches" is using a `ValueSource`.

My personal experience is that it's super hard to test at that level and also there is stuff you are not able to test like build cache, incremental build, different Gradle versions, etc... I either rely on:
• Unit test for individual methods that do not depend on the Gradle API/environment
• Integration tests using `GradleRunner` for everything else

Thanks. Extracting the logic to not depend on Gradle APIs helps, though it'd still be useful to have an ioc container

Well, the Gradle build itself uses it and the main developers of the plugin are part of this Slack too and one of them even is a Gradle Fellow, so I guess the answer is most probably "yes".

Untitled.kt

```kotlin {
    target {
        compilations.getByName("functionalTest") {
            associateWith(compilations.getByName("main"))
```
tried that?

It hadn't occurred to me that this was more a Kotlin issue than a Gradle one. Thank you!

That works perfectly, although the latest Android Studio doesn't recognize the change even after sync and restart

Why is JandexMain executed through `javaexec`? Could you maybe rewrite it to use the <https://docs.gradle.org/current/userguide/custom_tasks.html#worker_api|Worker API>? You could get the same kind of isolation but also classes from the plugin.

Well, I’m not familiar with the Worker API to begin with. Also, in its early days the API had issues thus I stayed clear of it. If it’s more stable now I guess I could switch to it. Do you know any good examples to get started with such API?

I feel like the docs themselves do a rather good job at how to get started (with the addition of javadoc and/or IDE autocompletion). Here you'd probably use classLoaderIsolation, possibly processIsolation.

See <https://github.com/vlsi/vlsi-release-plugins/blob/45865c3186a7ecc0d30b1f88c2f31160b5e1b13a/plugins/jandex-plugin/src/main/kotlin/com/github/vlsi/jandex/JandexTask.kt#L85|https://github.com/vlsi/vlsi-release-plugins/blob/45865c3186a7ecc0d30b1f88c2f31160b5e1b13a/plugins/jandex-plugin/src/main/kotlin/com/github/vlsi/jandex/JandexTask.kt#L85> :wink:

<@UN84RHR40> Why did you use `.await()` with that error handling here?

Maybe as a quick fix you might be able to find original (not-instrumented) plugin jar from buildscript classpath.
So if you are able to pass `project.buildscript.configurations.getByName("classpath")` to a task and then do smth. like:
```buildscriptClasspath.find { it endsWith "jandex-gradle-plugin-.*.jar" }```
I see that in `buildscript.configurations.getByName("classpath")`  artifacts paths are in “repository cache” `~/.gradle/caches/modules-2/files-2.1`  and not in “instrumented cache” `~/.gradle/caches/jars-9` that you get when you do `SomeClass.class.protectionDomain.codeSource.location` .

I didn’t test it though. :slightly_smiling_face:

<@U02DF9JDXJM> I don’ think that’s a solution. The root cause for this change is a CNFE triggered by missing classes/annotations on the compiled plugin classes. Gradle 7.5 now performs some transformations that were not there before

I recommend <@U023HSZL88M> content. There's a lot of nice resources, videos and GitHub repos at <https://github.com/jjohannes>

I'm not aware of such a service. I _think_ you could use slf4j in your action, though.

But would that connect with the project’s logger settings?

So I have tested a little script and unfortunately it seems that the logger is not bound to the project logger, so logs are swallowed it seems :disappointed:

Do you use process isolation or a different level? (you both)

And there is no injectable logging service yet, no

test script:
```buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath 'org.slf4j:slf4j-api:1.7.25'
    }
}

interface MyWorkParams extends WorkParameters {
    Property&lt;String&gt; getName()
}

abstract class SayHello implements WorkAction&lt;MyWorkParams&gt; {
    
    private static final Logger LOGGER = org.slf4j.LoggerFactory.getLogger(SayHello)
    
    void execute() {
        LOGGER.debug("Hello, {}!", getParameters().getName().get())
    }
}

abstract class SayHelloTask extends DefaultTask {
    @Inject
    abstract WorkerExecutor getWorkerExecutor()

    @TaskAction
    void execute() {
        workerExecutor.noIsolation()
            .submit(SayHello) {
                name.set("Cédric")
            }
    }
}

tasks.register("hello", SayHelloTask)```
calling `hello` shows nothing

I mean, how else would a `WorkerAction` pass information to console/log which is compatible with the build’s logging settings? Sure I can use `println` but, … if so, what’s the point of logging APIs then?

Well, did you enable debug logging <@UAFGSU86M>?
Otherwise it of course does not show anything.

The user guide says
&gt; You can also hook into Gradle’s logging system from within other classes used in the build (classes from the `buildSrc` directory for example). Simply use an SLF4J logger. You can use this logger the same way as you use the provided logger in the build script.


info is only shown with `--info` or `--debug`

trace and debug is only shown with `--debug`

no, because I'm looking for a log level which would bind to `lifecycle` so that it shows something like `println`

That log level is not a standard log level, it is only available with the Gradle own logger subclass

stdout is by default bound to lifecycle if not reconfigured

I know, but if you want to avoid `println`, you have to find _one_ level which works :smile:

Or you cannot use slf4j but have to use the Gradle logger

You can probably use `Logging.getLogger` to get a Gradle logger where you can use `lifecycle`

But that level (and higher) should be used sparsely anyway. :slightly_smiling_face:

so, verified:
• `info` -&gt; logged if you run with `-i`
• `warn` -&gt; logged as lifecycle
• `debug` -&gt; logged if you run with `-d`

Gradle used to log much more information by default, but at some point shifted more and more output to `--info` or below to not clutter the output with info logs and thus better see important messages like warnings and errors.

ah right. But at least shows something on the console.

like no way to only log a single task type

but at least, Andrés, you have your answer: slf4j works

interesting, I had no idea Gradle supported j.u.l

correct, it’s `org.gradle.api.logging.Logging`

Just what I suggested some messages above

ahhh, mixed with `Logger`. `TooManyLoggersException`

It works fine as long as you have the Gradle APIs available and write Gradle specific code.

My gut feeling tells me when there are too many options, go with the Gradle flavor class, as this is the case. thanks <@URQ797WRW>

You can get it from `gradle.startParameters`

But actually, in most cases you should better not do it :slightly_smiling_face:

My use-case is injecting `local.properties` support so I need to have the following resolution in increasing priority:
1. gradle.properties
2. local.properties
3. -P properties

Ah, the good old <https://github.com/gradle/gradle/issues/12283>

Maybe you want to have a look at <https://github.com/stevesaliman/gradle-properties-plugin/>

In the end I just wrote a custom `Project::findProperties` accessor, which is not ideal, but works for most of the cases where I need them.

And of course declaring the dependenc(y,ies)

I'm aware, but that's not what I'm after. I'm looking for some (possibly separate) executable that would download an executable jar, resolve dependencies and would make it available for me to use on the cmdline (like "npm install --global" does for example)

Gradle here would only be relevant for publishing and dependency resolution during install in this scenario

Such a tool could also not require gradle at all and instead choose to implement pom and gradle metadata resolution by itself (although i fail to see why would anyone bother)

Ah, I see, I thought you mean using Gradle as this is the Gradle Community Slack. :slightly_smiling_face:
But well, you could still simply do it with a trivial Gradle build using the `application` plugin.
It would download the dependencies, put start scripts and so on.
You just also define the "global" place where it should be installed and maybe an executable directory that then already is in the path.

But other than that at least I'm not aware of such a tool,
especially as it is quite unusual to publish executable libraries
to a Maven repository as they are more meant for libraries.

Another option would be Kotlin `*.main.kts` scripts.
As any such tool that is not Gradle would probably need some initial installation,
It would be Kotlin that needs to be installed.
In the `*.main.kts` script you can declare your dependencies and invoke the `main` method.
So the "installation" would be to copy the script, change the dependency, and change the main class basically.

Ah, I expected as much. I've posted here simply because the way I envision this is combo of a gradle plugin for publishing and some binary an end user would install to consume said publications

The consuming binary would then use gradle (another prerequisite) to resolve and download dependencies and generate shell scripts after which the installed jar can be invoked on the cmd as standalone

Although it just might be easier to use embeddable kotlin compiler and some simple generated kts files

The downside would be missing gradle metadata support, but that should only be relevant in a very minor number of cases

That way the only prerequisite would be a JVM

I've been using '*.main.kts' for that purpose. It's good and preinstalled in GitHub runners.
You could if you wanted to do some shell magic to add a bootstrap header that installs Kotlin. Kscript has helpers for this

Pretty interested by removing the Kotlin dependency (or even Graalvm maybe?)

I'll proably just build on top of embedded MainKts scripting engine to produce a cli that would download and "install" or run executable jars from maven repos or simply `*.main.kts` scripts from the url. The idea is to have something like `ktx install org.company:exe-jar:1.0.0` and then have `exe-jar` command available on your PATH

i.e. experience would be on par with `dnf install curl`

Actually, it might be more complex though.
For example different executables might need different Java versions, ...

I'm aware, for that I plan to support some known keys in jar manifest or simply a cmdline option to provide custom JAVA_HOME to use for the executable

But that's not my immediate concern, for now I just want to easily install my own scripts and jvm binaries after I reinstall my OS :smile:

FWIW, if you're on MacOS, I've had some success with Homebrew inspired by <https://github.com/JakeWharton/homebrew-repo/blob/trunk/Formula/dependency-watch.rb>

For example, I have a task that modifies the inputs to `maven-publish` tasks which leads to a few files changing.  My assertions are  manually parse the output files to determine if the changes reflect.  The API provided by testkit can help confirm the task status.
But it would be great if I could check the config setup for the task too.

Maybe have a task that validates the other tasks config and call that in your test, validating that it was successfully run?

Hmm.This task would then be created by the plugin?  If I understand you correctly, this would work, but  I’d not be able to use this with the usual junit assertions.  My validation task could either succeed or fail and the error handling and correct messaging would need to be a  part of that task, instead of leveraging the test framework here.

Would you also have any references I can use for plugin tests to understand if I’m approaching the problem incorrectly?  Preferably in kotlin/java :sweat_smile: . I see groovy to be the preferred language for testing with TestKit

Unfortuantely not, no, was just an idea how you could approach it.
And no, I didn't mean that task to be created by the plugin, but to be part of your test build script.
You can also have multiple tasks to test different aspects of the changes either in different tests or in the same test.

Regarding writing tests in Java / Kotlin,
when I first contributed to Gradle itself, I was forced to learn how to use Spock for testing to test my changes.
Now I'm more than happy I was and exclusively use Spock for any tests wherever possible.
Spock is what makes testing fun again instead of a burden. :slightly_smiling_face:
But Spock is the one that enforces Groovy.
You can probably use TestKit just fine with Kotlin or Java,
but my recommendation would actually be to embrace Spock instead and cope with it being Groovy-based.
Luckily almost any valid Java code is also valid Groovy code, so you could actually write your test code "Java-like" even if it then is bad practice. :slightly_smiling_face:

Thanks , let me look into having a task created as a part of the test script.

On using spock,  I’m sold based on what your feedback  and reading tests written in Spock :slightly_smiling_face: . It reminds me of <https://www.spekframework.org|Spek> .

My reasons for doing this in Kotlin/Java right now is to reduce the entry barrier for writing tests  for other folks in my team. I’ll start with  writing tests in Spock on personal/smaller projects and then hopefully get my teammates to use it too :slightly_smiling_face: .

Latest when it comes to writing data driven tests or the integrated mocking and stubbing, they will be caught too. :-)

Well, actually, the thing I love most are the failures. You don't need to take any measures like using extensive Hamcrest API or similar to get helpful assertion failures, you just write simple statements and you automatically see each steps value if it fails.

A slightly stripped down version of the power assertions was even integrated into Groovy itself after they have seen their beauty in Spock. :-)

```val rootProject = ProjectBuilder.builder()
            .build() as ProjectInternal
val subProject = ProjectBuilder.builder()
            .withParent(rootProject)
            .build() as ProjectInternal

subProject.plugins.apply(JavaPlugin::class.java)
subProject.plugins.apply(MyPlugin::class.java)

subProject.evaluate()
rootProject.evaluate()

assertThat(subProject.tasks.getByName("myTask")).isNotNull()```

it lets you get the actual task objects, and interact with a project just like how you would in a plugin / buildscript

Yeah, more unit tests then, than functional tests like with testkit :slightly_smiling_face:

TIL about ProjectBuilder. Which will check that out. Thank you :slightly_smiling_face:.

My immediate goal was to have some functional tests in place  to prevent regressions and I love testkit for that! Will explore what other approaches to use for testing gradle plugins going forward.

Does ProjectBuilder have a user manual / documentation on best practices/ usage?My google fu only leads me to the  javadcocs <https://docs.gradle.org/current/javadoc/org/gradle/testfixtures/ProjectBuilder.html>

I use the javadoc. One gotcha is, try to avoid tests that would cause dependencies to resolve. They tend to be flaky on file locking issues

I can hardly guess what your setup is as you don't show what you have.
But I guess you should simply configure the publications added by the plugin instead of adding own publications.