This message was deleted.

In my org, its easily 90+%. We only use a few oss plugins directly, but many under the hood of our custom plugins. In most of our projects, the scripts are just declaring plugins, dependencies, and maybe configuring a couple extensions. We have about 15 custom plugins (not all applied directly, we have convention plugins). Some totally original, others applying oss plugins with our standard config. I'm just wondering how typical this is

We have around 50 common plugins (most are published in private nexus repo but some are on the plugin portal to simplify bootstrapping). Mostly they are convention plugins and/or wrappers around open source plugins (~35 of total count). Other group is plugins to develop plugins and simplify tasks like repo, plugin, platforms/boms and version catalogs management. It allows typical project to have something like this in `settings.gradle.kts`:

plugins { id("ws.gross.private-repo") version "0.15.0" }
privateRepo {
  manifests {
    create("build-logic") { from("com.example.gradle:manifest:x.y.z") }
  }
}

which configures private nexus repository for plugins, dependency management and publishing, fetches manifest containing list of plugins and version catalogs and applies it allowing

subproject/build.gradle.kts

to look like:

plugins {
  id("com.example.java-library")
  id("com.example.lombok") 
}
dependencies {
  implementation(commonLibs.guava)
  testImplementation(testingLibs.mockito)
}

Last category of plugins we use is adhoc in a project separated into the in-project included build, for example for codegen exposing simple extension to configure it. Fortunately it's all green field since we lived with Apache Maven (and Ant in some legacy apps) for years. And we are very small org. So we try to have only project specific code in the

build.gradle.kts

(at most plugins, dependencies, test suites dependencies and config via extensions). Not entirely clean, of course, things like adding

--add-opens

to

quarkusDev

task are not worth extracting them to convention plugin usually. Almost all projects are that way except abandoned/archived ones but 80-85% projects are still on maven.

I'm just wondering how typical this is

I don't know how typical it is, but optimally all bigger builds should look like that. At least that is what is called an "idiomatic" build. :-) Our main build is far from that as it grew from pre-1.0 in an organic way, but I'm trying to clean up that mess. :-)

some pain if you want target several Gradle versions and want use new features if available (heil reflection/method handles)

Alternatively to reflection you could for Gradle <7 also check the version and have the version specific code in separate classes. And just in case you didn't know, since Gradle 7 you can asleep have different variants of the plugin that are selected automatically according to the consuming Gradle version. :-)

As an IT services and consulting company, our code eventually belongs to our customers, so we don't share plugins across projects. Each Gradle project generally has a couple to a handful convention plugins (generally, a

base

plugin setting up Ktlint through Spotless for the

*.gradle.kts

, a

java-base

one configuring a toolchain, JavaCompile options, Error Prone, Forbidden APIs, and Google Java Format through Spotless, and a

dependency-management

plugin for

logging-capabilities

and the like; and that's it). In terms of third-party plugins we almost always use: Spotless, ErrorProne and Nullaway, FlywayDB, logging-capabilities, Forbidden APIs, and CycloneDX, and sometimes Nebula packaging when we need Deb or RPM packages (we're slowly moving to separate debhelper and rpm steps though). We do mostly zero-framework magnificent monoliths in Java with React or Vue SPAs and, for instance, we never call Node or Docker from Gradle; orchestration is done at the Jenkinsfile level (Node and Gradle in parallel for client and server, then Docker to package it all ; the Gradle build only prepares directories for the subsequent Docker build)

Wow thanks everyone for the insight. I think our environment is closest to @grossws although we use a custom wrapper instead of settings plugin to configure plugin repo/deps. What are the pros/cons there? Do you need to publish that plugin publicly? For supporting gradle versions, I have had success supporting 2 major versions at a time, giving people time to upgrade to the second version before dropping support for the old one and adding support for the next one. But that does mean we aren't able to leverage the latest and greatest features in our plugins

Imho Kotlin is always worth it for any build script. And it is very much worth it for custom tasks and plugins, but only if the Game versions to be supported is a fixed small defined list. For a publicly published plugin that should be compatible with many Gradle versions, I would always prefer using Java and there the lowest version supported to run Gradle in the lowest version that should be supported. With Groovy or Kotlin there is too much shenanigans with compatibility and runtime versions and so on in that context.

For a context I have several kinds of gradle projects: our internal projects (which need to use private repo), some open source projects (whose repos/plugins I don't want to affect) and some customers' projects with their own repo setup (either repos defined in the projects or via separate init script). Main pros of a settings plugin is using vanilla gradle infra (wrapper, distribution, wrapper-validation-action) while keeping enough flexibility. I tried to make plugin more or less generic to it be useful not only for our org but to the others as well (it still needs a lot of work). Another approach that we used was custom distribution with init script embedded but it has some pros as well: you need infra to serve these distributions (preferably without auth), have to use custom wrapper task or full distribution url all the time.

Yeah the hosting of the custom wrapper was a problem I had to solve. I ended up putting it in our private artifactory, and using the gradle properties to set auth

With Groovy or Kotlin there is too much shenanigans with compatibility and runtime versions and so on in that context.

Is this true of Kotlin even with the classloader isolation provided by the

plugins

DSL block?

Not sure what classloader isolation you are talking about, but I think the Gradle provided runtime should always be in a higher class loader, even if the plugins would be isolated from each other which would be news to me and would be problematic even because they could then hardly with hand in hand. Besides that plugins are also used as normal dependencies in plugin projects for convention plugins and so on.

I'm referring to this section of the userguide, where it says the plugins DSL allows "different plugins to use different versions of dependencies"

It says it allows, it does not say it does so. 😄 I'm not sure why this is written there and whether it has any relevance. I did a quick try and I don't see any isolation. I created two separate included builds that use different versions of a dependency and just print their version that is retrievable through API. If I only apply the plugin with the older version, the older version is printed. If I apply both plugins, both print the newer version. So I don't see any isolation there. And actually even if it were isolated in this specific case, as I said, there are many more cases where it would not help, like when using legacy apply method, or when using convention plugin project with both plugins, and as I said even if there would be isolation, the runtime would probably still come from a higher class loader. I'd recommend to post an issue about the doc statement, so that it gets either removed or clarified.

😞

I have seriously mixed feelings about internal plugins. One the 'one hand' they are a very compelling abstraction with the potential of solve vast amounts of build pain, especially in complex projects that need to work togher -- the ability to 'fix' the dependancy versions, core configuration values, versioning schemes, publishing -- all the Good Stuff On the other hand - my success both in individual attempts (developing them) and in larger context (inheriting plugins from prior projects/teams) -- have been a very rocky road with a benifit/problem ration below 1.0 sometimes near 0 Why? Modifying shared plugins is painful -- especially plugins related to version control and dependancies. Typically I have to push a plugin update through all layers of the CI system (to all dependent builds) and build everything in order to tell if it worked and how bad the damage is. Example, bumping the kotlin version and dependent libs , has to make it through all dependent builds upfront - you cant just rebuild the part your working on, everything else has to built first and then published and then new versions referenced etc. This can turn a '1 hour project' into a week or more Debugging plugins is a real pain. Debugging gradle itself is painful, but debugging plugins even more so -- everything lifecyle and defered/lazy dependency issue, ordering issues etc multiple going from debugging build.gradle to debugging a plugin. All the docs say its awsome and easy, and maybe for everyone else it is, but not me or anyone I work with. Pretty much the definiton of hell. The compelling beauty of turning a 1000 line gradle script into a 10 line one -- is incredible. But at the cost of visibility -- when 'stuff doesn't work right' -- figuring out why and where is vastly more painful. In a real-life example, I inherited an android build system that made use of a single internal plugin that did 'all the rights stuff' -- or so the authors belived. Established naming conventions, publication tweaks, enforced use of release builds vs snapshots at the right place, integrated with the Jenkins file for CI/CD, even created build tags - awsome. Except it was horribly flawed in a few places -- one was a mistaken understanding of how git tags worked which would mysteriously result in builds pulled from the wrong branch and published with no evidence that the code was from the wrong branch, created tages every CI build that make finding real commits hard, enforced a naming convention that didnt follow a pattern anyone could figure out, and finally was written at a fixed point in spacetime -- which meant every time we tried to upgrade gradle or kotlin or android extensions -- the build would break and we couldn't figure it out -- it would be some use of internal android or gradle classes in the build script hat had become deprecated or changed -- it was *extremely obscure and impossible to maintain In fact it took me several months finally LOCATE the damn thing then almost a year to decipher it enough to extract it out of the builds without breaking them for weeks on end -- The Jeweled Garden that was created turned into self-made Cage of Thorns that in the end caused 10 to 100x more damage then it created benifit. I would not want to put anyone else through what I had to do .. My takeaway -- if you are going to do internal plugins , you need them built and actively maintained by people who are gradle experts, and who must be considered a critical part of the team for the entire project lifecycle. IMHO this is not an area to express ones inner creativity. Its an area you need to be very conservative, using all of the core software methodology seriously, and do not even think of doing unless the organization is committed to supporting its development and maintenance as much as it does the product. The skills needed to debug a plugin are skills most developers will never acquire and should not be expected to, its a very specialized skill that takes a lot of work and time and requires a personality that enjoys extremely difficult challenges under pressure in a very closed technology stack.

Yeah setting dependency versions seems not great. I think keeping that in build script land is the right call. One exception I have found: we have an internal framework that publishes a BOM. We use a plugin to determine the latest "stable" and non-"banned" version of that framework, and apply that version of the BOM as an enforced platform. That way, people get the latest automatically, but without dynamic versions, and if we ban a version (due to a bug being found) everyone will fallback to previous version automatically until we get a fix out.

Thanks for the issue @Ryan Schmitt. This sentence was added years back and was wishful thinking at the time. Here's a PR to remove it https://github.com/gradle/gradle/pull/21448

@Paul Merlin Nice in theory. In practice -- not really. It has gotten better over the last few versions, but even today (gradle 7+) in the projects I maintain, included builds are problematic in a chained dependency, and often even in a simple one. When they work perfectly, its great, but then when they dont -- not so. TO the point that they do not work as a validation that a plugin wont break the builds. Both false positive and negatives. Plus all the caching layers -- A->include B,C,D B->include D, F , D -L> include C,G,E etc .. to get those all to 'swap over to included builds' -- all at the same time and without caches pulling in the wrong stuff without noticing (and dont even start on the IDE) -- success chances ? near 0. the range of things where included builds "dont quite work the same" is large -- very large. This is a sort of self-defining problem. Strategies like included builds tend to work better on smaller and simpler projects exactly the same projects where its not to hard to debug them and probably don't need a bunch of plugins to manage. But on larger project where you really could use a plugin to help manage things is where the includedBuilds tend to get messy. And the whole idea of not needing to be a gradle expert falls apart -- you have to know quite a lot to construct a build that will always work perfectly as an included build in a large project. You have to get unique project names, not do any 'tricks' in the publishing, always use the default configrations etc - and the list of 'WTF' not documented is large but hidden. All that said -- I do use org wide self-made plugins for exactly all the 'Good Reasons' -- and have -- after a long long time (years) gotten them stable and have memorized the happy path to update them -- if I dont touch them and dont stray off the happy path -- they do wonders. They do achieve the desired goals. I set the kotlin version, hard-fix the versions of key dependencies to force all projects to use the same version of jackson, kotlinx etc, and the most difficult one, I recently added integration to AWS CodeCommit including a build plugin (3rd party) that launches an internal local maven repo and maven authenticator inside gradle to enable the bootstrapping of pulling the initial core plugin itself from code commit without having to put authentication in any project -- and that is the plugin that implements the codecommit integration. It is really cool -- but I would never do it again, and would not recommend it. What I thought would be an afternoon of fun coding turned into 2 weeks+ of hell -- and this is a small project base - < 10 git repos with only 5 active ones -- so why so hard ? Getting all the right repos for resolving and publishing across main, feature/snapshot and release branches, bootstrapping the initial plugin into the CI/CD tool, managing different sources of auth secrets depending on if its a local/dev vs CI build, managing caching of SNAPSHOT of the build plugin itself, the difficulty of debugging a remote CI build running inside the build server where its got an entirely different environement, kicking off huge chains of builds and propagating all the built artifacts into the next chain to discover a bug 4 levels down but not knowing if they pulled different cached versions or was it a bug. And this is for a 1-person team - me. I can tolerate breaking my own builds -- painful but self inflected. At my 'day job' -- not so much tolerance for builds breaking, you cannot test stuff like this in different environments reliably as the core domain of what the build tools tend to do is manage environment specific build configurations -- so you only get so far in a sandbox before you have to commit it to the real thing -- OR -- get IT to allocate you a cloned environment -- good luck, and when/if you get it, how close is it to the real one? not close enough. Then when its all working perfectly -- there are the other developers who are extremely creative at doing things slightly differently then I did -- and then they start having problems that they don't know where they came from --- What I did in my 10 project personal build -- would have taken 6 months at my day job -- if they didn't fire me first. I dont mean to be depressing and 'gradle bashing' -- gradle is at the very top of my list of amazing tools I do not want to live without. Its also at the top of my list of "Extremely complex and subtle tools you better not touch unless you have the time and patience 100x what you would predict". Perhaps my experience is not common. However, I think it is more common then people admit. Most engineers are embarrassed to admit they cant figure out gradle at all -- if it doesnt work out of the box they are totally lost, or worse , try to fix it by adding more and more code to the build script --> exactly the wrong thing to do. Note: All the above commentary is from the perspective of a lead developer attempting to 'play devops' in teams where there were no other experts to draw from, attempting to improve things 'in my spare time' . Your Org may be quite different. Opinion: If you have good experience in all this and dont share my experience, good ! go for it, ignore the above as a unusual exception to the norm. If you have NOT attempted to manage a set of active projects in a team with custom (or 3rd party) common build plugins as described and are considering trying it for the great anticipated reward -- I recommend doing so very carefully and strategically and make a back-out plan to handle the strong possibility that you get far into the whole thing and it just doesn't quite work -- and the end of the tunnel looks ALMOST THERE -- for days and weeks making good progress but somehow not getting closer to the end. Many of the most severe pain points are very specific and coupled to the intricacies of integrations (source control, build servers , publication and resolution repositories, build versioning conventions or lack, distribution of workers by location and skills, access-or-lack of good connectivity, differences between developers 'desktop environement', company culture, management, tolerance for 'failure' etc. Its all those 'non gradle' things that really make it go from 'very complicated, but solvable' to 'never-ending-toruture'