https://gradle.com/ logo
Join Slack
Powered by
This message was deleted.
# dependency-management
s
This message was deleted.
g
Have you read https://docs.gradle.org/current/userguide/platforms.html? Version catalog can be easily used by multiple project and even builds (including publishing them to maven repo and consuming by GAV coordinates later). And you didn't mention main downside of using 
Version
class in the `buildSrc`: ANY change to it invalidates build classpath and leads to reconfiguration and rebuild of the entire build tree. Change in the catalog only affects configurations that use changed dependency/version. Some points in your cons list are either untrue or just the same as with 
Versions
in the 
buildSrc
.
• your versions are now hidden away in a subdirectory
And 
buildSrc/src/main/kotlin
isn't a subdirectory? So same here.
• changing versions is much more complicated
What's complicated in changing 1.2.3 to 4.5.6 in toml file? It isn't any more complicated than changing it in a groovy/kt file.
• adding a dependency means you might miss adding it to a bundle
You aren't required to add it to a bundle. Same as with 
Versions
class you will add the dependency either to the 
dependencies { .. }
block or to some collection.
• versions catalog isn’t actually the source of truth
And? It's not different to 
Versions
class. The only source of truth is dependency resolution result (potentially with lock file and/or validation).
• command clicking (or jump to) for each dependency actually just jumps to generated code rather than the actual version
It's lack of IDE feature and should be fixed by IDE vendors in future. Currently it's a real downside.
s
Yes I’ve read that.
Version catalog can be easily used by multiple project and even builds (including publishing them to maven repo and consuming by GAV coordinates later).
which you can also do with a plugin, so not really a benefit there
And you didn’t mention main downside of using Version class in the buildSrc: ANY change to it invalidates build classpath and leads to reconfiguration and rebuild of the entire build tree. Change in the catalog only affects configurations that use changed dependency/version.
I don’t see why this is a downside. Anytime you change a version you should recompile anyway to make sure that caching isn’t messing with your dependencies. Maybe if you had a massive multi million line project that makes sense, but I don’t think the majority of apps would need that.
And buildSrc/src/main/kotlin isn’t a subdirectory? So same here.
buildSrc is treated like a module/subproject, not a random directory. I doubt most devs have ever bothered to look in the 
gradle
folder for anything (I have and I understand it, but it’s supposed to be usable by the entire team, not just me).
You aren’t required to add it to a bundle. Same as with Versions class you will add the dependency either to the dependencies { .. } block or to some collection.
Sure but then what have you gained? You’ve now written more code than you would have with a buildSrc folder or even plugin, your dependencies are now less readable, and the catalog doesn’t actually indicate at all what dependencies are actually being used.
And? It’s not different to Versions class. The only source of truth is dependency resolution result (potentially with lock file and/or validation).
Sure, but there has to be some major benefit to catalogs to entice a switch. I see no major benefit, only a list of downsides. This is one of the downsides. I’m not saying that there aren’t downsides to buildSrc, but that this is a list of things I see as downsides of the catalog with hardly any benefit.
It’s lack of IDE feature and should be fixed by IDE vendors in future. Currently it’s a real downside.
even then this doesn’t solve the bundle issue. If bundles were part of the declaration then it would be a different matter, for example in the 
buildSrc
example I can simply do this
Copy code
object Quarkus {
        val kogito = listOf(
            ":kogito-drools",
            "kogito-quarkus",
            "kogito-quarkus-rules",
        ).map { "org.kie.kogito:$it:${Versions.kogito}" }
    }
and my dependencies will never diverge from my bundle. It’s readable and understandable since it’s simply kotlin and there’s no hidden logic behind the scenes. It can be extracted to a plugin if needed or even made configurable on a case by case basis. It’s command clickable. It’s iterable so I could even do case by case exclusions or strategies. I love using new gradle features because in general they make life much better. But this decision is just baffling to me. Why not use the gradle.properties file? Why switch to toml? Why have both settings and toml styles? Why put the toml in the 
gradle
directory (seriously this is so baffling to me, everything else is not there, why this?).
I’m not saying there isn’t a good reason. Clearly I’m misunderstanding something here. But the given reasons so far are not convincing me. Maybe if you had a multimillion line project with 100 subprojects nested 10 levels deep then this might make sense. But I seriously doubt the majority of people have that, so I don’t understand why it’s suddenly the main recommendation I’m seeing here for version management.
I had a logic flaw here:
I’m not saying that there aren’t downsides to buildSrc, but that this is a list of things I see as downsides of the catalog with hardly any benefit.
If I say that then I do need to consider that you can deploy the catalog to artifactories as a benefit.
g
Anytime you change a version you should recompile anyway to make sure that caching isn’t messing with your dependencies.
I don't see why should you recompile modules that don't depend on the changed dependency explicitly or transitively. To me it looks like a downside of the 
Versions
.
buildSrc is treated like a module/subproject ... it’s supposed to be usable by the entire team, not just me
More like included build but I see your point. On the one hand changes to a build tool (be it configuration avoidance, caching, version catalogs, test suites, configuration cache or any other feature) requires a team to invest a little time to use it efficiently, on the other hand it allows to use gradle more efficiently and reduce feedback loop without sacrificing soundness of the build.
Some points in your cons list are either untrue or just the same as with 
Versions
in the 
buildSrc
.
How's that? Both approaches require one to declare dependency somewhere either on bundle/group in a class from the 
buildSrc
or library aliases/GAV coordinates/constants from a class in the 
buildSrc
. I don't see any significant difference between these approaches in code volume.
and my dependencies will never diverge from my bundle
What did you mean here? That you can easily see what's in the bundle? Since diverge might also be about version alignment which neither approach gives, only platform provides it.
Why switch to toml?
It's richer and allows right amount of flexibility. For example to declare rich versions or declare a version and reference it both for plugin and library/bom. For example:
Copy code
toml
[versions]
quarkus = "2.16.4.Final"
[plugins]
quarkus = { id = "io.quarkus", version.ref = "quarkus" }
[libraries]
quarkus-bom = { module = "io.quarkus:quarkus-bom", version.ref = "quarkus" }
declares version in one place (with ability to override in on catalog import if required).
Why have both settings and toml styles?
To allow build catalog programmatically, of course. It's quite useful when you build catalog from a plugin. TOML is just a convenient and concise way to declare the same.
Why put the toml in the gradle directory (seriously this is so baffling to me, everything else is not there, why this?).
Well, a lot of projects stored script plugins in 
gradle/
, it's quite common convention. And dependency verification metadata is stored there iirc. So why not. It's not like you can't store it anywhere else, you'll just have to create catalog explicitly and call 
from(files("path/to/catalog.toml"))
.
Maybe if you had a multimillion line project with 100 subprojects nested 10 levels deep then this might make sense. But I seriously doubt the majority of people have that, so I don’t understand why it’s suddenly the main recommendation I’m seeing here for version management.
It depends. You could have 10k or 100k loc and 10 modules but still have significant build time reduction, esp if you use Scala (or Kotlin but it compiles much faster than Scala), use build augmentation like Quarkus/Micronaut, build native binary with GraalVM or use things like proguard in case of Android development. If it's not a case for you nobody forbids you to use your current approach. Another thing that would be simpler with a catalog is automatic version update in it. I know there were plugins to do it for both your approach and versions and dependencies in the 
ext
vars.
s
Ok wait, I think I might be understanding. This is less about version management and more about providing a centralized location that you can refer to dependencies, even across projects. So you could deploy several different version catalogs maybe segmenting them by major versions of a framework you are using and then in different projects refer to the specific version catalog you need.
let me read over what you wrote.
👍 1
I don’t see why should you recompile modules that don’t depend on the changed dependency explicitly or transitively. To me it looks like a downside of the Versions.
Hm I guess that’s fair. Most of the projects I’ve built have generally had dependencies that were needed across all the modules or I would split them out into separate repos.
How’s that? Both approaches require one to declare dependency somewhere either on bundle/group in a class from the buildSrc or library aliases/GAV coordinates/constants from a class in the buildSrc. I don’t see any significant difference between these approaches in code volume.
I’m not sure which quote you were responding to here.
What did you mean here? That you can easily see what’s in the bundle? Since diverge might also be about version alignment which neither approach gives, only platform provides it.
What I meant was that with bundles you have to explicitly declare them. If someone adds a dependency but forgets to also add it to the bundle then you’ve now got incorrect code with a very difficult to debug error. With the example I gave, adding the dependency implicitly adds it to the bundle. So you never have a situation where you might have added a dependency but not added it as part of the bundle, unless you add it straight to the build.gradle.kts file in which case you won’t have the issues that you would have with the catalog where if you added a dependency there and forgot to add it to a bundle then you might just have non working code.
It’s richer and allows right amount of flexibility. For example to declare rich versions or declare a version and reference it both for plugin and library/bom. For example:
Hm. ok, I guess I can see that. It just seems to provide a lot of downsides for what I was thinking the use case was. But like I said I think I fundamentally misunderstood the purpose of catalogs.
Well, a lot of projects stored script plugins in gradle/, it’s quite common convention. And dependency verification metadata is stored there iirc. So why not. It’s not like you can’t store it anywhere else, you’ll just have to create catalog explicitly and call from(files(“path/to/catalog.toml”)).
Hm. I did not know that.
It depends. You could have 10k or 100k loc and 10 modules but still have significant build time reduction, esp if you use Scala (or Kotlin but it compiles much faster than Scala), use build augmentation like Quarkus/Micronaut, build native binary with GraalVM or use things like proguard in case of Android development.
If it’s not a case for you nobody forbids you to use your current approach. Another thing that would be simpler with a catalog is automatic version update in it. I know there were plugins to do it for both your approach and versions and dependencies in the ext vars.
Ok, now you got me very interested. How would you speed up build time with quarkus and graalvm using this? As far as I know GraalVM still has to compile the entire build tree every single time, does it not? My use case is literally about 40 different Quarkus GraalVM lambdas. And how would version updates be easier?
g
I’m not sure which quote you were responding to here.
Sorry, inserted text from wrong buffer. It was about following paragraph:
Sure but then what have you gained? You’ve now written more code than you would have with a buildSrc folder or even plugin, your dependencies are now less readable, and the catalog doesn’t actually indicate at all what dependencies are actually being used.
So you never have a situation where you might have added a dependency but not added it as part of the bundle
No but is stems from not using bundles much. I have a couple for Quarkus for example but little else.
Ok, now you got me very interested. How would you speed up build time with quarkus and graalvm using this? As far as I know GraalVM still has to compile the entire build tree every single time, does it not? My use case is literally about 40 different Quarkus GraalVM lambdas.
Depends on your project & repo structure and versioning/release policy. In my case I have several applications (which apply 
io.quarkus
plugin and produce jar or binary and docker image) and they use some different dependencies (and a lot of common ones of course). When I update dependency used in one of projects 
./gradlew build
rebuilds only that project and rest stays up-to-date. So heavy 
quarkusBuild
task runs only for the affected module but not others. Also build cache is invalidated only for affected modules whereas with 
buildSrc
change all build cache entries should be invalidated iirc. Whatever you gain something with such approach heavily depends on how you build (e.g. using build caching, reusing build directory like Jenkins runners did some time ago or using ephemeral builders with fresh clone each time). In my case it mostly speeds up local builds but not CI ones.
And how would version updates be easier?
You mean for plugins? Just that they could you standard Gradle API (
VersionCatalogExtension
) to analyze dependency declaration. Both 
ext
and 
Versions
class approaches are more fragile and harder for plugin authors. For downstream users there should be little difference if you adhere to expected format and naming for relevant plugin. As for sharing catalogs I use it for several versioned org-wide catalogs with declarations for common libraries (thing like 
guava
, Apache Commons, logging libs), testing libs and frameworks like Quarkus and gRPC), all versioned and having separate release cycles. Although I did make a step further and have layer over that to simplify bringing both catalog and convention plugins with single declaration in 
settings.gradle.kts
using my open source plugin. This way I have simple settings and build scripts like these:
Copy code
// settings.gradle.kts
plugins { 
  // configures private Nexus/Artifactory repo, its content policy 
  // and provides support for manifests below (to import published catalogs and declare plugin versions)
  id("ws.gross.private-repo") version "0.17.0"
} 

privateRepo {
  manifests {
    create("common") { from("org.example:common-platform:A.B.C") // creates catalog commonLibs, brings convention plugins for java ecosystem and common plugins like lombok, immutables, jandex etc
    create("quarkus") { from("org.example:quarkus-platform:X.Y.Z") // brings quarkus-related plugins and creates quarkusLibs catalog
  }
}
// + includes
Copy code
// some-app/build.gradle.kts
plugins {
  id("org.example.quarkus-application") // applies io.quarkus, brings bom, arc, applies conventions for java etc
  id("org.example.immutables") // brings and configures org.immutables
}

dependencies {
  implementation(quarkusLibs.jaxrs) // brings quarkus-resteasy-reactive-jackson
  implementation(commonLibs.guava)
  implementation(quarkusLibs.bundles.observability) // otel+micrometer+health
}
s
Hm. I think I’m understanding more… Though it’s late and I am having trouble concentrating. I am starting to think this is exactly the solution I need for multiple projects, but for a single project would probably still stick to the buildSrc folder. I’ll need to read over this again when I’m not tired, I think I still have questions.
m
It's lack of IDE feature and should be fixed by IDE vendors in future. Currently it's a real downside.
https://youtrack.jetbrains.com/issue/KTIJ-21844/Gradle-Kotlin-script-support-code-navigation-to-TOML-file-dependency-with-Gradle-Version-Catalogs
⭐ 2
➕ 2
s
@grossws so what are some bad practices with version catalogs then? First one is listed in the docs, which is don’t mix version catalogs with hardcoded strings in the build.gradle. I’d guess the second is to probably not overuse catalogs because then you might be pulling in unneeded dependencies.
g
I'm not sure if there are best practices formed already, catalogs are rather new feature. Still same things as with usual version declarations apply: don't overuse strict and rejected versions without reason especially in libraries and if you use dynamic versions or ranges strongly consider using version locking to have reproducible build (at least from dependency resolution side). As for pulling unneeded deps version catalog wouldn't pull any deps for you (only the catalog itself if you import it using GAV coordinates). Only explicitly used aliases from a catalog would be pulled (and only when relevant configuration is resolved. For example if you use 
testImplementation(libs.hamcrest)
it will be resolved and a jar fetched when 
testCompileJava
is preparing to run but wouldn't be resolved on `compileJava`/`classes`/`jar` etc. Same as using 
testImplementation("org.hamcrest:hamcrest-core:2.2")
.
s
sorry, I meant from a bundle perspective, if you declare 
implementation(libs.bundle.whatever)
then you are going to pull in all those dependencies right?
also, question about the version-catalog plugin, are you not able to declare multiple catalogs with it?
Copy code
plugins {
    `version-catalog`
}

catalog {
// do I just declare multiple of these? 
    versionCatalog {
// doesn't allow 'create' here
        // aws
        version("aws.sdk", "2.17.291")
        library("aws.sdk.apache", "software.amazon.awssdk", "apache-client").versionRef("aws.sdk")
        library("aws.sdk.appconfig", "software.amazon.awssdk", "appconfig").versionRef("aws.sdk")
        library("aws.sdk.lambda", "software.amazon.awssdk", "lambda").versionRef("aws.sdk")
        library("aws.sdk.netty", "software.amazon.awssdk", "netty-nio-client").versionRef("aws.sdk")
        library("aws.sdk.sns", "software.amazon.awssdk", "sns").versionRef("aws.sdk")
        library("aws.sdk.ssm", "software.amazon.awssdk", "ssm").versionRef("aws.sdk")
        library("aws.sdk.urlconnection", "software.amazon.awssdk", "url-connection-client").versionRef("aws.sdk")
        bundle("aws.sdk", listOf("aws.sdk.apache", "aws.sdk.apache",
            "aws.sdk.appconfig",
            "aws.sdk.lambda",
            "aws.sdk.netty",
            "aws.sdk.sns",
            "aws.sdk.ssm",
            "aws.sdk.urlconnection",))
    }
}
g
In case of bundles yes, since they are intended for that. And no, 
version-catalog
plugin allows declaring only one catalog per project (module). If you aren't creating it programmatically it may be simpler to just define it in a toml file and use something like:
Copy code
// aws-catalog/build.gradle.kts
plugins {
  id("version-catalog")
  id("maven-publish")
}

catalog {
  versionCatalog { from(files("../gradle/aws-libs.versions.toml")) }
}

publishing {
  publications.create<MavenPublication>("catalog") { from(components["versionCatalog"]) }
  repositories.maven { /* repo configuration */ }
}
Also this way you could import same toml as a version catalog for the current build for example to create a java platform or some helper libs with same release cycle as the catalog.
s
I actually found out you can publish it like so
Copy code
create<MavenPublication>("VersionCatalog") {
        groupId = "com.quarkus"
        artifactId = "com.quarkus.version.catalog"
        version = libs.versions.toml.get()
        artifact("../gradle/quarkus.versions.toml")
    }
so I might just do that instead.. but I would rather generate this stuff programatically…
Also this way you could import same toml as a version catalog for the current build for example to create a java platform or some helper libs with same release cycle as the catalog.
I don’t understand what you’re saying here.
g
You could publish it this way and it might work but it wouldn't add 
versionCatalogElements
variant with required attributes (
org.gradle.category=platform
and 
org.gradle.usage=version-catalog
).
I don’t understand what you’re saying here.
Given you have catalog defined in 
gradle/aws-libs.versions.toml
you could do the following within one build:
Copy code
// settings.gradle.kts
include("aws-catalog", "aws-platform", "aws-my-helper-lib")

dependencyResolutionManagement {
  versionCatalogs {
    create("awsLibs") { from(files("gradle/aws-libs.versions.toml")) }
  }
}
Copy code
// aws-catalog/build.gradle.kts
plugins { `version-catalog` ; `maven-publish` }
catalog {
  versionCatalog { 
    from(files("gradle/aws-libs.versions.toml"))

    val acmeVersion = version("acme-aws", project.version)
    library("acme-platform", project.groupId.toString(), "aws-platform").versionRef(acmeVersion)
    library("acme-lib", project.groupId.toString(), "aws-my-helper-lib").versionRef(acmeVersion)    
  }
}
publishing { .. }
Copy code
// aws-platform/build.gradle.kts
plugins { `java-platform` ; `maven-publish` }
dependencies {
  constraints {
    // here we use type-safe accessors generated from version catalog declared in settings
    api(awsLibs.aws.sdk.lambda)
    api(awsLibs.aws.sdk.netty)
    // etc
  }
}
publishing { .. }
Copy code
// aws-my-helper-lib/build.gradle.kts
plugins { `java-library` ; `maven-publish` }
dependencies {
  // here we use type-safe accessors generated from version catalog declared in settings
  api(platform(":aws-platform")) // use our platform to align aws sdk libs
  api(awsLibs.aws.sdk.lambda)
  implementation(awsLibs.aws.sdk.netty)
}
publishing { .. }
Then in different project you use 
versionCatalogs { create("awsLibs") { from("org.acme:aws-catalog:X.Y.Z") } }
in the settings script to import published catalog and use something like this:
Copy code
// some-app/build.gradle.kts
plugins { java }
dependencies {
  implementation(platform(awsLibs.acme.platform))
  implementation(awsLibs.acme.lib)
  implementation(awsLibs.aws.sdk.netty)
}
s
Oh that’s dope. Yeah I did want to create helper functions for deployment too 😂
g
I use it for different things but general idea and project layout above with some convention plugins for versioning (based on repo tags), release management, publishing etc simplify that even more. It may not be very useful for your projects though, as they say YMMV
s
Yeah that’s pretty much exactly what I’m going for. I used gradle at my last job, but it’s been a few years. I built a plugin to do pretty much all that. We’ve been getting by at my current job, but we have so many repos at this point and we’re pretty monorepo averse that I’m trying to solve this with gradle, which we can then maybe one day switch to a monorepo.
@grossws so how are you managing the versions for these? Do you just increment patch or minor versions every time you update your version catalog? Or are you tying your catalog version to a version like the aws sdk or quarkus bom or something like that?
g
I use independent semantic versioning with version per repo, a middle ground between monorepo and microrepos. E.g. for Quarkus minor version update or new feature in one of my convention plugins for Quarkus I bump minor version, for Quarkus patch update or smaller changes I just bump a patch version. But they stay different to Quarkus own versions.
Also you might like to look at this blog post about catalogs: https://gradle-community.slack.com/archives/C04FMAL8RTM/p1678645014599919
s
awesome. I’ll read it today. Thanks!
I made a nice kotlin dsl to solve the problems I had with bundling. Hopefully I don’t need to use bundling too much, but this gives me the best of both worlds:
Copy code
libraries("aws.sdk") {
          group("software.amazon.awssdk", "2.17.291") {
              +lib("apache-client", "required")
              +lib("appconfig", "required")
              +lib("lambda", "required")
              +lib("netty-nio-client", "required")
              +lib("ssm", "required")
              +lib("sns")
              +lib("url-connection-client")
          }
      }
this is what that generates.
libs.versions.toml
👍 1
initially I had done this, but it’s super messy.
Untitled.kt
they generate syntactically the same catalog
much cleaner
12 Views
Open in Slack
PreviousNext
Powered by