From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

<https://docs.gradle.org/current/userguide/build_lifecycle.html>
I suppose this would be your best entry point.

Basically said, the setting script defines which projects are part of your build and some build-global settings, build scripts configure the actual build. Some settings you can only do in either of them, some settings you can make centrally for all projects in the build within the settings script. Repositories for example can be configured in each project build script as needed, or you can just configure it centrally for all projects in the build within the settings script.

I'm on 6.9.1 on mac with a monitoring agent that routinely makes file access take seconds, the biggest I've seen is a variation of about 20 seconds

Okay - Im wondering if this is it: <https://github.com/gradle/gradle/issues/14927>

I’m thinking that I’m hitting this because I use IntelliJ for editing but tend to return to a separate shell to execute gradle commands.

A build scan will give some insights in to where your buidl is spending time during the configuration phase.

There are definitely memory leaks somewhere. I experience this all the time with Gradle (since v7) + AGP + Kotlin, and each of those have their own issues. 

I've been filing a lot of memory leak bugs, and while some have been fixed, many still haven't been, and new ones keep popping up. 

In the interim I just kill the daemon every few builds.

Sounds like the annotation has class retention, not runtime retention. Did you use the correct annotation? 

Btw. I wouldn't use Maven local but better use a composite build.

Thanks <@URQ797WRW>. I was able to use `pluginClass.getConstructors()[0].getAnnotations()` to find that the actual annotation on the constructor is `@shadow.javax.inject.Inject()` . So it seems the publication is shadowing annotations. That gives me something to go on.

What are your configuration times like at the moment?

depends, can be quite high when invalidating `buildSrc`

My opinion on this has shifted over time. For context I maintain a buildsystem for a Java project (with some limited JS components). The main repo is 360 kloc, from bare the config takes sub 60 seconds, and once everything is warm it’s sub 10s. I tried to go “full bore” configuration avoidance and you know what, I realized my time was much better spent working on integration/unit test runtimes, or even on kit assembly and improved incremental-ness.

I think Gradle’s emphasis on configuration avoidance says a lot about whats important to their customers, but not necessarily much about its importance to any given build.

There’s also the problem that the necessary API changes needed to reach full config avoidance haven’t reached far enough in to the wider plugin ecosystem (or even the entire Gradle API last time I checked) to make it actually possible to achieve “full config avoidance”

<https://twitter.com/CedricChampeau/status/1493516333170016256?s=20&amp;t=oMvU7OyT1cf6yKjiDPsVgA>

I am not using `configureEach` a lot because of that, if it should be used maybe getting a lint warning...

The biggest issue I see with the snippet is that its ascribing meaning to task names. A correctness problem probably trumps a performance problem. :wink:

agreed - we definitely need a lint warning to catch these issues

<@U01DUV4Q882> what do you use instead then? You should always use `configureEach`. The problem in that blog post is not the `configureEach`, but the unguarded `matching`. To evaluate the condition, the tasks need to be realized. So if you do `matching` directly on `tasks`, each and every task needs to be realized.

Sadly the first one because I forget to add the configureEach because both gives the task in the lambda.

Personally I think Gradle has an issue about default or intuitive things not being the recommended approach, overall in Kotlin

This is an example but I thought exactly the same when I saw that the properties should be set via `someProp.set(…)`.

A delegate or whatever which allows using `someProp = …` should be great for example

I understand that this issue is because the intuitive things were the “previous mode” and it can't be removed or deprecated, so altering it can be difficult to be achieved by Gradle engineers without affecting a lot to everyone

Ah, misunderstood "because of that". I thought you do that intentionally because of the blog post you linked to.

Right about the unlucky defaults. Unfortunately not easy to change. The legacy way was there first and thus convenience methods for it created. Now you cannot easily change that without breaking things.

A delegate for the properties was also discussed already and there is even a lib for plugin authors providing that for their users, but you break provider chains with that, which is probably the main cause that is not built-in

Ah, sad :cry:. Maybe with compiler plugins this can be improved, but this is not going to happen soon I guess 

You might want  to thumbs-up and follow this ticket: <https://github.com/gradle/gradle/issues/9268>

because `Provider&lt;Task&gt;` is the `dependsOn(..)` type, the task dependency ends up being ignored

What you may end up having is a provider without any _implicit_ task dependencies. If you flatMap into a `Provider&lt;Task&gt;` you should be fine. Basically, the flatMap will use the task dependencies of the returned provider from the transform. At the dependsOn level, you can also use a `Callable` which can return anything Gradle understand as a dependency, i.e. task path as String, task instance or `Provider`. Not that every time Gradle is face with a `Provider` it would unpack the value for the _implicit_ task dependencies but instead use whatever the provider returns (which is an internal feature hence “implicit”). It all boils down to `map` (use this provider dependencies) vs `flatMap` (use returned provider dependencies).

I’m fairly certain both `map` and `flatMap` suffer this problem

I solved this by just pulling the gradle property directly during configuration time.

```val deviceTestRunner: Provider&lt;String&gt; = // gradle property..
dependsOn(
    when (deviceTestRunner.get()) {
        MARATHON -&gt; marathonTask
        SPOON -&gt; spoonTask
        AGP_CONNECTED -&gt; agpConnectedTask
        null -&gt; error("was null")
    }
)```


According to the PR you posted a system property called `org.gradle.internal.http.idleConnectionTimeout`

Thanks <@URQ797WRW> - I think I might have been lacking some coffee earlier :stuck_out_tongue:

Yes, it’s possible. Whether it’s a good idea depends on why you want to do it. How much pain you feel depends on the very precise details of what you want to achieve.

(I’ve not done it with modules… but have with regular JARs)

I want to use modules to enforce architectural constraints between packages in a single application. (And possibly, if I can get my head around how to do it, it to avoid jar hell by using classloader isolation for unexported dependencies.)

I've done it with subprojects, but you do end up with a bit of a mess of nested directories to navigate (even if, as I do, you flatten down the default structure a bit).

In an ideal world I'd have a single sourceset, but drop in multiple `module-info.java` files to enforce the module constraints about which packages could depend on each other, so that it would look like a "normal" project with all the packages in one sourcset . But I'm not sure that's possible.

It's possible that multiple sourcesets would give me what I want... I'm pretty sure I can remove the need to nest a `java` directory under the sourceset's directory.

How many modules are we talking about here? And how many kloc (to get an idea of the size of the whole build)

My POC has 10 modules &amp; 4 kloc, but I'm not entirely sure why that's relevant? It's the concept of being able to enforce the dependency graph between packages I'm interested in, but without introducing vastly more directory nesting than you would get if you made it just one module.

It sounds like you want to have a small number of gradle projects generate a much larger number of JPMS modules? Or more concretely it sounds like you want a more compact directory structure to represent a large number of modules than Gradle can current achieve?

Here's a hugely simplified take on what I currently have:
<https://gist.github.com/Mahoney/d81c82eae13eba7bb9b6298ebed625db>

And here's what I'd ideally have:
<https://gist.github.com/Mahoney/0438153af917ab8cf13c2327e2c82205>

Okay - that is possible. From my experience the biggest issue with trying to make this work will be publication complications - but depending on your deployment model I think this may not matter too much.

If I were you I might play around with how well Gradle handles it if you ho more aggressive on your current strategy.

and then had your kotlin source dir be `.`, then I think you can get it this flat without throwing out the submodules and making the gradle very complex

That's interesting - I never thought of making the source dir be the project dir. I guess I can exclude the `build.gradle.kts` from getting bundled up in the output.

publication is another aspect that matters in some cases.

I would say beyond just enforcement, it’s much easier to conceptually model and understand a large application that is split across multiple projects. Just as you split code across methods, classes and packages.

this applies to both the Java code , but also the gradle aspects. For example, I normally have a subproject dedicated to the “main” class, plus the surrounding build logic to assemble the docker image.  As well as splitting the Java code, it feels tidier to have this encapsulated in a separate gradle project, rather than one monolithic one (once it’s over a certain size)

literally because as soon as you need to deliver your application, or part of your code that makes up your application, in any flexible way - you need to have multiple projects. It helps with scaling too, as the project grows in size

In the same sense that single responsibility is valuable, so are multiple modules (subprojects)

<@UAB9WQKJ4> do you mean when you want to publish a single artifact from each subproject?

Yes and also when your code grows and you start wondering if everything should be published in the same artifact. But maybe variants are not the right concept, etc …

When I give introduction to Gradle, I usually tell that Gradle first is a language which you may use to _model your software_ (aka _define the structure_ of  or _define the architecture_ of your software_)._ Then the Build Tool is a tool that reads this model to build the software (but other tools, like IntelliJ, read that model too).
I guess that’s your point “enforcing application layers” just expressed a bot more generic. And it’s also the point <@UDWDHK89G> made.

Then you have:
1. Stronger modelling concept (e.g. compared to plain java packages) like “implementation” dependencies that enforce a certain encapsulation between the components of your software
2. Performance improvements (which you listed) that Gradle is able to offer due to this ‘rich’ model it has of the software

My thought is that I would have 3 styles of plugins:

```myApp.codeCoverage.aggregator
myApp.codeCoverage.library.android // for android subprojects
myApp.codeCoverage.library.java // for java (non-android) subprojects```
`myApp.codeCoverage.library.android` would create tasks that expose 3 types of code coverage values - jvm tests, android tests, and merged jvm/android test coverage.

`myApp.codeCoverage.library.java` would expose a task to create code coverage reports for jvm tests. Though I might want to explore a different report for different test suites in the future.

`myApp.codeCoverage.aggregator`’s job would be to take the above information and merge it to create a unified report across multiple subprojects. However, I would like this to be dynamic. I might have 5 subprojects that make up one feature, and I would like to have a single coverage report for just those 5 subprojects.

What is the idiomatic gradle way to achieve this?

Does the jacoco report aggregation do what you want?
<https://docs.gradle.org/7.4-rc-2/userguide/jacoco_report_aggregation_plugin.html>

it might (I’m not sure how compatible it is with Android at the moment).

What I would be designing would be a wrapper around it

<@USNE4GL0G> I think you might be suggesting that the `jacoco-report-aggregation` plugin automatically pulls in transitive dependencies

I’m a bit hesitant to adopt `Kover` given its design issues around project isolation. Created <https://github.com/Kotlin/kotlinx-kover/issues/144|Project Isolation support>

I wrote a blog post about it <https://developer.squareup.com/blog/herding-elephants/>

one of my favorite posts! I probably should I have asked my question differently - it’s more about what are the _differences_.

i.e. - what features do I lose when making `buildSrc` an included build?

After renaming `buildSrc/` to `buildSource/`, and doing `include(“buildSource”)` - my code no longer compiles.

For example, I’m still in the process of migrating to version catalogs. So my `Dependencies.GradlePlugins.android` that are added to the root `build.gradle.kts` are no longer found.

```buildscript {
 dependencies {
   classpath(Dependencies.GradlePlugins.android)
}```

I guess I would change the above to something like:

```dependencies(project(“:buildSource”))```

Or does that defeat the whole purpose of `:buildSource` as an included build?

ish? Might be a worthwhile incremental step forward. First step is always to get your code to compile, then iterate to make it compile faster :slightly_smiling_face:

so there’s really 3 pieces of logic that I need to be migrated:

• dependencies/versions (handled by version catalogs, so we’re good here)
• helper functions
• plugins
It seems that plugins will be handled automatically. I have plugins like `id(“com.conventions.library.android”` - but currently using the generated accessor - `com.conventions.library.android`. Do I lose this?

I also have some helper extension functions. For example:

// variant that will respect `local.properties
```
fun ProviderFactory.gradleProperty(project: Project, propertyName: String)

fun Provider&lt;String&gt;.mapBoolean(default: Boolean? = null): Provider&lt;Boolean&gt;```
// looks up named task, or registers it if it does not exist
```
fun &lt;T&gt; TaskContainer.namedOrRegister(taskName: String): TaskProvider&lt;T&gt;```
My initial thought is, after putting everything directly on the classpath like `dependencies(project(“:buildSource”))` to break it up like `:buildSource:helper-functions` and `:buildSource:plugins`. Then only put `:buildSource:helper-functions` on the global classpath.

Or is there a better way to do this?

we actually do exactly that right now. My goal is to get rid of such helper functions entirely and rely solely on `plugins {}` for adding plugins from my included build. But I believe in making progress towards that incrementally

you lose accessor generation but you can create plugin aliases if you migrate to catalogs 

oh nice! I would like to see enhanced IDE navigation for plugin aliases in that case. Would be nice to be able to cmd + B `com.conventions.library.android` and navigate directly to that plugin, but that’s a side note

IDE support has a long way to be completely polished. 

No wrong unresolved references, toml autocomplete, shortcut navigation to original files and not generated files and so on

question for you <@UES7RHBDZ> :

Given the following design:
```:buildSource:helper-functions // on `buildScript { .. } classpath
:buildSource:plugins```
Is `:buildSource` a multi-project build, or are `:buildSource:helper-functions` and `:buildSource:plugins` each single project builds?

I also need to figure out the best way to share `settings.gradle.kts` and `gradle.properties` configuration between each of these included builds. I could use symlinks I think - but would lose the ability to modify `settings`/`gradle.properties` differently between included builds (not sure if that is needed at all)

and I could migrate code from `:buildSource:helper-functions`
```fun ProviderFactory.gradleProperty(project: Project, propertyName: String)```
into `:buildSource:plugins` by making it a plugin, and changing the API to:

```// original
providers.gradleProperty(project, "propertyName")

// original
providersEnhanced.gradleProperty("propertyName")```
Where `providersEnhanced` (yes I need a better name lol) is a gradle extension.

just throwing this stuff out there to make sure I’m headed in the right direction

&gt; Is :buildSource a multi-project build
yes. You almost always want multi-project builds, with empty root projects, except for the most trivial cases

for your helper functions, moving them into a plugin is probably the most idiomatic way to do it from gradle's perspective. We have a small handful of such modules left in our build, which we add to the root `buildscript.dependencies.classpath...` One possible pattern for you is to have those modules be normal JVM library modules, add them as dependencies to your convention plugin modules, and ensure that a convention plugin (like `com.mycompany.base-plugin` is applied to all modules/projects in your main build. Thus you have your utility functions available everywhere.

Having said that, what you should really strive for is declarative build scripts that don't use those functions directly at all. Rather, your plugins use them, and you elide this question entirely

we have a lot of redundancy in our explosion of settings.gradle scripts, unfortunately. Right now I just live with it.

so structure becomes:
```// root (main) settings.gradle.kts
includeBuild(":buildSource) // multiproject```
where the multi-project consists of two subprojects:
• `:buildSource:plugins`
• `:buildSource:helper-functions`
And the main build’s root `build.gradle.kts` file declares:
```buildscript { 
  dependencies {
    classpath(project(":buildSource:helper-functions"))
  }
}```
And slowly migrating away from `:helper-functions` into `:plugins` where possible.

I’m guessing it would be good to break apart `:plugins` eventually into dedicated subprojects for each plugin, too?

one thing I don’t get though is how classpath changes to `:plugins` would affect the main build. With `buildSrc` this is easy - everything invalidates the main build!

For example, if I add a new `internal fun` within `:plugins` - did I just invalidate my entire main build, given that `internal` is package private, so to java code it’s a public API change?

I don't quite recall, I think there may be some Kotlin ABI analysis that would prevent that, but you can experiment pretty easily

for the included build, you might want
```// settings.gradle
pluginManagement{
  includeBuild(":buildSource")
}```
This is the preferred way to include build that provide plugins. And it's the only way if you want to include builds that provide _Settings_ plugins

Don't think so as all the tests are running in the same jvm fork. I guess you could create a bunch of test tasks yourself and filter them to only execute a test per task, but that's gonna be a big overhead for gradle.

Can you describe your use case more?
I actually do not understand what you would achieve with your request.

I have a ktor integration that check if there is connection available to do A or B. This integration is KMP too.

I could create a UI test in an Android emulator and setting the internet connection to off but it would be annoying to create an Android UI test for this and I would lose the verification on the rest of platform.

The other solution in changing the implementation to inject a fake and setting that check to false, but I would like to know if there were a way to force in a specific test the condition of not having access to the network or something similar which would be having a more realistic use case.

probably
```FilenameKt.whenModuleConfigAdded(project, action)```
unless your Kotlin file uses `@file:JvmName` to override the generated class name

In Gradle 6.3 `gradleApi()` adds the Gradle api of Gradle 6.4? That's pretty unlikely, it should add the Gradle api of 6.3.

And regarding the upgrade, you shouldn't. If you are developing a plugin / task for Gradle, you have to deal with the Kotlin version Gradle ships with or you get into problems easily.

In short, I have a task that I always want to run after a specific type of task

No, overriding it makes no sense. You are expected to call it

And don't filter tasks like that, that would eagerly realize all tasks and totally go against task configuration avoidance.

`mustRunAfter(getProject().getTasks().withType(Jar.class))`

That's then also lazy, catching jar tasks registered later after you made that call

From where do I call it? From within the Taskaction or from the plugin where I create the task?

So something like
```var qp = project.getTasks().create(QPTask.create(GROUP_NAME));
qp.mustRunAfter(project.getTasks().withType(Jar.class));```

From the plugin is a good place, yes.
You could also do it for example from the constructor of the task.
But it is more idiomatic to not have opinon (like these orderings or also default values) within tasks and extensions, but have them as unopinionated as possible and add the opinion by the plugin adding them.
This way they can also more easily be used for a different purpose.

How are you "setting it up"? Are you using `./gradlew`? Are you building inside an ephemeral Docker container?

yes I am using .gradlew and I am using the gradle jdk image -&gt; <https://hub.docker.com/_/gradle>

```podTemplate(
    name: 'test-pod',
    label: 'test-pod',
    containers: [
        containerTemplate(name: 'gradle2', image: 'gradle:7.3.3-jdk11', ttyEnabled: true, command: 'cat'),
       
    ],```


If you're using a `gradle` image, then use `gradle` (from the image) rather than `./gradlew` (which will download the distribution *even if it's the same version as in the image*).

As for dependency caching, AFAICT you're using ephemeral containers so it's expected to start empty and share nothing with previous builds.
See <https://docs.gradle.org/current/userguide/dependency_resolution.html#sub:ephemeral-ci-cache> for hints.

(fwiw, we don't use Kubernetes, so our workspaces are bind-mount into the agent container and we configure the Gradle User Home inside the workspace such that everything is "cached" inside the workspace, which is reused between builds)

<@UDC9Q9RD0> the gradle tests are not getting cached on *Jenkins*. It runs the tests everytime on Jenkins. (though locally the tests run only when the code has been changed. It works fine in local setup). Any pointers on how to debug this?

Depends on what you "cache" between builds. Look at which task has been rerun (possibly use `--console=plain`) and see what more you can cache. Maybe enable build cache and cache the `~/.gradle/caches/build-cache-1/` (or wherever you configured the local build cache to be stored), or use a <https://docs.gradle.org/current/userguide/build_cache.html#sec:build_cache_configure_remote|remote build cache>.
As said above I don't really have that problem as we keep the whole gradle user home (dependencies, build cache, etc.) and the whole workspace (project's `build/` directories) so it works the same as on the developers machines.

Thank you Thomas! Will try out the above ways

Do you need to set the maxParallelForks as well?
```test.maxParallelForks = System.getProperty("maxTestParallelForks")?.toInt() ?: ((Runtime.getRuntime().availableProcessors() / 2).takeIf { it &gt; 0 } ?: 1)```
and when you run your build, you need `--parallel` or gradle.properties needs `org.gradle.parallel=true`

You can read more about these system properties here: <https://junit.org/junit5/docs/snapshot/user-guide/#writing-tests-parallel-execution>

Sure, but you probably need to see how Gradle gets to that point: <https://docs.gradle.org/current/userguide/java_testing.html#sec:test_execution>

You're applying the system properties to the `test` suite, not the `smokeTest` one.

Correct. This solved my issue. Thank you so much Thomas for the reply.

basically our library has a dependency with variants for desktop and android JVM.

If a user consumes our library and picks up this transitive dependency is there any way for them to choose the variant and have all further down stream uses not enter variant selection

More information here: <https://github.com/awslabs/aws-sdk-kotlin/issues/539#issuecomment-1051309004>

I answered in the mentioned issue.
But for reference here too.
If you configure attributes on configurations, then this is just a local thing.
If you want it published and used by downstream projects, define the attributes on the dependency.
In your case I'd say the proper solution is to define a constraint for that transitive dependency that defines the attribute as shown in the diff I attached to that issue.

<@URQ797WRW> Thank you! I learn something new about gradle everyday it seems.

I guess you just add an `Add-Opens` manifest entry

Copy it there or actually create it there right-away?
Shouldn't need more than `buildDir = rootProject.file("${rootBuildDir}/${name}")`.
Besides that it of course will confuse anyone new to the project. :smile:

Thanks - am I right that your solution will flatten the structure? (Which might not be the end of the world.)

An existing plugin to copy them there is what I'm looking for, yes. I'd rather not mess with the conventions if I can avoid it.

Flatten in what sense?
It would instead of `my-project/subproject1/build/foo/bar` right away create `my-project/build/subproject1/foo/bar`

I think it would produce
```my-project/
└── build/
    ├── subproject1/
    ├── subproject2/
    └── subproject3/```
rather than
```my-project/
└── build/
    ├── subproject1/
    └── project-collection/
        ├── subproject2/
        └── subproject3/```

Ah, yes, that's right.
But that should easily be adaptable.
Something like `buildDir = rootProject.file("${rootBuildDir}/${path.replace(":", "/")}")` or similar.
But as you want a copy it is anyway not what you want.

Thanks, this has given me a good enough approximation:
```val rootBuildDir = buildDir

subprojects {

  val relativeProjectPath = rootProject.projectDir.toPath().relativize(this.projectDir.toPath())
  buildDir = rootProject.file("${rootBuildDir}/$relativeProjectPath")
}```

<https://docs.gradle.org/7.0/userguide/declaring_dependencies.html#sec:type-safe-project-accessors>

`enableFeaturePreview("TYPESAFE_PROJECT_ACCESSORS")` , not sure why it isnt stable yet

then `projects.someSubproject`  will be available

care about NOT using `implementation(project(projects.someSubproject))` and use directly `implementation(projects.subProject)`

<@U01DUV4Q882> do you know if there is a type unsafe API for this, akin to `VersionCatalogsExtension`? Though I guess one isn’t really needed - can just do `project(":subproject")` lol

I don't know, but I think you replied yourself hahah

Don't use `Copy` for that.
Just use a custom task, getting the file through class path and putting it in the destination.
For example like
```project.tasks.register("copyFile") {
    def outputFile = project.layout.buildDirectory.file("test.yaml")
    outputs.file(outputFile)
    doLast {
        outputFile.get().asFile &lt;&lt; getClass().getResource("lib/test.yaml").text
    }
}```

You mean it’s not seen as part of the `sourcesElements` outgoing variant?

```    {
      "name": "sourcesElements",
      "attributes": {
        "org.gradle.category": "documentation",
        "org.gradle.dependency.bundling": "external",
        "org.gradle.docstype": "sources",
        "org.gradle.usage": "java-runtime"
      },
      "files": [
        {
          "name": "testkit-truth-1.0-SNAPSHOT-sources.jar",
          "url": "testkit-truth-1.0-SNAPSHOT-sources.jar",
          "size": 3473,
          "sha512": "918e200ba2d912ae4ef5fe7587a244b65aac881cf9d376c7e72c3c004212ae7889929ac629cff2ef5d09c157eaf14bc14391c302a3d8545103e797894de5496c",
          "sha256": "0a02a5dcb8d96d1104eb421e7e4eefdd63b83b81cfce635660927830cddb61b6",
          "sha1": "0b628e4fb67a3d5350ec44dd8514592f9f9b0ff9",
          "md5": "a87112d777f22b9cc6e09c81cf0be356"
        }
      ]
    }```

`withJavadocJar()` calls into <https://github.com/gradle/gradle/blob/2cb4012f55636266937b504cf580c1077c070969/subprojects/plugins/src/main/java/org/gradle/api/plugins/internal/JvmPluginsHelper.java#L251> which is the the bit that adds it to the software component so that it gets published in the module metadata. not sure how to set that up with Dokka but maybe that'll point you in the right direction

you can add as it as an artifact to the outgoing of the `sourcesElements`

but consumers will probably want it identified explicitly… this might mean creating your own outgoing configuration and setting appropriate attributes so that it can be resolved when needed

something like this… since I’m deep in some of this goo atm:
```jvmPluginServices.createOutgoingElements("dokkaElements", builder -&gt; builder.withDescription("Dokka documentation stuff")
        .published().artifact(dokkaJarTask).providesAttributes(attributes -&gt; attributes.documentation("dokka").asJar()));```

Then in principle if someone had an incoming configuration that had a documentation attribute with the value “dokka” they would resolve to your variant. Of course that string value is completely non-standard… so no one will pull it unless told to.

one of these days (maybe soon?!), I'll feel like I actually understand variant-aware publishing and artifact resolution

ah, turns out I can just hijack `withJavadocJar()` with
```val javadocJar = tasks.named&lt;Jar&gt;("javadocJar") {
  from(dokkaJavadoc)
}```
and that seems to work

the .module file has it now, and the javadoc jar is published

Always when you use API directly on the “Publication” interface it is too late for the metadata (like the `artifact(...)` , don’t use that anymore!). So ideally you should only use `from(...)` there. (+ Maybe some POM-Metadata additions for informational data that can only be represented there - like `&lt;licenses&gt;` , `&lt;scm&gt;` etc. - if you need it.)

Jendrik, do you happen to have a recipe for such a thing? I know you maintain a repo with many recipes :slightly_smiling_face:

Well… now there is one. :slightly_smiling_face: Hope it helps: <https://github.com/jjohannes/gradle-demos/tree/main/extending-publications>

And the error:

```FAILURE: Build failed with an exception.

* Where:
Build file '/home/jbrein/data/customers/waitrose/repos/helpers/gradle/test/build.gradle' line: 21

* What went wrong:
A problem occurred evaluating root project 'test'.
&gt; Could not set unknown property 'greeter' for root project 'test' of type org.gradle.api.Project.

* Try:
&gt; Run with --stacktrace option to get the stack trace.
&gt; Run with --info or --debug option to get more log output.
&gt; Run with --scan to get full insights.

* Get more help at <https://help.gradle.org>

BUILD FAILED in 414ms```

You try to set a property of `project` that does not exist in line 21.
And even if that worked, for example because you declare it as local variable by prefixing it with `def` , then line 25 is just a no-op assigning a property to itself.

If you meant to assign a variable, then you're missing the `def` declaring it, but then this likely won't work as intended due to scoping:
```def greeterVar = 'test'

greeting {
    message = 'Hi'
    greeter = greeterVar // naming the variable 'greeter' would read the greeter property of the greeting extension, not the variable
}```

"naming the variable 'greeter' would read the greeter property of the greeting extension, not the variable"

Actually everything is a bit clearer when using Kotlin DSL, as it is type-safe then and with much better IDE support. At least if you use IntelliJ. :slightly_smiling_face:

I use VIM... but since I'm doing more gradle intensive I think I'll switch at some point

Ah, ok, with good-old vim the difference is much smaller of course. :smile:
But I'd still prefer Kotlin DSL as errors are much better and usually already at compile-time, not just at runtime.
Due to Groovys duck-typing it just cannot error out earlier as anything could be available at runtime.

Hey Chris,

There is no holistic description of these internals AFAIK.

I am however surprised that resolving one configuration locks another one … unless they have an `extendsFrom` relationship. In that case, it is true that resolving a configuration locks it and all the ones it extends from, transitively.

I believe the configurations are independent… but I’ll do some digging to be sure.

I’ll definitely hit you up if I can’t fix it though :smile:

okay - it looks like it’s happening (unless I’m doing something dumb) - consider this me hitting you up!

relevant (maybe) stack trace:
```        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.preventFromFurtherMutation(DefaultConfiguration.java:1070)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultLocalComponentMetadataBuilder.createConfiguration(DefaultLocalComponentMetadataBuilder.java:70)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultLocalComponentMetadataBuilder.addConfiguration(DefaultLocalComponentMetadataBuilder.java:44)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultRootComponentMetadataBuilder.addConfiguration(DefaultRootComponentMetadataBuilder.java:102)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultRootComponentMetadataBuilder.getRootComponentMetadata(DefaultRootComponentMetadataBuilder.java:96)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultRootComponentMetadataBuilder.lambda$buildRootComponentMetadata$0(DefaultRootComponentMetadataBuilder.java:86)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry$ProjectStateImpl.fromMutableState(DefaultProjectStateRegistry.java:352)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultRootComponentMetadataBuilder.buildRootComponentMetadata(DefaultRootComponentMetadataBuilder.java:84)
        at org.gradle.api.internal.artifacts.ivyservice.moduleconverter.DefaultRootComponentMetadataBuilder.toRootComponentMetaData(DefaultRootComponentMetadataBuilder.java:70)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.toRootComponentMetaData(DefaultConfiguration.java:1193)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver$DefaultResolveContextToComponentResolver.resolve(DefaultArtifactDependencyResolver.java:227)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolve(DependencyGraphBuilder.java:139)
        at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver.resolve(DefaultArtifactDependencyResolver.java:145)
        at org.gradle.api.internal.artifacts.ivyservice.DefaultConfigurationResolver.resolveGraph(DefaultConfigurationResolver.java:186)
        at org.gradle.api.internal.artifacts.ivyservice.ShortCircuitEmptyConfigurationResolver.resolveGraph(ShortCircuitEmptyConfigurationResolver.java:85)
        at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingConfigurationResolver.resolveGraph(ErrorHandlingConfigurationResolver.java:74)```

Minimal Reproducer:
```repositories {
    mavenCentral()
}

configurations {
    foo
    bar

    all {
        beforeLocking {
            logger.warn "Locking $it"
        }
    }
}

dependencies {
    foo 'org.slf4j:slf4j-api:1.7.25'
    bar 'org.slf4j:slf4j-api:1.7.30'
}

tasks.register('resolveFoo') {
    doFirst {
        configurations.foo.resolve().forEach(f -&gt; logger.warn("Resolved $f"))
    }
}```
outputs:
```$ ./gradlew resolveFoo

&gt; Task :resolveFoo
Locking configuration ':foo'
Locking configuration ':bar'
Resolved /Users/cdennis/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25/da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar

BUILD SUCCESSFUL in 542ms
1 actionable task: 1 executed```

What happens if you mark `foo` and `bar` as not consumable?
I will need to confirm later, but we might be indeed locking down all consumable configurations because we need the metadata for the current project, which is at the root of the resolution graph.

But I believe that limitation will be in your way since you wanted to change outgoing capabilities, so most likely on a consumable configuration

```canBeConsumed = false```
stil locks both configurations

right - I want the capabilities of the outgoing configurations to be derived from the incoming configuration. I’m trying to reduce redundant gradle in my shading plugin

The error claims that the plugin requires that Gradle is run at least with Java 9, but you are running it with Java 8. Can't you run it with Java 9+, especially in a Java 17 project?

Well I am running with J17 already, and the java toolchain is set to that version, too

Well, the error disagrees. :smile:
Can you maybe knit an MCVE?
Or try running with `--version` to see what it says.
Toolchain version should be irrelevant here I think.

A `Publication` "publishes" a `SoftwareComponent`. A `SoftwareComponent` is not necessarily bound to `Configuration`: it's an abstract way to define a deliverable. So you can see a `SoftwareComponent` as the model for "something built by Gradle".  There are different kinds of components, based on the different ecosystems. For the typical Java/JVM world, the easiest is to use `AdhocComponentWithVariants`, which provides a convenient way to map "outgoing" `Configuration`s to variants.

In order words, `SoftwareComponent` is the concept, there are specific implementations (for example for Gradle native (C/C++)), and a generic implementation, `AdhocComponentWithVariants`, which can be used to create custom publications: <https://docs.gradle.org/current/userguide/publishing_customization.html#sec:publishing-custom-components>

I understand that but I don't really see why the need for a different concept? C/C++ could also very well use `Configurations` that would be added to a `Publication` ?

In my mind:
• `Configuration` = list of artifacts + attributes
• `Publication` = list of artifacts + attributes + pom information. A `Publication` could potentially publish several `Configurations` (or no Configuration at all if artifacts are added manually)

`Configuration` is also "something build by Gradle", right ?

`Configuration` is too many things at the same time. Long term, this should go away IMO, because it's confusing. It's used for declaring dependencies, carrying artifacts, publications, representing a dependency graph, etc.

the reason C/C++ has its own model is because the software component idea was explored there first

the adhoc component thing is really a bridge to make it easier to use existing concepts to build components

Can I already replace my outgoing configurations with SoftwareComponents ?

no. I wouldn't do that. Outgoing configurations are used everywhere, especially for integration with composite builds.

also the APIs for building software components from scratch are IMHO cumbersome. It is, frankly, complicated.

Alright, I'll keep the middleman :thumbsup:

As far as I remember, the intention was at some point to make the “Component”-Concept stronger in a way that you can use it to directly declare what a project represents to the outside - what _consumable configurations_ are today. But there is also a lot of history of ideas that were neglected /changed. Thus right now, it is a bit confusing.

I think at the moment though “Publication” is the thing that is (almost) redundant. Ideally you would only have one Publication per project and that would be “from(components[“java”])“. Without anything else. So the Java plugin could declare do that for you and then you wouldn’t have to deal with “Publication” at all yourself. (The plugin-publish-plugin is actually doing that.)

The “Component” on the other hand lets you control which of your “consumable configurations” you want to publish. It’s also the base from which the Gradle Metadata (and POM) metadata is created. That’s why you should add a “consumable configuration” to the “java component” (instead of adding the artifacts to the Publication). Because then you do not only get the artifacts, but also the dependencies etc. (a full ‘Variant’).

It’s a bit related to that other Thread where I just shared this example:
<https://github.com/jjohannes/gradle-demos/tree/main/extending-publications>

*If* you know which APIs to use, it is really nice to see how well this all works. Well done Cédric :smile: :muscle: (as far as I remember a lot of this was your doing).

Or let me ask another way: is there a way to execute a given task on all dependants? (not buildDependants, but more something like `myCustomTask` on all Dependants)

Are you looking for something like
Project a has a dependency on project b, you want myCustomTask run on both projects? 

Yes, but I don't want to run this task on say Project C, not related to these 2 connected projects

I'm able to do all this manually by looping on each subproject, their configurations and their dependencies, and assemble another graph, but it's not clean and it's very complex to cover all cases...

We used to do something like this a long while back on our Android project, but removed it. Our use case was:
We have project A and we wanted a single command to run lint and unit test and any upstream module.

The code below is how we resolved the upstream and got a certain task to pull into the command. It is rather old as we altered our CI process so lint + unit test is simply. `./gradlew lintDebug testDebugUnitTest --parallel --continue` so we can run each module in. parallel and couldn’t find a way to do it with our single command. This code may not compile/work properly with current versions as this is rather old code. (GlobalConfig was a simple utility we wrote to look at the project and. return the build variant in the format needed)

```class ModuleDependencyResolver {
  static def getDependentModulesTasks(Project project, String variant, String prefix) {
    ArrayList&lt;String&gt; tasks = new ArrayList&lt;&gt;()

    project.configurations.each { configuration -&gt;
      configuration.getAllDependencies().each { upstreamDep -&gt;
        if (upstreamDep.hasProperty('dependencyProject')) {
          Project depProject = project.rootProject.findProject(":${upstreamDep.dependencyProject.getName()}")
          project.evaluationDependsOn(depProject.path)

          if (depProject.hasProperty('android')) {
            String requiredVariant = GlobalConfig.getFormattedBuildVariants(depProject).find { it -&gt;
              return variant.toLowerCase().contains(it.toLowerCase())
            }

            requiredVariant = requiredVariant == null ? depProject.android.defaultPublishConfig.capitalize() : requiredVariant
            tasks.add(depProject.getTasksByName("${prefix}${requiredVariant.capitalize()}WithDeps", false))
          }
        }
      }
    }
    return tasks
  }
}```
The above groovy code would get invoked in this sort of groovy script:
```  private static def configureUnitTestTask(Project project, String formattedBuildVariant) {
    project.task("unitTest${formattedBuildVariant}WithDeps", type: DefaultTask, dependsOn: "test${formattedBuildVariant}UnitTest") {
      group "Test"
      description "Runs Spec for this module and all its upstream deps"
      ModuleDependencyResolver.getDependentModulesTasks(it.project, formattedBuildVariant, 'unitTest').each { task -&gt;
        dependsOn task
      }
      ModuleDependencyResolver.getDependentModulesTasks(it.project, formattedBuildVariant, 'lint').each { task -&gt;
        dependsOn task
        mustRunAfter task
      }
      project.getTasksByName("check${formattedBuildVariant}Coverage", false).each { task -&gt;
        dependsOn "lint${formattedBuildVariant}"
        finalizedBy task
      }
    }
  }```

my only idea right now is that the command line is huge. This is a CI job and we collect many dozens of fully qualified paths to test tasks and invoke them all at once. Could it be too long? Is there a max length?

Another fact: this is happening with AGP 7.1, but wasn't happening with 7.0

I usually see these error when the test executor crashed on CI. The configuration hint is misleading in those scenario. I get a lot more of those on Windows than macOS/Linux/FreeBSD. I haven’t looked too deeply in what is causing the crash but most likely a memory leak/out of memory issue causing the test executor to cash. That is just an assumption as it’s flaky :disappointed: I can’t say for AGP but if the situation is flaky, it may be a similar issue as I’m experiencing. In some of my research, some systems will abort non-essential/memory hungry process under memory pressure. That could also be a reason.

The next step in debugging this, if the situation is flaky (hinting toward memory issues), would be to look at the total memory consumption on the CI agent (not Gradle) to see if the system is under pressure which could result in a global out of memory scenario for all process. Of course, there should be swapping happening but some CI setup recommend disabling swapping to avoid crawling the system to a halt.

exit code 134 = killed by signal 6 = SIGABRT = JVM itself is in a bad state. if it left behind `hs_err_pid*.log` file, that may provide hints as to what went wrong

I ran the same job locally and it succeeded. Unfortunately, the build scan was too large to even finish uploading, apparently

have you looked at <https://docs.gradle.org/current/userguide/gradle_daemon.html>. ?

I think this is referring to Java compilation. The wording is quite confusing. There are three different ways how Gradle can run Java compilation. I couldn’t find good docs on this. So this what it roughly is:

1. If you configure nothing, Java compilation will run *in the Gradle daemon process* using the same JVM Gradle runs on to compile the code (using Java’s Compiler APIs).
2. You can also run the compilation in a *compiler daemon* (or *compiler worker*, how it would probably be called today to be in line with Gradle’ general *Worker API*). This means that a separate JVM process is started in which the compilation is performed. But the process stays alive and communicates with the Gradle daemon. So it is reused when one compile task finished and another needs a compile daemon/worker again. It’s not super clear when this mode is chosen (I always forget). If you use the “toolchains” feature, and configure a JVM different from the one Gradle is running on, it will definitely use this mode.  If you set “options.isFork = true” on the compile task, you will also trigger this mode to be used. You can then control details of the “compiler daemon” JVM (for example “options.forkOptions.memoryMaximumSize”).
3. A third alternative is to configure a `javac`  executable (or a Java Home where the executable is in) in the “Fork Options” of the compile task. Then Gradle directly starts a “javac” process for each compilation. This is usually more expensive.
