From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

<https://github.com/gradle/gradle/issues/15021|https://github.com/gradle/gradle/issues/15021>

For Groovy you would add a closure under the name `flattenJson` as `extra` property.

For both to work seamlessly and without import, you would add an extension to the tasks that has that method, so you have one level more nesting, or qualification with a dot, but it works seamlessly.

At least if accessors are generated for task extensions, which I'm not 100% sure

extensions apply to individual objects, not the whole class, right? so I'd need to add it to all `AbstractCopyTask` s with a withType then

I'll try that and see how it goes, thanks

nope, no accessors generated after
```tasks {
    withType&lt;AbstractCopyTask&gt; {
        extensions.add(typeOf&lt;JsonFiltersExtension&gt;(), "filters", JsonFiltersExtension(this))
    }
}```
the extension got added though, confirmed with
```tasks.processResources {
    println(extensions.get("filters"))
}```

Likely I'll fall back to import for kotlin and extra for groovy then. Kotlin DSL should have good enough IDE support to figure it out

One very dirty thing your could do, but I wouldn't recommend it, is to put the extension method into the `org.gradle.dsl` package or what it was. Then you don't need an import. But imho it is pretty bad to harvest foreign packages.

decided to move my extension to CopySpec so it can be used as e.g.
```from("abc")
into("def")
processJson {
    outFormat = JSON5
    pipeline {
        flatten()
        mapStrings { it.replace("foo", "bar") }
    }
}```
what's the correct way to add stuff onto MetaClasses from java/kotlin? I can make a groovy method that just does `CopySpec.metaClass.processJson = { ... }`, but nothing else in my plugin is groovy and I'd rather not add it just for that

also having trouble configuring a precompiled script plugin to publish to mavenLocal

Probably something like this:
```val generateResources by tasks.registering {
    inputs.files(sourceSets.main.map { it.allJava })
    outputs.dir(layout.buildDirectory.dir("my-generated-resources"))
}

sourceSets {
    main {
        resources {
            srcDir(generateResources)
        }
    }
}```

I tried doing something similar -&gt; I got a `SourceTask` and was doing the following

```val sourceSetContainer = project.findProperty("sourceSets") as? SourceSetContainer
val mainSourceSet = sourceSetContainer?.findByName("main")
myTask.source(mainSourceSet?.output)

mainSourceSet?.resources {
    it.srcDir(myTask.outputDirectory)
}```

but gradle was complaining about cyclic dependency on sources :disappointed:

```Circular dependency between the following tasks:
:ktor-graalvm-server:classes
\--- :ktor-graalvm-server:processResources
     \--- :ktor-graalvm-server:graphQLGraalVmMetadata
          \--- :ktor-graalvm-server:classes (*)```

You are not depending on the sources, you are depending on the outputs.
And the outputs do not contain the sources, but the compiled classes and also the resources.
So you have the cyclic dependency.

Even `sourceSet.allSources` contains the resources, as they are also sources in that regard, that's why my example uses `allJava`

tried using all java and got the same cyclic warning

I can just say that my example works without, I tried before sending.
Can you give a full <https://stackoverflow.com/help/mcve|MCVE> where it does not work?

here's my current code -&gt; maybe you can see something odd there

<https://github.com/dariuszkuc/graphql-kotlin/blob/graal_wip/plugins/graphql-kotlin-gradle-plugin/src/main/kotlin/com/expediagroup/graphql/plugin/gradle/GraphQLGradlePlugin.kt#L231-L235|https://github.com/dariuszkuc/graphql-kotlin/blob/graal_wip/plugins/graphql-kotlin[…]n/com/expediagroup/graphql/plugin/gradle/GraphQLGradlePlugin.kt>

wondering whether my issue is because of the usage of source task vs regular task with files used as input

Actually, `project.findProperty("sourceSets")` per se is fishy.
But regarding the circular reference I don't see the issue right from looking.
But I don't have the time for an in-depth review of that.

`project.extensions.findByType&lt;SourceSetContainer&gt;()`

Or even with `getByType` as you require it to be there

found the culprit -&gt;
```myTask.projectClasspath.setFrom(mainSourceSet.runtimeClasspath)```
I'm setting up the task classpath to use project classpath so I can auto-generate stuff based on available code

so when I attempt to add back to the main resource set it blows up as I already have dependency on runtime classpath

I don't think so. And even if, it would probabyl be bad practice.

possibly by using an internal Gradle API <https://github.com/gradle/gradle/blob/463a0116adbeaf058ebb80e6aa81523de48f4abc/subprojects/core/src/main/java/org/gradle/api/internal/FeaturePreviews.java>

The ‘internal’ is just a convention, so you can use it without restriction, but it would be more risky.

As an alternative, you could provide two plugins? One with the typesafe-project accessors, another without?

The included build is usually executed before the main build, isn't it?
So the `enableFeaturePreview` calls would not be called yet.
You could maybe delay the action using `gradle.parent.settingsEvaluated { ... ]`, and there get the `FeatureFlags` internal service that holds the state for the feature flags, but even if it works - and that is a big if -, I really wouldn't do it.

do you mean that it wouldn’t be possible to use the generated project accessors inside a project-plugin <@URQ797WRW>? Because yes, I think that’s right :) But maybe that’s not the behaviour changes <@U031904KXEY> wants to make :woman-shrugging:

No, that's not what I meant, though I also think that is true.

Probably more an XY problem, so what is your actual intent / use-case <@U031904KXEY>?

My use-case is modifying this plugin to support it: <https://github.com/autonomousapps/dependency-analysis-android-gradle-plugin>

This plugin has a task, `fixDependencies` , that rewrites your gradle dependencies based on the `advice` for the correct dependency configuration. For example, if the plugin detects that
```dependencies {
    implementation projects.myLibrary.domain
}```
should use `api`  configuration instead, it will suggest it. HOWEVER, it rewrites it as:
```dependencies {
    api project(':my-library:domain')
}```
instead of the type-safe way..

Thus, I'm trying to modify the plugin, and attempting to make it smart, by detecting that the client has `TYPESAFE_PROJECT_ACCESSORS`  enabled so that our output can be:
```dependencies {
    api projects.myLibrary.domain
}```


Ah, right, I think I totally got you wrong.
Forget almost all I said so far.

So yeah, you can probably get the `FeatureFlags` service and check.
But it is an internal service, so there is no compatibility guarantee wahtsoever.

Or you just check where it is called textually whether the project accessor is used or not

yeah, i've debugged the objects, but I can't seem to get a reference to the active `FeatureFlags`

what do you mean by:
&gt; where it is called textually

`FeaturePreviews.java`  has a method `getActiveFeatures()`  but doesn't have any usages. Gradle version `7.6`

Besides that it is also an internal class and already different in 8.0, active features are not the same as enabled features.

Active features are the features that can be enabled and have an effect, opposed to deprecated features.

instead of trying to determine if the feature flag is enabled/disabled, perhaps it might be a little better to search to see if the accessors generated source code exists.

They’re generated in the project’s directory, `.gradle/&lt;version&gt;/dependencies-accessors`, with Javadoc, so you could try searching to see if there’s any file that contains

```Creates a project dependency on the project at path ":my-subproject"```
This approach is also hacky and fragile!

I don't think this is a good idea, as there might be old accessors when the feature is switched off again already.

As I said, safest is to probably just look visually and use what you find.
It is the fix... task, so you do String manipulation anyway. Just that the feature is enabled also does not necessarily mean it is used consistently everywhere.

the best indicator, without relying on internal behaviours, would be to let the user explicitly opt-in - so add a `enableTypesafeProjectAccessorReplacement: Property&lt;Boolean&gt;` on the plugin’s extension, so that it can be configured manually.

You’d probably want to do that anyway, because maybe someone has the typesafe project accessors enabled but doesn’t want `fixDependencies` to use them (like maybe there’s a bug in the generator, or maybe they want to manually convert them).

And then maybe one day the typesafe project accessors will be enabled by default, and then trying to determine if the features is enabled or not doesn’t matter

thanks for the feedback! a new property on the plugin's extension is probably the best approach here :pray:

```def ext = extensions.create("micronaut", MicronautExtension) as ExtensionAware
def property = objects.property(String)
def supertype = property.getClass().genericInterfaces[0]
ext.extensions.add(TypeOf.typeOf(supertype), "fooVersion", property)

interface MicronautExtension {
    Property&lt;String&gt; getMicronautVersion()
}```

basically, I would like to extend the `micronaut { ... }` extension with dynamic properties, in a Kotlin DSL compatible way, so that a user can have completion and write:
```micronaut {
    fooVersion.set("666")
}```

the code above works, but it looks quite fragile, and I'd like to make sure I'm not missing another more obvious way to do this...

Didn't try, but how about
```TypeOf.parameterizedTypeOf(TypeOf.typeOf(Property), TypeOf.typeOf(String))```
?

it says that `Property` is not a parameterized type

Strange.
Btw. does for you the property has the right type downstream?
For me it shows as `Property&lt;Any&gt;`

I tried with your code ported to Kotlin though as that was what I had at hand

no, it works with string if I use:
```def superclass = property.getClass().genericInterfaces[0]
def type = TypeOf.parameterizedTypeOf(
        TypeOf.typeOf(superclass),
        TypeOf.typeOf(String)
)
ext.extensions.add(type, "fooVersion", property)```

```ext.extensions.add(new TypeOf&lt;Property&lt;String&gt;&gt;() {}, "fooVersion", property)```
is the magic.
Peeked at Gradle code. :slightly_smiling_face:

cool, thanks. Wondering if there's something which doesn't involve `TypeOf` as I'm not even sure it's considered public API

<https://docs.gradle.org/current/userguide/designing_gradle_plugins.html#restricting_the_plugin_implementation_to_gradle_s_public_api>
&gt; How do you know if a class is part of the public API? If you can find the class referenced in the <https://docs.gradle.org/current/dsl|DSL guide> or the <https://docs.gradle.org/current/javadoc|Javadocs> then you can safely assume that it is public. In the future, we are planning to clearly separate public from internal API which will allow end users to declare the relevant dependency in the build script.

<https://docs.gradle.org/current/javadoc/org/gradle/api/reflect/TypeOf.html>

I've learnt to not trust the Gradle docs :)

Yeah, well, more I cannot provide :slightly_smiling_face:

ah dang, that's not nice, if I use the assignment operator in the Groovy DSL, it fails with:
```There's an extension registered with name 'httpNettyVersion'. You should not reassign it via a property setter.```

Can't you namespace these things inside `micronaut {}`?

that's what I do, but the list is dynamic

It may be that the Groovy assignment operator is no handling the fact that the extension itself is of type `Property`.

I meant add another level inside `micronaut {}`.
```micronaut {
    httpNetty { version = "" }
    httpNetty.version = ""
}```
It's less nice but might be more future proof if you need to add more than the version later.

yeah I can do that too, but that's not super nice

especially because we already have `micronaut { version = ... }` for the main version

Then open a PR to Gradle to improve the assignment operator :smile:

so it's not super consistent to nest everything (in addition to being verbose)

if you ever need to add more than `version` you'll be happy to have the nesting

`s/PR/issue/` works too
I wonder how the upcoming Kotlin DSL assignment operator for Property behave in that case. If it works as you'd expect then such an issue might get prioritized given this is currently in focus to make both DSLs behave similarly with assignments.

the new assignment operator is in 8.1, right?

yep <https://docs.gradle.org/release-candidate/release-notes.html#kotlin-dsl>

it's still behind an opt-in because of some IDE issues that will be solved soon

nope: e: file:///tmp/grotest/build.gradle.kts:6:4: Val cannot be reassigned

I'll call it a day :slightly_smiling_face:

oh wait it does work with Kotlin DSL, I had used the wrong type

I hear an issue report rushing in :smile:

that's how I found out, I was already writing a report and checking my code :slightly_smiling_face:

<https://github.com/gradle/gradle/issues/24719>

Cool, I was expecting this to work in kotlin

I’m really curious about you’ve got `interface MicronautExtension` and you cast it with `as ExtensionAware`, which I’m really surprised ‘just works’.

Maybe that’s some Groovy magic? Doesn’t `MicronautExtension` need to implement the `ExtensionAware` interface?

no, any object created via Gradle's object factory is `ExtensionAware`

the `as ExtensionAware` was just a hint for the IDE to know about this :slightly_smiling_face:

You can also explicitely declare that your type implements `ExtensionAware` so it is discoverable and for the IDE to not require hints. Gradle will provide the implementation.

Wasn't it somewhat awkward to have it explicit which was the reasons so few Gradle built-in types declare it?
Or did that change?
Or do I confuse something.

The reason for built-in types not having it was mostly history and the fact that dynamic Groovy doesn't care much.

<https://github.com/gradle/gradle/issues/7548> is probably what I have in mind, where you say that it is not simple to implement `ExtensionAware` :slightly_smiling_face:

```&gt; Task :publishPlugins
Publishing plugin org.projectnessie version 0.30.0
Thank you. Your new plugin org.projectnessie has been submitted for approval by Gradle engineers. The request should be processed within the next few days, at which point you will be contacted via email.
Publishing artifact ...```

```class MyPluginTest {

    @TempDir
    lateinit var testProjectDirectory: File
    private lateinit var buildFile: File

    @BeforeEach
    fun setup() {
        buildFile = File(testProjectDirectory, "build.gradle.kts")
        buildFile.appendText("""
            plugins {
                id("com.my.plugin.id")
            }
        """.trimIndent())
    }

    @Test
    fun testGradleRunner() {
        val subject = GradleRunner.create()
            .withProjectDir(testProjectDirectory)
            .withArguments("myTask")

        val result = subject.build()

        assertEquals(TaskOutcome.SUCCESS, result.task(":myTask")?.outcome)
    }

    @Test
    fun testProjectBuilder() {
        val project = ProjectBuilder.builder()
            .withProjectDir(testProjectDirectory)
            .build()
        project.pluginManager.apply("com.my.plugin.id")

        assertNotNull(project.tasks.getByName("myTask"))
    }
}```

Error is:
```Plugin [id: 'com.my.plugin.id'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (plugin dependency must include a version number for this source)```

You don't have any `withPluginClasspath()`, that's probably the problem here: the plugin is available in the classpath of your test so `ProjectBuilder` can find it, but TestKit is isolated so you have to somehow give it your plugin's binary.
<https://docs.gradle.org/current/userguide/testing_gradle_plugins.html#functional-tests>

Oh, I didn't realize that was the purpose of that statement. Thanks a lot!

<https://gradle-community.slack.com/archives/CAHSN3LDN/p1677073056461669>

looks like just marking it with `@Optional` works

You mean that the thread you linked to is not correct anymore?

It is correct. I suppose you just don't have to do that `@Internal` dance anymore

I don't know what you mean, nothing changed.
```abstract class MyTask : DefaultTask() {
    @get:Optional
    @get:InputFile
    abstract val inputFile: RegularFileProperty

    @TaskAction
    fun doIt() {
        println("${inputFile.get().asFile.absolutePath} : ${inputFile.get().asFile.isFile}")
    }
}

tasks.register&lt;MyTask&gt;("foo") {
    inputFile.set(layout.buildDirectory.file("not-there"))
}```
fails with "Type 'Build_gradle.MyTask' property 'inputFile' specifies file 'D:\Sourcecode\other\showcase\build\not-there' which doesn't exist." while
```abstract class MyTask : DefaultTask() {
    @get:Internal
    abstract val inputFile: RegularFileProperty

    @get:Optional
    @get:InputFile
    val optionalInputFile = inputFile.map {
        if (it.asFile.isFile) it else null
    }

    @TaskAction
    fun doIt() {
        println("${inputFile.get().asFile.absolutePath} : ${inputFile.get().asFile.isFile}")
    }
}

tasks.register&lt;MyTask&gt;("foo") {
    inputFile.set(layout.buildDirectory.file("not-there"))
}```
still works as expected.

Maybe you were fooled by the example that did not have a task action, was printed as up-to-date and did not throw the error then.

Whether that is correct behavior or a bug is another topic. :smile:

I marked the input as `Optional` and configured it conditionally, if the file exists then and only then configure it. It worked without having to add anither extra `optionalInput` field

Sure, if at configuration time this is determinable already, that's fine. It becomes problematic if the build would maybe produce the file or not.

It seems there was an issue accessing a file in outputDirectory and you got exception:
```Failed to create MD5 hash for file content.```
Probably using --stacktrace will tell you more.

My guess is, that some other process opened a file in outputDirectory and Gradle can’t access it. Windows locks file in such situations while Linux and Mac do not.

FWIW the build ran “fine” with Java 11 + Gradle 7.6. Just upgraded to Java 11 + Gradle 8.1.1 and found that issue. Running with `-S` now to see if there’s more detail

I see error is not best and doesn’t tell you what is the problematic file. Is it possible that logger keeps some file open?

It is indeed a file locking error
```   Caused by: java.io.IOException: The process cannot access the file because another process has locked a portion of the file
  	at java.base/java.io.FileInputStream.readBytes(Native Method)
  	at java.base/java.io.FileInputStream.read(FileInputStream.java:254)
  	at org.gradle.internal.hash.DefaultStreamHasher.doHash(DefaultStreamHasher.java:52)
  	at org.gradle.internal.hash.DefaultStreamHasher.hash(DefaultStreamHasher.java:35)
  	... 133 more```

the `outputDirectory` points to the current basedir where the project resides

full stacktrace at <https://github.com/jreleaser/jreleaser/actions/runs/4784551780/jobs/8506192968> line 5249 if it helps

What’s odd is that Gradle `7.6` did not show this behavior but `8.1.1` does

I don’t think there was any difference in snapshotting between 7.6 and 8.1.1.
But since you set project dir as an output, Gradle might try to snapshots .gradle directory (or any other directory) inside a project, and there might be something that causes a problem with 8.1.1.

So in the end the suggestion is correct, since you set a project dir as an output, then adding `doNotTrackState()`  or `@UntrackedTask` is a good option to avoid troubles. In the end I doubt your task will be up-to-date ever, since project dir is changed all the time

Best is not to do it afair. Can't you just disable it?

You can, but this still registers the task with this (now reserved) name, shows up the task in the task graph, and shows the tasks in the IDE. Ideally, I want to rename the task. Or simple remove the old task completely and register a new task instead.

If the name is reserved, would removing it even work? Doesn't it fail on registering it already?

If it was added already, there could be reactions to it, like a `tasks.all` that wires it somewhere, or the one originally adding the task wiring it somewhere else too and so on.

Also, iirc you cannot "lazily" remove the task, but need to realize the task to remove it.

You are right, removing a task is not supported anymore since Gradle 6.

*Use case #1 - Load dependency repository credentials*. In our `settings.gradle`  we want to load the credentials for our private maven artifact repository through 1Password’s CLI tool.
For this use case I declared a simple Kotlin function `fun read(opReference : String) : String` that will execute an op command using `Runtime.getRuntime().exec("op read $opReference")` , it will read the standard output of that command and return it as a string. It will throw a Gradle exception if the command fails (error output not empty), which will be the case e.g. if the `op` CLI is not installed or authorized.
This function is made available from a Settings scope through `settings._extra_.set("onePassword", ::read)` when applying the plugin.

*Use case #2 - Inject `*.properties` files containing 1Password references.*  We want to declare properties files with references to 1Password instead of the actual values, and the plugin should inject these references and make them available to Project scopes (without actually modifying the underlying file, which is checked into version control.
For this use case I declared an extension that has a function `fun loadProperties(propertiesFilePath : String)` that will read the file, inject the actual secrets from 1Password and make them available to Projects through a extension. For example, `keystore.properties` would be injected and provided as `keystore` extension, and a key-value like `GOOGLE_MAPS_API_KEY=<op://Vault/Google> Maps/API Key` would be made available to the project as `keystore["GOOGLE_MAPS_API_KEY"]`.
The `loadProperties` function is then called through an extension `onePassword { loadProperties(keystore.properties) }`  in Project’s `build.gradle`

While this approach works, I’m pretty sure it doesn’t fit the Gradle model. So I have some questions:
1. Should I be using Tasks instead of functions to provide this functionality as a plugin? 
    a. If so, considering all this needs to happen during the configuration phase, and not execution, should it be using a `TaskExecutionGraphAware` Task for that?
2. The `op` CLI tool is fast, but does slow don builds, specially considering this is done during the config phase. Can I add cache if I’m not using Tasks? Are the functions deterministic enough to cache them (authorization might change over time, for example) 


I don’t think that tasks would be a good fit for fetching credentials and providing them for other tasks. Basically the only way for a task to get some variables and pass them to another task is to write them to a file - which is not what you want for secrets!

if you want to run an external process and capture the result, the Gradle-idiomatic way of doing that is with a custom <https://docs.gradle.org/current/javadoc/org/gradle/api/provider/ValueSource.html|ValueSource>. This then plays nicely with Gradle’s <https://docs.gradle.org/current/userguide/lazy_configuration.html|lazy configuration>, so a value can be re-used (improving performance) and execution is avoided if the value isn’t required during execution.

Rather than using extra properties (like `settings.extra`), I’d prefer using a <https://docs.gradle.org/current/userguide/implementing_gradle_plugins.html#modeling_dsl_like_apis|custom extension> for the settings plugin. This would give a more stable base for accessing properties and helper utilities in build scripts.

You _might_ want to consider creating a <https://docs.gradle.org/current/userguide/build_services.html|custom build service> that would be a wrapper for the 1Password CLI, but it depends if you want to avoid concurrency problems - probably that’s not an issue?

Thank you very much for the input <@U02KX5B7EVC>.
I’ll take a look at ValueSource, it sounds like the right tool for the job.
I will also avoid using extras directly. It always felt a bit hacky.
I don’t think I’ll run into any concurrency problems since I’m only reading from the external service. I don’t know if I’ll want to use a custom build service, but it’s nice to know about :+1:

there’s also <https://docs.gradle.org/current/javadoc/org/gradle/api/provider/ProviderFactory.html#exec-org.gradle.api.Action-|ProviderFactory.exec()>, which is a bit more simple than value source

I’ve struggled with this recently too. I couldn’t figure it out - I think it was because the included-build project wasn’t 100% Gradle idiomatic.

The best workaround I could find is to publish the plugin to a local directory (MavenLocal will work too), and use <https://docs.gradle.org/current/userguide/declaring_repositories.html#sec:repository-content-filtering|a filter> so Gradle would only look in that repo for my plugin.

```// settings.gradle.kts
pluginManagement {
  repositories {
    mavenCentral()
    gradlePluginPortal()
    mavenLocal {
      mavenContent {
        includeGroup("package.of.project")
        includeGroup("my.plugin.id")
      }
    }
  }
}```
I had to include the group of the <https://docs.gradle.org/current/userguide/plugins.html#sec:plugin_markers|plugin marker artifact> (which is just the plugin ID)

Thanks, Adam! I ended up with a local Maven repo, too. I didn’t apply a filter, though, that looks like a nice approach. But I was hopping that `includeBuild` could still somehow work.

what happens if you try removing `classpath("my.settings.plugin:gradle-integration:1.0-SNAPSHOT")`, and using `pluginManagement { includeBuild(...) }`?

Hm… If I do what you suggested and additionally switch to the new syntax (`plugins {id("...")}`), it seems to partly work. At least it looks like my plugin is being properly loaded! Now, it seems to lack dependencies. My original example is oversimplified. In fact, I have not only the plugin but also related libraries under `buildscript.dependencies`.

```pluginManagement {
    includeBuild("gradle-integration")
}
buildscript {
    repositories {
        ...
    }
    dependencies {
        classpath("my.settings.plugin:gradle-integration:1.0-SNAPSHOT")
        classpath("my.library:dependency:1.0-SNAPSHOT")
    }
}
plugins {
    id("my.plugin.group")
}```

Yes, you need to do `includeBuild` within `pluginManagement` as it is treated specially, exactly for the use-case to provide settings plugins.

And yes, you need to use the `plugins` block to apply the plugin and not the legacy way, so that Gradle will find the plugin in the included build.

Declaring dependencies of your plugin manually should not be necessary, as long as all dependencies are either found in the included builds, or in the declared plugin repositories.

<@URQ797WRW> Thanks! As to dependencies, some of them can be changed dynamically that’s why they are not in the dependency graph of the plugin. Is there a way to substitute them via `includeBuild` as well?

I see, if they are "optional" dependencies, you need to declare them of course.
Well, actually it would then probably be better to use a feature variant.
But both are not really nice with plugins.
It would probably be most convenient to have different plugin IDs for the different dependency-sets / feature variants.

<https://github.com/gradle/gradle/issues/12388> is supposed to be fixed in 8.0

I still have this problem in 8.1.1. The workaround I used is this:

```provider.flatMap {
  providers.provider {
    if(someConditionDependingOnIt) it.convert() else null
  }
}```

You do not, your IDE is lying at you :smile:

```println(
    tasks.compileJava.map {
        if (Random.nextBoolean()) it.toString() else null
    }.orNull
)```
works perfectly fine.
Actually IJ shows it as type-mismatch error.
The Kotlin compiler shows it as type-mismatch warning.
But it works as expected.
Not sure where this should be reported to. :smile:

But probably the KT and KTIJ projects on <http://youtrack.jetbrains.com|youtrack.jetbrains.com>

ahh okay, I see it does compile and run, but with a warning.

```&gt; Configure project :
w: file:///projects/adam/x/build.gradle.kts:15:5: Type mismatch: inferred type is String? but String was expected
null```

&gt; But probably the KT and KTIJ projects on <http://youtrack.jetbrains.com|youtrack.jetbrains.com>
Kotlin compiler does his work fine here, it’s because the whole Gradle’s package is marked with <https://github.com/gradle/gradle/blob/a0fc74acf9b31ac4282d8e763d2c8b49225633cd/subprojects/base-annotations/src/main/java/org/gradle/api/NonNullApi.java|@NonNullApi>, and Gradle <https://github.com/gradle/gradle/blob/f56bc19f6ea2c430ab44b038658c4c110ffd2079/subprojects/kotlin-dsl/src/main/kotlin/org/gradle/kotlin/dsl/KotlinDslStandaloneScriptCompilationConfiguration.kt#L40|specifies> that it enables strict jsr305 mode but on the other side <https://github.com/gradle/gradle/blob/f56bc19f6ea2c430ab44b038658c4c110ffd2079/subprojects/kotlin-dsl/src/main/kotlin/org/gradle/kotlin/dsl/KotlinDslStandaloneScriptCompilationConfiguration.kt#L42|disables> strict type enhancements via a highly experimental flag.
Worth filing issues to the KTIJ and to the Gradle trackers.

We're aware of issues with this, currently this is in a transitional stage because the rules Kotlin used for nullability were still changing until the jspecify rules were implemented in ~1.8. So our current setup does not quite work perfectly with the new rules, but it is intended to change in the near future, hopefully with the adoption of jspecify annotations instead of jsr305.

see also: <https://github.com/gradle/gradle/issues/24767>

Afair there are some defined start points like the `Project` and the `DependencyContainer` (or how it is called). From there all added extensions and all their extensions transitively get accessors generated. But it does not work for arbitrary types, probably because Gradle would need to investigate the whole model for extensions and that would probably potentially need too much time.

I’m always really confused by how to generate accessors for DomainObjectContainers too. I couldn’t get it to work for consistently. It must work for 3rd party plugins somehow.

For example, if I apply the Kotlin Multiplatform plugin in a convention plugin (that defines a bunch of targets/custom source sets) then I can use the generated accessors in the build script

```plugins {
  buildsrc.conventions.`kotlin-multiplatform-native`
}

kotlin.sourceSets {
  // generated accessors
  commonMain {}
  nativeMain {}
  windowsMain {}
  linuxX64Main {}
} ```

document: <https://docs.gradle.org/current/userguide/kotlin_dsl.html#kotdsl:accessor_applicability|https://docs.gradle.org/current/userguide/kotlin_dsl.html#kotdsl:accessor_applicability>

do you have any examples of `DomainObjectContainer` that you expect generated accessors for but aren't `NamedDomainObjectContainer`? without the named part, I'm not sure what you'd expect...

it’s not a simple example, but here’s one <https://github.com/adamko-dev/dokkatoo/blob/cefd52c39b646cd85a132cd84632af53cbec7dd7/examples/gradle-example/dokkatoo/build.gradle.kts#L10>

Dokkatoo <https://github.com/adamko-dev/dokkatoo/blob/cefd52c39b646cd85a132cd84632af53cbec7dd7/modules/dokkatoo-plugin/src/main/kotlin/adapters/DokkatooKotlinAdapter.kt#L54-L61|reacts to the Kotlin plugin>, and for each KotlinSourceSet it will register a DokkatooSourceSet with the same name

<https://github.com/adamko-dev/dokkatoo/blob/cefd52c39b646cd85a132cd84632af53cbec7dd7/modules/dokkatoo-plugin/src/main/kotlin/adapters/DokkatooKotlinAdapter.kt#L98-L114>

I would really like it if I could use

```dokkatoo {
  dokkatooSourceSets.main { }
}```
It’s the same for the <https://github.com/adamko-dev/dokkatoo/blob/cefd52c39b646cd85a132cd84632af53cbec7dd7/modules/dokkatoo-plugin/src/main/kotlin/DokkatooExtension.kt#L92-L98|plugins configuration NDOC> (although this is a `ExtensiblePolymorphicDomainObjectContainer` so I don’t know if it counts as a NDOC <https://github.com/adamko-dev/dokkatoo/blob/cefd52c39b646cd85a132cd84632af53cbec7dd7/modules/dokkatoo-plugin/src/main/kotlin/internal/gradleTypealiases.kt#L7|src>).

```dokkatoo {
  pluginsConfiguration.named&lt;DokkaHtmlPluginParameters&gt;("html") { // no accessor generated
  }
}```

as far as I can see, `DokkatooExtenstion.dokkatooSourceSets` is just a regular property, not an extension that Gradle knows about

Yep, seems so. If you add it as extension to your extension instead, it should probably work.

oh that’s wild. I assumed because `DokkatooExtension` was a <https://docs.gradle.org/current/userguide/custom_gradle_types.html#managed_types|managed type> then that would be enough for Gradle to figure it out and create accessors. And I was basing this off the `KotlinProjectExtension`, which looked like it was doing the same thing in <https://github.com/JetBrains/kotlin/blob/f8c51dcfeec5ce32f8b7dd216fe563fe9909ab60/libraries/tools/kotlin-gradle-plugin-api/src/common/kotlin/org/jetbrains/kotlin/gradle/plugin/KotlinTargetsContainer.kt#L20-L22|KotlinTargetsContainer.kt>

```interface KotlinSourceSetContainer {
    val sourceSets: NamedDomainObjectContainer&lt;KotlinSourceSet&gt;
}```
But actually the implementation <https://github.com/JetBrains/kotlin/blob/f8c51dcfeec5ce32f8b7dd216fe563fe9909ab60/libraries/tools/kotlin-gradle-plugin/src/common/kotlin/org/jetbrains/kotlin/gradle/dsl/KotlinProjectExtension.kt#L160-L165|KotlinProjectExtension.kt> is using an extension under the hood

```open class KotlinProjectExtension @Inject constructor(project: Project) : KotlinTopLevelExtension(project), KotlinSourceSetContainer {
    override var sourceSets: NamedDomainObjectContainer&lt;KotlinSourceSet&gt;
        @Suppress("UNCHECKED_CAST")
        get() = DslObject(this).extensions.getByName("sourceSets") as NamedDomainObjectContainer&lt;KotlinSourceSet&gt;
        internal set(value) {
            DslObject(this).extensions.add("sourceSets", value)
        }```

I’ll give it a go, thanks for the hints!

adding `NamedDomainObjectContainer&lt;*&gt;`s as extensions instead of properties is working well :+1:

For the `ExtensiblePolymorphicDomainObjectContainer` it partially works. The DSL accessors are generated, but they’re not cast to be the actual type. just the generic type of the container.

Is it possible to generate these accessors so the configuration block is the correct type?

```dokkatoo {
  pluginsConfiguration.html {
    // this is `DokkaPluginParametersBaseSpec`
    customStyleSheets.from("logo-styles.css") // Unresolved reference: customStyleSheets
    // but it should be `DokkaHtmlPluginParameters`
    require(this is DokkaHtmlPluginParameters)
  }
}```

(p.s. sorry for taking over your thread <@U022JB3L18U>!)

sounds possibly related to <https://github.com/gradle/gradle/issues/23740>

Sorry, maybe you already said it but you can use this code too:
`val cobols = project.objects.domainObjectContainer(CobolSource::class.java)
project.extensions.add("cobol", cobols)` and `cobols.add`. For each Cobolsource, which implements Named, a typesafe accessor is generated, IFF you don’t use afterEvaluate. 

If you check the componentId on the root of the `ResolutionResult`, then you can find the buildPath on its build identifier there. That will always be the build path of the current project. Then you can compare this buildPath to the ones in your artifact view.

In Gradle 8.3 we will introduce a `buildTreePath` on `Project`, so you could use the `buildTreePath` of the root project in your current build to compare.

Does anyone know if it's possible to do gradle flavour selection by the consumer dependencies?

For example, I'd like to have flavour lib-b if consumer has B dependency and default to flavour lib-a otherwise without requiring any extra configuration on consumer side

Please don't cross-post to multiple channels, that wastes the time of anyone reading the message multiple times and splits discussions. Pick one appropriate channel according to name and subject and at most post a link in other channels if you really feel you need more attention.

Have you tried using IntelliJ instead of AS? There’s been some improvement for composite builds in newer IJ versions, and I think that AS is a few versions behind so it might not have received them yet.

you could also try `api("com.android.tools.build:gradle:8.0.0")` instead of `implementation("com.android.tools.build:gradle:8.0.0")`, that _might_ make AS realise it needs to index the dependencies. But I wouldn’t recommend using API just to fix an IDE issue.

Sorry about the crossposting! Will not do anymore. The AS I'm using is Hedgehog Canary 2, so it should be 2023.1, but I'll check just in case

... Community Edition 2023.1 does the trick. Do I really want to know why ? Probably not. Thanks <@U02KX5B7EVC>!

although that might indicate that AS needs to clear its caches

I'm not sure if it's the best approach, but what I did is:
1. make two separate tasks - MetadataTask and CodegenTask
2. android's assemble tasks depend implicitly on CodegenTask's output. CodegenTask depends explicitly on MetadataTask (since there are no module specific outputs).
3. MetadataTask creates a properties files in the root project and each module appends to that properties. It's done via worker api, which hopefully prevents concurrent writes between different modules' MetadataTasks
Since modules create a DAG, there should be no missing metadata before any module's CodegentTask is run.

I'm not sure about writing to the root project's dirs, but it's my best idea so far.

If I understand correctly, your workflow is like below:

AGP does not provide public APIs for this requirement by default. I did some wrap up to add a hook point in MergeResources Task which you can get all resources(including sub modules) at one place in Application module. Then you are able to collect metadata and generate the code.
```val allInputResources = variant.artifactsPolyfill
    .getAll(PolyfilledMultipleArtifact.ALL_RESOURCES)
val postUpdateResourceTaskAction = PostUpdateResourceTaskAction(buildDir, allInputResources)
variant.artifactsPolyfill.use(
    action = postUpdateResourceTaskAction,
    toInPlaceUpdate = PolyfilledSingleArtifact.MERGED_RESOURCES
 )```
<https://github.com/2BAB/Polyfill/blob/master/polyfill-test-plugin/src/main/kotlin/me/xx2bab/polyfill/test/TestPlugin.kt#L81>

Leveraging the `JavaRes` merge is also feasible if you generated metadata to each module’s java resource folder.
```            val javaResPathsOutput = project.objects.fileProperty().fileValue(
                getOutputFile(
                    buildDir,
                    "all-java-res-by-preUpdateJavaResTaskAction.json"
                )
            )
            val preUpdateJavaResTaskAction = PreUpdateJavaResourcesTaskAction(javaResPathsOutput)
            variant.artifactsPolyfill.use(
                action = preUpdateJavaResTaskAction,
                toInPlaceUpdate = PolyfilledMultipleArtifact.ALL_JAVA_RES
            )```
<https://github.com/2BAB/Polyfill/blob/master/polyfill-test-plugin/src/main/kotlin/me/xx2bab/polyfill/test/TestPlugin.kt#L98>

The library is still under incubating, currently supporting up to AGP 7.2 only. You can check the source code and feel free to take some hook approach from that.

Thanks for responding <@U01HL1HCLKY> :slightly_smiling_face:

The metadata is not really code or resources. It's some import mappings, which are used for the code generation, but can be dismissed afterwards, because they do not serve any purpose in the generated code.

Okay, my approach of reusing the metadata across different modules seems to be identified as problematic by gradle
&gt; Execution optimizations have been disabled for task '*:feature:account*:generateStuffProdDebug' to ensure correctness due to the following reasons:
&gt;   - Gradle detected a problem with the following location: '.../build/intermediates/prodDebug/metadata.properties'. Reason: Task '*:feature:account*:generateStuffProdDebug' uses this output of task '*:feature:auth*:generateMetadataProdDebug' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to <https://docs.gradle.org/7.5.1/userguide/validation_problems.html#implicit_dependency> for more details about this problem.
I guess that using metadata generated by one module in another module breaks some gradle optimizations. Back to drawing board :disappointed:

Well, even though they should not appear in final APK, you can still serialize/persist per module metadata in file(resources), and then exclude from APK packaging process. I do not see it hurts/breaks any other Gradle/AGP rules. :)

I could, but using non compiled code / resources as is during compilation time and then discarding it seems hackish. I'm thinking of a different approach. Maybe I should only apply the plugin to the application module and gather all subprojects assets.
Added bonus it's easier setup in multimodule environments. Can't think of drawbacks yet

Yep it looks a bit hacky at first glance. However, consider how AGP is doing, for example:
&gt; For each AAR or library module, AGP generates a R.txt which will be bundled into the AAR.
&gt; Later in Application module, R.txt is not gonna packaged into the APK, instead, they are consumed by some tasks to help merge and generate the final artifact resources.arsc/R.java/etc.
&gt; <https://developer.android.com/studio/projects/android-library#aar-contents>
So basically we are working on the same page just like AGP.

I tried a hybrid approach today, i.e. creating these metadata.properties files in `build/intermediates` and making the higher level module use all of its submodules metadata files as input. Problem is that using these files as inputs doesn't create implicit dependency between these tasks. And apparently the order is not guaranteed, even if one module directly depends on another, e.g.:

```:moduleA -&gt; implementation(project(:moduleA:moduleB))
   :moduleB```
It seems that `moduleA:generateMetadata` will be resolved before `moduleB:generateMetadata`, so I won't be able to access the generated metadata in `moduleA` at the right time :disappointed:

I wonder if doing it like you suggested <@U01HL1HCLKY>, i.e. putting the generated metadata in the sources would ensure the right order. I'll try that as the next step.

Yeah, didn't work :confused:

I used the moduleA / moduleB setup like above and tried to generate the metadata to Java resources directories. But still, the moduleA are generated before moduleB, so I can't use them in the right order or I am missing something important here.

Use something else than workers ? Just plain coroutines/threadpools ? Then you control parallelism?

hmm, I think it won't fly unless I stray away really far from Gradle's model

the work is still driven by tasks and the outputs are necessary

just that the computation happens somewhere else so I'd like to say that these tasks/workactions can run more than the standard max parallel worker limit

I mean you can still have a task with outputs etc... just make the work happen in parallel inside your single task? Like if you have a task that encodes H264, it will spawn threads to encode blocks of video in parallel. This is no different

Worker is useful if you want Gradle to handle parallelism (which you don't want here) or for process/classpath isolation

but lets say I have an encoder task that takes an input video file and an output video file and encodes somewhere outside of the local machine

I can only have up to max parallel workers input-output pairs

more concretely, in this case I'm running tests remotely

&gt; encodes somewhere outside of the local machine
You'd need to get the result on the local machine

I think I could multiplex all of the modules and their tests into a single task, but it would get really weird with Gradle's model

So that the output can be tracked. But I think worker do that?

&gt; really weird with Gradle's model
Looks like you don't want Gradle to handle your parallelism here

I do :slightly_smiling_face: I just want to say that these tasks aren't cpu-bound

At the end of the day, Gradle is plain JVM, you can execute the code you want in there

so that Gradle can use the cpu for running other tasks

Not sure there's a way to tell Gradle to override max-workers

&gt; I'm running tests remotely
So it's different test tasks, each one with a single worker ?

which overall is working great! the test tasks within a module run parallel, and across modules I get parallelism as well

Rereading that Gradle documentation. I'm always confused what is executed in parallel and what not. I though inside a single module, all tasks would execute serially?

`org.gradle.parallel=true`  allows you to run different modules in parallel but inside each module, I think tasks run one at a time?

it's really poorly documented but workers allow you to run things in parallel within a module as well

Like a single task can use a workQueue to run work in parallel

nope, you can get two tasks within a module to run in parallel

I was looking at this: <https://docs.gradle.org/current/userguide/performance.html#parallel_execution>

```Yet by default, Gradle only runs one task at a time. To execute tasks belonging to different subprojects in parallel, use the parallel flag:```

But this hints at parallel tasks: <https://docs.gradle.org/current/userguide/multi_project_configuration_and_execution.html#sec:parallel_execution>

Mark mentions this here - <https://discuss.gradle.org/t/can-we-run-tasks-in-parallel-in-single-build-project/18321/14>

I don't get it. The worker API is used from inside the task action, right?

So you'd have to run the task actions in parallel anyways whether or not you use the worker API