From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

Which information do you need that is not given to the operation completion listener?
Yes, it is only triggered after a task finished, but it contains also information about when the task started.

What I'm missing is an officially supported way of getting this data, and with start events. The external API I'm tracing to requires the parent to be there before the child, so with only `finished` events, I'd have to keep a cache of all events until the build is fully finished before I can process them. Not good, since the main build this should be used for generates about 100k events.

&gt; What I'm missing is an officially supported way of getting this data, and with start events.
I asked which information it is you miss from the officially supported `OperationCompletionListener` that you currently cannot get out of it.
&gt; The external API I'm tracing to requires the parent to be there before the child, so with only `finished` events, I'd have to keep a cache of all events until the build is fully finished before I can process them.
Unfortunately, I didn't get what you mean, can you elaborate a bit on what you mean / what you try to achieve? Then I might be able to suggest some proper way to do it.

I'm trying to collect as much data about build configuration and execution as I can, to be able to find performance degradations and other problems, so the more events I can get, the better. I might be wrong, but `OperationCompletionListener` only seems to support task executions, which is way too coarse for my purposes and completely excludes the configuration phase.

To correlate all that information in the external system I'm reporting to, I need proper parent-child relationships at the time I'm seeing the child, which is the opposite order of `finished` events.

I can get all that if I register a `BuildOperationListener` via an injected `BuildOperationListenerManager`, but all of that is internal, and if I don't manage to remove my listener again, it even stays active in the daemon for all subsequent builds, which is _really_ not what I want.

I don't think there is a supported way for what you are trying to do, besides "simply" using build scans.

I was afraid you'd say that.
So just to make sure: there is nothing better than`OperationCompletionListener` that's officially supported, right? The only way to get build information via listeners is for tasks and nothing else?

As far as I am aware of, yes.
You could maybe add a `doFirst` action to all tasks where you can to "before-task" actions.

Yeah, but that would still only give me information about tasks. Thanks for your help and for confirming my suspicions that I'll have to do this via internal APIs. As little as I like that.

<https://docs.gradle.org/current/javadoc/org/gradle/build/event/BuildEventsListenerRegistry.html> you only get notified on completion, but the events include start and end time in result

Thanks, I had tried that, because it was official, but it's only for tasks and backwards from the way I need it.

Btw. for task there *is* an official public API that also gets start events: `TaskExecutionGraph.addTaskExecutionListener`
It's just deprecated and not compatible with configuration cache. :slightly_smiling_face:

1. Get the `ResolvedComponentResult` of an `app` module
2. Iterate over its dependencies
3. If it's a project, call a lambda then walk down that component's `ResolvedComponentResult`
4. If it's a module and we haven't seen it before, do the same
I've looked at how Gradle natively prints out information using its `dependencies` task but truthfully couldn't follow the logic well enough

I might have found a case where there's cyclic dependencies - still testing

False alarm, looks like this is a non-issue. Sorry for cluttering up the channel!

Not really, other than refactoring out in yet another included build that can set both locations and that you then use in both.

An included build is a standalone build that is just built automatically before your including build, but other than that it is practically the same as publishing the included build somewhere and then using the published artifact. So in `build-logic/build.gradle` you need the repository to retrieve your build dependencies, in `rootProject/settings.gradle` you define the repositories where to get plugins and their dependencies. It is always in the responsibility of the consuming build to define the repositories where to get dependencies and plugins from. Otherwise you could for example hardly follow company guidelines that mandate to only use internal repository servers. And while plugins for example could add repositories technically - unless the consumer build prevents it - it is bad practice to do so in generally available plugins, at least if it is unconditional.

`tasks.jar.flatMap { it.archiveFileName }` or something like that?

Not entirely sure if this works for buildSrc - doing it using a composite build - drop something like this in your settings.gradle.kts:
```dependencyResolutionManagement {
    versionCatalogs {
        create("libs") {
            from(files("../gradle/libs.versions.toml"))
        }
    }
}```

<https://melix.github.io/blog/2021/03/version-catalogs-faq.html#_can_i_use_the_version_catalog_in_buildsrc>

thx <@UAFGSU86M>. The above link shows how to retrieve and add the dependency, for me it was `kotlinCompilerExtensionVersion` that is required for the Android config. Your hint helped me and I made it work by retrieving version from the catalog like this:
```val catalogs = extensions.getByType&lt;VersionCatalogsExtension&gt;()
        val libs = catalogs.named("libs")
        kotlinCompilerExtensionVersion = libs.findVersion("kotlinCompilerExtensionVersion").get().toString()```

chicken and egg.  The configuration phase (task graph/dependencies) has already happened by the time you execute a task.  Perhaps modelling this as a build service and doing all this at configuration time would work.

Yeah this is what I was afraid of. I'd rather not take that configuration time hit at the moment.

I've been pouring over <https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:adding_dependencies_to_tasks|this taskDependencies doc> but can't find a way that's both:
1. Configuration-cache-safe
2. Not run at configuration time

another option, perhaps viable (depends on the specifics / complexity of your soln) is to always add the task dependencies, and have an `onlyIf` to conditionally execute them (which would look at _something_ for that logic)

So in an example build graph:
app -&gt; library -&gt; core

I could define a `Task` that *always* adds `app:test` `library:test` and `core:test`  as task dependencies, but use `onlyIf` to only run `app:test`  based on some condition that I can run during the execution phase?

Interesting, I'll give this a try. Is that the only possible option, though?

One Issue I want to solve is we have two builds: `shared` and `platform`. I want to only run tests on relevant code, and there doesn't seem to be a way to register a task in `platform` that has knowledge of the substituted projects from `shared`

you’re looking for something beyond running `test` in the root project, which would run all `test` tasks in sub-projects (with their up-to-date checks, etc)?

Indeed! It would need to run tests for *every* buildable artifact in a given graph, not just the `subProjects` of a given `RootProject`

is each subproject not a buildable artifact?  If there are variants within a project, could their test tasks be added as dependenceis on the lifecycle `test` task?

From what I've seen `subprojects` doesn't include any `Project` that is not a direct descendent of the root, though I could be wrong in that.

e.g. if you have two builds, `platform` and `shared`, `platform:subprojects` won't have the projects it uses from `shared`

wasn’t suggesting `subprojects`, that’s legacy and broken.  When you run `test` on the root project Gradle finds all `test` tasks in any child project (of any level) and runs those.  This accounts for any hierarchy, project substitutions, etc.

if those are separate builds you could create a wrapper build that includes them as composite builds (or include one in the other, if there is a logical parent).

We do have a composite parent build, yeah, that includes both `platform` and `shared`. What's strange is if I run `test` from the root (parent) directory like `./gradlew platform:test` , I get a task not found exception

But when I cd into `platform` and run `test`, that works as expected (As you described)

check what you see in `./gradlew tasks`  Of note, the current project in front of me has an included  _plugin_ build, those tasks don’t show up (as they shouldn’t - it’s a different lifecycle being a plugin).  Or `./gradlew tasks --all`

`./gradlew tasks --all`  doesn't return anything from the included builds - is that to be expected?

ah yea.  yes, composite builds are separately executed, not merged into overall build.  They are intended to substitute dependencies.  If that is not what your composite builds are doing perhaps they are better reflected as an included build (settings.gradle.kts -&gt; include)

Btw if `subprojects` is legacy, what should we use instead?

convention plugins are the correct way to configure multi-module projects, avoiding coupling via `subprojects` (or `allprojects`) that breaks project isolation, etc.

So we have convention plugins for each of our different project "types", but we use `subprojects` to do things like:

``` project.subprojects {
            val testTask: String? = when {
                project.isAndroidApp() || project.isAndroidLib() -&gt; "$path:testDebugUnitTest"
                project.isJavaProject() -&gt; "$path:test"
                else -&gt; null
            }
            if (testTask != null) {
                this@TestAll.dependsOn(testTask)
            }
        }```
I've been meaning to get rid of this logic anyway - what we should convert this logic to?

This `TestAll` task was added so we could only run certain variants, as `Android` projects have both a `debug` and `release` test that runs

not an Android developer so may miss some specifics, but would suggest modelling this as:
• each convention plugin exposes ‘testAndroid’, ’testAndroidLib` as appropriate;
• In your root project create the same task names, which implicitly will run the corresponding task in all subprojects.
so `./gradlew testAndroid` or `./gradlew testAndroidLib`
As desired add these as dependencies of `test`  or have a `testAll` as you’ve noted.

So I have what I think is a very ugly initial implementation of this working:

Apologies for the text blob, thought it'd be formatted a bit better

My only question left: if I have those two builds `platform` and `shared`, and I want to run tests for `shared`, I can just run `test` on `platform` right?

depends: if its a composite build it’s a separate execution and won’t propagate tasks; if its an included build, it will propagate tasks.

That goes back to my initial question, then. I'm designing something that'll only run tests on changed projects - if I run `test`, that'll run `tests` on all dependencies even if we don't need to re-validate them. How can I run `test`  for *just* the paths in the graph that we know we need to validate?

For context, we do have Gradle Enterprise, some `build/` folder caching, etc setup on our CI server.

generally all that is handled by task up-to-date checks - only projects that have changes will need to re-run tests.

Gotcha, lemme do some testing validation then get back to you

Circling back to this discussion:
1. The "automatic task finding" feature (i.e. I run `X` in my root project, I expect `X` to run for all subprojects) works as expected but not for our case because we have included builds
2. I was able to dynamically generate a list of tasks to run based on files changed (on top of our Gradle Enterprise/task output caching), but I'm struggling how to model this logic.
You mentioned previously that I should model this all as a build service. I currently have three tasks:
1. Find list of changed files using git
2. Find affected projects by traversing `ResolvedComponentResult`
3. Use result of 2) to run stuff, but running into a chicken-and-egg problem as you mentioned
How would I migrate this logic to the configuration phase and not the execution phase?

UPDATE: I understand how to do this in the configuration phase, but now how do I ensure this is properly tested? I had a bunch of TestKit tests that ran a given task and asserted on its output - can the same be done for things done *not* in the execution phase?

Is there a way to listen for task cancellation and perform some logic? For graceful shutdown type scenarios?

sure, that should be possible in a few ways

The application plugin has a task to build the zip, and it also has a preparatory task that creates a directory of the files that go into the zip. Create a new Zip task that uses that prep-task, and does some juggling to get the files sorted.

Alternatively, use the ‘fat jar’ example, but instead of a Jar task, use a Zip task <https://docs.gradle.org/8.1.1/userguide/working_with_files.html#sec:creating_uber_jar_example>

Thanks, Adam!

I went with regular `Zip` and `Copy` tasks now:
```val distCopySpec = copySpec {
    val jar = tasks.getByName&lt;Jar&gt;("jar")

    from(jar)
    from(configurations.runtimeClasspath)
}

tasks.register&lt;Copy&gt;("distTest") {
    with(distCopySpec)
    into(distsDirectoryTest)
}

tasks.register&lt;Zip&gt;("distZip") {
    with(distCopySpec)
}```

looks a bit confusing, but if it works, it works :)

Do you have any recommendations how to improve it? :slightly_smiling_face:

my one note would be that Sync tasks are usually better, because they will clear out old files. For example, if you remove a dependency then you don’t want that dependency to linger in distsDirectoryTest. A Copy task will keep it, while a Sync task will remove it.

For context: I'm building a Debezium plugin and need the exploded distribution (i. e. all JAR files) in a local directory for testing _and_ the layout in the tests distribution should be the same as in the final ZIP archive, hence the shared `CopySpec`.

Thanks for the hint about the Sync task. I didn't know about that one. :heart:

What I’d suggest is flipping the order of your tasks. First task builds the zip, and the 2nd task unpacks the zip into the test directory. Then you don’t have to share the copySpec, and you can be more sure that you’re testing the actual distribution.

Makes sense, thanks!
I've ended up with this now:
```val distsDirectoryTest = project.distsDirectory.dir("test")
tasks.register&lt;Zip&gt;("distZip") {
    val jar = tasks.getByName&lt;Jar&gt;("jar")
    from(jar)
    from(configurations.runtimeClasspath)
}

tasks.register&lt;Sync&gt;("distTest") {

    from(zipTree(tasks.named&lt;Zip&gt;("distZip").map { it.archiveFile }))
    into(distsDirectoryTest)
}```

you can use a variable to reference the first task, so it’s not stringly-typed

```val distZipTask = tasks.register&lt;Zip&gt;("distZip") {
    val jar = tasks.getByName&lt;Jar&gt;("jar")
    from(jar)
    from(configurations.runtimeClasspath)
}

tasks.register&lt;Sync&gt;("distTest") {
    from(zipTree(distZipTask.map { it.archiveFile }))
    into(distsDirectoryTest)
}```
Or even use a <https://docs.gradle.org/8.1.1/userguide/kotlin_dsl.html#using_kotlin_delegated_properties|delegated property>, so the task will use the name of the variable

```val distZip by tasks.registering(Zip::class) {
    val jar = tasks.getByName&lt;Jar&gt;("jar")
    from(jar)
    from(configurations.runtimeClasspath)
}

tasks.register&lt;Sync&gt;("distTest") {
    from(zipTree(distZip.map { it.archiveFile }))
    into(distsDirectoryTest)
}```

One more note to answer the initial question.
What you actually are after is the `distribution` plugin.
The `distribution` plugin is the one providing the `distZip`, `distTar` and `installDist` tasks.

The `application` plugin and also the `java-library-distribution` plugin just apply the `distribution` plugin and then configure the `main` distribution, e.g. defining that the dependencies are copied to `libs`.

So while you of course can just write own tasks, you could also apply the `distribution` plugin and configure the `main` distribution however you need it. Or if one of the other plugins is also applied that configure the `main` distribution, you could then also simply create another `distribution` that you configure how you need it and automatically get the according tasks created.

<@URQ797WRW> Thanks for the suggestion!

Do you know if there's a "clean" way to avoid that the default `distTar` task is being registered by the <https://docs.gradle.org/8.1/userguide/distribution_plugin.html|Distribution plugin>?
I don't need the tar archive and it feels "wasteful" to have that task automatically being executed when `assemble` is being executed.

I was hoping that I was missing a setting which would control whether the task was being registered to begin with, but disabling it will definitely work, too. Thanks!

Changing the configuration of a task during its own execution phase is even worse than changing the configuration of a task from the execution phase of another task.

So no, this is not the correct approach. You for example disturb up-to-date checks and if it were cacheable also cache entries.

A valid approach would be (as ad-hoc version)
```val generateManifest by tasks.registering {
    val outputFile = layout.buildDirectory.file("MANIFEST-generated.MF")
    outputs.file(outputFile).withPropertyName("generatedManifest")
    doLast {
        outputFile.get().asFile.writeText(
            """
                Foo: bar
            """.trimIndent()
        )
    }
}

tasks.jar {
    dependsOn(generateManifest)
    manifest {
        from(generateManifest.map { files(it).singleFile })
    }
}```
Or as proper task class version:
```abstract class GenerateManifest : DefaultTask() {
    @get:OutputFile
    abstract val generatedManifest: RegularFileProperty

    @TaskAction
    fun taskAction() {
        generatedManifest.get().asFile.writeText(
            """
                Foo: bar
            """.trimIndent()
        )
    }
}

val generateManifest by tasks.registering(GenerateManifest::class) {
    generatedManifest.set(layout.buildDirectory.file("MANIFEST-generated.MF"))
}

tasks.jar {
    dependsOn(generateManifest)
    manifest {
        from(generateManifest.flatMap { it.generatedManifest })
    }
}```
Unfortunatley, it needs an explicit `dependsOn` as the implicit task dependency is not respected in this case. I just reported that as <https://github.com/gradle/gradle/issues/25435>

What a savior you are :slightly_smiling_face:
```manifest {
    from(generateManifest.flatMap { it.generatedManifest })
}```
Is nice. I didn't know about it.
Thank you for your help!

Side note: my `GenerateManifest` task actually uses a Build Service's properties (which can not atc as an task input at the monent) so I can not reuse the task's output. So UP-TO-DATE check doesn't bother me right now.

&gt; which can not atc as an task input at the monent
who says that?

This works perfectly fine for example:
```abstract class RandomService : BuildService&lt;BuildServiceParameters.None&gt; {
    val randomValue get() = Random.nextInt()
}

abstract class RandomTask : DefaultTask() {
    @get:ServiceReference
    abstract val randomService: Property&lt;RandomService&gt;

    @get:Input
    abstract val input: Property&lt;Int&gt;

    init {
        input.set(randomService.map { it.randomValue })
        outputs.upToDateWhen { true }
    }

    @TaskAction
    fun taskAction() {
        println(input.get())
    }
}

gradle.sharedServices.registerIfAbsent("random", RandomService::class) { }

val random by tasks.registering(RandomTask::class)```
The task `random` it out-of-date because the input changes. And if you change the build service to always return the same value, the task is properly up-to-date.

&gt;  who says that?
Well, <https://docs.gradle.org/current/userguide/build_services.html#using_a_build_service_from_a_task|the documentation> says: _Note that using a service with any other annotation is currently not supported. For example, it is currently not possible to mark a service as an input to a task._

But your approach:
```init {
    input.set(randomService.map { it.randomValue })
    outputs.upToDateWhen { true }
}```
looks promising. I didn't know that I can do it like that. I will try that. Thank you!!

Looks like this works
```task.inputs.files(fooConfiguration)```


Hello , my build.gradle(module:app) is missing due to which any of my new project is not running.

WhatsApp Image 2023-06-20 at 19.24.10.jpg

Please do not split topics across several threads.
This makes following the conversation in the threads view very hard as the context of the other messages is missing.
If you have additional information either edit the original message or post to its thread.

And please do not send photos of screens. They are always superhard to read. At least send a screenshot, or even better copy the text if it is only about the text.

Regarding the problem, you say your build script is missing, but that is not what your screenshot shows.
A missing build script is no problem at all, a project does not have to have on, it is just bad practice.
What the screenshot shows is, that you try to use an Android Gradle Plugin version that does not exist.

Ok , I am just new here, will keep that in mind from now onwards.

Please just read the whole error message. The last sentence shows the error: No matching variant was found. Without further details I can't add a solution, but a common mistake is JDK mismatch, AGP 8.0.1 requires JVM 17.

Ah, sorry, thought 8.0.0 was the latest. Was fooled by 8.0.0 release notes being the latest shown. So yeah, remaining error message would be needed.

```target.afterEvaluate {
    val implementationDependencies =
        target.configurations["implementation"]
            .allDependencies
            .filterIsInstance&lt;DefaultProjectDependency&gt;()
            .toSet()
    target.dependencies {
        for (dep in implementationDependencies) {
            "contextual"(target.project(dep.dependencyProject.path))
        }
    }
}```


I am unable to know if my plugin is being applied in the other dependency too, so I would like to add it in a safer way automatically.

I think for knowing the plugin part it can be only done in a settings plugin, right?

Have your configuration extend from implementation.

I am getting this issue

```- Incompatible because this component declares a component, packaged as a jar and the consumer needed a component, with the library elements 'contextual'```

the library which is reporting the issue is another one which is not a project and which doesn't have the `contextual` part, so it looks like this issue is "similar" to an android library which has two variants and other which has three and it doens't know what to pick, and you need fallbacks and so on.

cc <@UDC9Q9RD0>

• <https://docs.gradle.org/current/userguide/declaring_dependencies.html>
• <https://docs.gradle.org/current/userguide/feature_variants.html>
• <https://docs.gradle.org/current/userguide/variant_model.html>
I have read that and I ended up on this too
<https://github.com/gradle/gradle/issues/12126>

TBH, I "understand" what is the problem and it is expected, but I don't understand how to do optional attributes or variants.

My `contextual` is extending `implementation`, and one of the dependencies added to `implementation` doesn't know anything about `contextual` which is normal, because that dependency hasn't be processed by my plugin, but, that means that dependency has not added any artifact to the `contextual` as there is no `contextual`, so it must be skipped.

How can I mark this attribute as optional?

But you did not invent a new attribute, you added a value to an existing one

You can declare a compatibility rule, so that you can get `whatever` if `contextual` was requested, but then you get the `whatever` jar.

If you not want the dependency if `contextual` is not present, use a lenient artifact view.

This stuff is really hard and difficult to figure out. You can try using an artifact view with variant reselection

```val contextual by configurations.creating { ... }

contextual.incoming.artifactView {
  withVariantReselection()
  //lenient(true) // you can also try ignoring failures
}.files```
sometimes I found that ‘resolved artifacts’ was necessary - something to do with triggering transforming files

```val contextual by configurations.creating { ... }

val contextualContents: Provider&lt;List&lt;File&gt;&gt; = contextual.incoming
  .artifactView {
    //lenient(true) // you can also try ignoring failures
    withVariantReselection()
  }.artifacts
  .resolvedArtifacts
  .map { artifacts -&gt;
    artifacts.map { it.file }
  }```


interesting, I think `lenient` is what I was calling "optional" attribute, thank you both, I will try :slightly_smiling_face:

`lenient(true)` works, if I only use `withVariantReselection()` it doesn't work.

&gt; if I only use `withVariantReselection()` it doesn't work
Of course not, why should it?
That method means "ignore the attributes declared on the configuration and just resolve again with only the artifact view defined attributes".
That doesn't change that your dependency has no matching variant.

Thank you for the clarification! TBH it is hard to understand the whole variants concept

Yes, I also needed quite some time to wrap my head around it and am still not 100% sure I fully got it. :smile:
But once you got the grasp it is not that hard anymore. :slightly_smiling_face:

Does `--write-locks` imply `--refresh-dependencies`, or do you need both in order to guarantee the latest dependencies?

The "proper" way to access things from other projects is to have them expose them as artifact attached to variants: <https://docs.gradle.org/current/userguide/cross_project_publications.html#sec:variant-aware-sharing>
Fwiw, `sourceSets.main.output` and `configurations.runtimeClasspath` are already exposed as the `runtimeElements` of Java projects, so declare a configuration and add those target projects as dependencies to that configuration and it should "just work".

From your snippet, it's hard to understand what you actually want into your JAR.

(fwiw, your problem here is probably due to the `runtimeClasspath.get().files` that resolves the configuration too early)

I'd say he wants a bad-practice fat jar with all projects output and all dependencies crammed inside.

I strongly recommend you not do that, but instead build a proper distribution using the `application` plugin. See <https://fatjar.net> for some quotes on why it is a bad idea.

If you really need to follow that road, at least us the `shadow` plugin of John Engelman which mitigates at least some of the problems you get from trying this.

<@UDC9Q9RD0> Actually, I just want create `jar` artifacts so previous method I use `intellij` to create `jar` but I need to convert to create by command line for use in CI.

As I know to add from compare with previous method.
• all main of subproject
• kapt because I use some generate lib to create fule
• all dependencies each use in subproject

<@URQ797WRW> Regrading `shadow` are you mean this way?
```plugins {
    alias(libs.plugins.shadow)
}```
```shadowJar {

}```

My recommendation stands. Do not even try, but simply use the `application` plugin to build a proper distribution and save you a lot of headache.

But maybe I meant that, I cannot guess what you have in your version catalog :wink:

Sorry, `com.github.johnrengelman.shadow`
I can't use normal way to build app because I need to create `jar` for mod game and that game to call function in my  `jar`  file :sob:

You could declare that task from an init script so it's kept separate from the Gradle projects' themselves.

And if you do not do it via init script, but external script, it is probably better to use the result of `outgoingArtifacts` task, as - if I got you right - you are not actually interested in the group and project name, but in the produced artifact coordinates. Those can differ, even if in most cases they are the same.

Since I'm looking to do an artifact promotion into a releases repository but it's not directly writable instead need to execute a command that tells the repository which namespace (groupId), package (artifactId), and verison to retrieve to make available.

Looks like `OUTPUT=$(./gradlew outgoingVariants --variant runtimeElements --console plain | grep -A1 "^Capabilities" | tail -n1 | xargs | cut -d' ' -f2)` might get me what I want, but I can certainly see that it might be better to add a task for this to just output the capability information of interest and then that can be parsed in a script to split it into the fields `&lt;groupId&gt;:&lt;artifactId&gt;:&lt;version&gt;` which would be ideal

using `compileOnly` + `testmplementation` is the right thing to do

Thanks, Cédric! Just wanted to make sure I'm not missing something. :slightly_smiling_face:

(note that technically it could also be a `testRuntimeOnly`, so covering all cases would require quite a few configurations, probably overkill)

try taking a look at <https://github.com/ljacomet/logging-capabilities>

even if it builds, a plugin will be loaded into Gradle's classpath at runtime, where the slf4j implementation will be the one from Gradle and not whatever the library wants

e.g. that library is just incompatible with being used in a Gradle plugin

<@U019L3NM4PK> what would be your recommendation in this case? Rewrite the java libraries to be compatible w/ the plugin’s logging library?

I’m assuming that I won’t be able to rewrite the plugin without using the `groovy-gradle-plugin` and thus rewriting the plugin is not a viable solution for my use case?

it doesn't matter if you create the plugin via `groovy-gradle-plugin` or manually set up the compilation and output, it simply can't work at runtime

you'll have to either fix the java library or use something else

Yeah it's a class path issue so as is how things are done, you should probably pick the latest backwards compatible library. But yes, you don't get much of a choice in java lands

I know, you should not use `project.evaluationDependsOn`, but how does it work at all when called in a project build script/plugin? Does Gradle stops the current evaluation, evaluate the other project, and resume the evaluation later at this point?

I guess you talk about the one in the gradle user home?
Would be bad if it did by default as that would mean the test runs different depending on who runs it.
The TestKit tests have separate gradle user home directories to be isolated and reproducible.
If you want the ones from the executing user in there, it is probably easiest you copy them in.

isn’t this the point of init scripts though?

to make adjustments to gradle builds such that they execute correctly in that environment

Yes, but not to tests of a project.
Tests should be written in a way they work everwhere self-contained, and run the same everywhere.
A test should not behave different just because someone else is executing it.

in my case I have a CI machine that uses init scripts to setup Gradle to use a Nexus proxy… buildsystem tests now cannot resolve dependencies because they don’t execute the init script and try to use repositories directly.

Maybe somehow extend the init script to manipulate test tasks to inject init scripts or something like that?

Or maybe instead use a DNS redirect or iptables entry to redirect the calls to the real repository to the proxy?

ended up with:
```systemProperty("gradle-init-scripts",   gradle.startParameter.allInitScripts.join(File.pathSeparator))```
and then:
```private static GradleRunner gradleRunnerWithInitScripts(Path projectDir, String ... arguments) {
  return GradleRunner.create().withProjectDir(projectDir.toFile()).withPluginClasspath()
          .withArguments(concat(of(getProperty("gradle-init-scripts").split(File.pathSeparator))
                  .flatMap(initScript -&gt; of("--init-script", initScript)), of(arguments)).collect(toList())
          );
}```


depends on your git needs, may not need to execute git - something like eGit could suffice.

I use the output of `git diff HEAD...master` and thus need to have this information tracked by git - It looks like that tool is just built on top of git?

if you need that specific output it’s likely cleanest (most predictable) to use git directly.  eGit is an API for git that would allow you to do git operations directly from code - while it would be possible to do the diff, you’d have to invest in formatting it etc.

Word, yeah it's good to know I'm on the right path. Inside of a Gradle Runner test, though, what's the preferred way to use third-party tools? Through an `Exec` task?

If it’s the test - and not your Gradle script that you are testing - you can use standard java/groovy/kotlin APIs to exec that command, or any other convenience-library that will eventually execute that command.

If your Gradle script under test is intending to execute the command then  you could use an Exec task which would drop the output in the logs for your test to then parse out (that latter part is fugly, dislike how TestKit makes you parse the log output)

programatically executing Git porcelain (like `git diff`) isn't a great idea if you're expecting the same output, since it can be hugely influenced by local `~/.gitconfig`/`/etc/gitconfig`
Gcalling

This command will run on a pre-configured container whose environment we control

even the output and behavior or Git plumbing can vary based on config too, but usually not by as much

Possibly once again not doing this the best way. I've reworked to define a curried function that I can then pass to the build.gradle by adding it to the `gradle.ext` global and referencing it via the same property name.
e.g.:
```def configureRepositories = { configObject, token -&gt;
        configObject.repositories {
                maven {
                        name repoName.replace('-', '')
                        url "https://${domain}-${domainOwner}.d.codeartifact.${region}.<http://amazonaws.com/maven/${repoName}/|amazonaws.com/maven/${repoName}/>"
                        credentials {
                                username 'aws'
                                password token
                        }
                }
        }
}

def configRepos = configureRepositories.rcurry(artifactAuthToken)
configRepos(settings.pluginManagement)
configRepos(settings.dependencyResolutionManagement)
gradle.ext.configRepos = configRepos```
Then at the end of build.gradle:
```gradle.ext.configRepos(publishing)```
Possibly there are better ways, but this means just need to drop the one file in to most projects and it's a small addition to the root build.gradle and should then work

To start with it would be great if Gradle were consistent. Annotation processor e.g. use a different dir than the Antlr plugin.

I tend to use the task’s temporaryDir, which is less likely to clash with other directories, and then don’t reference the output directory directly: instead reference the task’s output directory (<https://docs.gradle.org/current/userguide/incremental_build.html#sec:inferred_task_dependencies|example>)

I half-remember seeing some sort of discussion about providing a service for creating temporary files (possibly based on the internal <https://github.com/gradle/gradle/blob/v8.1.1/subprojects/file-temp/src/main/java/org/gradle/api/internal/file/temp/TemporaryFileProvider.java|TemporaryFileProvider> ?). But I can’t find it, and I don’t know whether the purpose was for one-off temporary directories, or more stable directories where the specific location doesn’t matter, but can be managed by Gradle so as not to clash with other dirs


There’s <https://docs.gradle.org/current/javadoc/org/gradle/api/file/SourceDirectorySet.html|SourceDirectorySet>, which is supposed to be a language-independent way of describing a set of files, but it’s pretty limited. There’s been some talk improving how to define sources used by an abstract language <https://github.com/gradle/gradle/issues/727>, because <https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/SourceSet.html|SourceSet> is very Java focused

There is no convention unfortunately. I think in most cases it make sense that you namespace your output, e.g. with task name: `build/&lt;task-name&gt;/` , but in some cases that is harder if task name is very generic.
But generally a good advice is: don't put outputs to a shared/common folders since that won't work ok with up-to-date checks.

I believe we won't go in the direction of some constants, since that will probably mean, users will put outputs for a lot of tasks to the same folder and that doesn't work well with up-to-date checks.

We have some ideas that Gradle would automatically assign output location and user would need to wire outputs of one task to another input via a property, something like Adam mentioned (and what is actually a proper way). Or at least have an annotation that would mean that Gradle should automatically assign an output. But there is nothing more than ideas at the moment for such feature and we don't have any actual plans for it.

So something like `build/generated/&lt;my namespace&gt;` would be ok but `/build/generated` would not?

Depends on the details.
If you have specific output files, it is quite irrelevant where they are, as long as they are not within the output directory of another task.
If you define the whole directory as output directory, then it is ok as long as no other task in your build writes anything to that directory, which could quite easily happen with such a generic name though.
So even if you use `build/generated/&lt;your namespace&gt;` but then apply some plugin which thinks it is a good idea to define the whole `build/generated` as output directory, you are in trouble again.

The most important point is, that no two tasks should have overlapping outputs or modify the outputs of other tasks in-place.

I recently encountered something interesting where an AGP API that depends on the output of a task seems to be moving the output to a new location.

Well, that is in the bad category if you really meant moving, not copying

I haven't looked through their code yet, but I'm assuming that's what it is. I was going crazy looking for the output I specified until I realized that the file was in this other directory.

there isn't a public way to react to registration without also realizing the object

The only way currently is to react to the same source that registers those tasks instead of reacting to the actual tasks.

Btw. if that tasks copies into the output directory of another task, it is a bad idea to do so anyway.

The task is to copy test resources to the runtime path of the test binary, so it can read the resources during tests. It is the output path of the task that produces the binary, but also needs to be the location of the resources for common API consistency.

Other examples of similar need would be copying the resources to a framework directory that is also the output path for the compilation's binary.

The task that produces the binary and the task that copies the resources to the same directory would always be run together, as the one depends on the other.

that's not true: on re-run, depending on inputs, one or both or neither may be run. Gradle isn't able to track where files are going when you do this.

Don't do that. It disturbs up to date checks, it disturbs cache entries of cacheable tasks, Gradle will detect it if you use the task output of one of these tasks and use it in another task and then complain that you miss a task dependency on the other task, when using the output of the one task, your will eventually get the output of both tasks or not depending on situation, ... Just don't do it, but find a better way during your use-case.

Well, if it works for you, fine. But don't wonder if anything starts to behave strange. :-)

if you really had to,
```abstract class CopyToTestWorkingDir @Inject constructor(private val files: Any) : Action&lt;Task&gt; {
    @get:Inject
    abstract val fileSystemOperations: FileSystemOperations

    override fun execute(task: Task) {
        fileSystemOperations.copy {
            from(files)
            into((task as KotlinNativeTest).workingDir)
        }
    }
}

kotlin.targets.withType&lt;KotlinNativeTargetWithTests&lt;*&gt;&gt; {
    testRuns.all {
        tasks.named("${target.name}${name.capitalized()}") {
            val files = files("src/commonTest/resources")
            inputs.files(files.asFileTree)
            doFirst(objects.newInstance(CopyToTestWorkingDir::class, files))
        }
    }
}```
would avoid the issues mentioned above, but you'd still have no way to get rid of stale resources in the destination

basically if you want to have resources in that directory properly, it needs support upstream

for my K/N binaries and tests, I use an environment variable for to pass along the data directory. easy and no interference with any other infrastructure

I've definitely run across where this can be an issue. I had to hack together <https://github.com/Kotlin/dokka/issues/2977#issuecomment-1571550626|this workaround> to get Dokka working with Gradle 8.

What would you suggest as an alternative for these cases? There seems to be a gap between ideal Gradle methodology and some real world requirements.

For the framework use case, I could see creating a downstream task `assembleFrameworkWithResources` that depends on the framework build task, but then copies both the binary compilation and the resources to yet another path. But then that could easily be confused and another user expect the framework build task to have created a complete useable framework.

For the test task, the task dependency has to be upstream from the task that eventually runs the tests from the location the binary was built in and those tests expect the resources to be present there at runtime.

Also, creating these new tasks, whether upstream or downstream that depend on existing tasks of the same type always end up with the initial problem of not being able to be configured lazily. There are several framework build tasks for example for different architecture combinations. Whether the copy resources task is a dependency or depends on these existing tasks, they all need to be named uniquely and associated with the existing tasks.

for the Java ecosystem, you simply add to the test sourceset's resources and you don't need to care about anything else. no clobbering since there's multiple directories, all consumers use the same sources, etc.

Kotlin/Native needs to grow support and then none of this hackery is needed

Yes, JVM resources work great. Android mostly works, at least outside of KMP common resources. But all other Kotlin multiplatform targets don't have good support for resources currently.

So it seems like solutions just aren't ideal, in one way or another, when you need to add features to an existing plugin's tasks.

&gt; when you need to add features to an existing plugin's tasks
For that you just add `doFirst` or `doLast` actions to those tasks

That's a good point. I could use those APIs instead of creating new task dependencies. Thanks for the ideas and feedback.

You depend on what you need on the classpath from the build script of the convention plugin project.

Usually adding a dependency (_implementation_ or _runtimeOnly_) to the build.gradle of your convention plugin project (e.g. `buildSrc/build.gradle`) replaces that.

hm, odd. wonder why i'm not experiencing that, and still have to add buildscript{} in the downstream convention plugin consumer

ok, something is weird, no code changes(zero commits since last worked), suddenly gradle is not finding a dependency - clearly looking for it in the wrong repository - checked, the repository that has the right version is listed

any way to force it to resolve this specific dependency in that specific repo?

If that is the case, it's possible that repository is not accessible one way or another, so Gradle will try the next repository.

But you can also set the filtering:
<https://docs.gradle.org/current/userguide/declaring_repositories.html#sec:repository-content-filtering>

Apparently JFrog started redirecting JCenter to Maven Central. Many artifacts are missing.

e.g:
```curl -vv <https://jcenter.bintray.com/org/ysb33r/gradle/grolifant/0.17.0>
&lt; HTTP/1.1 301 Moved Permanently
&lt; Location: <https://repo1.maven.org/maven2/org/ysb33r/gradle/grolifant/0.17.0>```

that's breaking all our builds :man-facepalming:

Confirmed

<https://gradle-community.slack.com/archives/CA7UM03V3/p1687511840205859>

:disappointed: My resolution for the time being was to find another repo that had the artifact that was missing and add manual override of the pluginManagement/repositories - it is really ugly, but works or the moment

I guess jCenter was set to be read-only back in 2021 :confused:

Its annoying because some versions of dependencies exist only in JCenter :disappointed:

Yes, annoying like every of the past JCenter outages.
One point of why I would have never used it for serious publishing. :slightly_smiling_face:

If this is really the end of jCenter are Gradle going to commit to sourcing and hosting the missing artifacts, or should those of us with affected builds start trying to fix things directly? I would imagine 8 hours is enough to know if JFrog care to fix anything this time, even if it’s not enough to actually fix it?

I'm pretty sure that this was not intentional.
I personally am not going to change anything for the builds not working currently.

yeah… I’m holding until at least Monday unless I hear otherwise… but if it was accidental you would imagine JFrog would have made a mea culpa statement by now.

Why? They also never did that for the regular outages they had in the past. At least not that I recognized it. :smile:

Hanlon’s razor in action… maybe? To me a 301 redirect seems much more like intentionally taking the off-ramp than their usual neglect/ineptitude.

Well, my suspicion is, that they wanted to change the forwaring to MC for libs they don't have themselves to be redirects instead so that they do not go through their infrastructure and accidentally was made also for artifacts that are exclusively available on JCenter. But just a guess of course.

We've gotten word through back channels that this has been fixed (not sure what/why it changed yet)

I've checked a few URLs, could you check on your side too?

```$ http -v -F <https://jcenter.bintray.com/org/ysb33r/gradle/grolifant/0.17.0/grolifant-0.17.0.jar>
GET /org/ysb33r/gradle/grolifant/0.17.0/grolifant-0.17.0.jar HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: <http://jcenter.bintray.com|jcenter.bintray.com>
User-Agent: HTTPie/1.0.3



HTTP/1.1 200
Accept-Ranges: bytes
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: public,max-age=3600
Content-Disposition: attachment; filename="grolifant-0.17.0.jar"; filename*=UTF-8''grolifant-0.17.0.jar
Content-Length: 374092
Content-Type: application/java-archive
Date: Fri, 23 Jun 2023 18:37:48 GMT
ETag: 278ef7b9d9b786ed7ff7e99d8f2002f38c5f4678
Last-Modified: Sat, 02 May 2020 09:12:35 GMT
Via: 1.1 google
X-Checksum-Md5: 24cf04b62d7efbad85702f0bbb146f29
X-Checksum-Sha1: 278ef7b9d9b786ed7ff7e99d8f2002f38c5f4678
X-Checksum-Sha256: 3c9f99effe32c4718f73cb538ca1aaad4fd84a99c14104b5abe6d35a22043843



+-----------------------------------------+
| NOTE: binary data not shown in terminal |
+-----------------------------------------+```

I think we're still going to put out some more communication about this.  We were doing an analysis to see how many plugins were dependent on JCenter-only artifacts