From mobile apps to microservices, from small startups to big enterprises, Gradle helps teams build, automate and deliver better software, faster.

Gradle Community

Hi everyone, what could be the best way to run a few jobs parallelly in a Transform Action? :thinking_face: 

docs im reffering to
<https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.dsl.RepositoryHandler.html#org.gradle.api.artifacts.dsl.RepositoryHandler:google()>

hm. okay. so having both

```google()

and

maven { url '<https://maven.google.com>' }```
is redundant.

teammate is saying its not redundant. and since they are different urls. i thought they were right

<http://maven.google.com|maven.google.com> simply points to <http://dl.google.com|dl.google.com>, <https://maven.google.com/com/android/tools/sdk-common/30.4.2/sdk-common-30.4.2.pom> redirects to <https://dl.google.com/dl/android/maven2/com/android/tools/sdk-common/30.4.2/sdk-common-30.4.2.pom> for example

Do you have subprojects with the same name? (For example, if you had `:alpha-modules:foo` and `:blah:test:foo` there are two subprojects are named `foo`.

<https://github.com/gradle/gradle/issues/847>

yes, we have a pattern for subprojects that they're either:
• `:library:foo:public`  
• `:library:bar:public`  
• `:feature:baz:public`
• `:feature:foo:public` 
• etc.

I have no idea if it's related, but sometimes Gradle gets confused if the group + name of two subprojects are the same.

You could quickly try this workaround to make the group of all subprojects distinct: <https://github.com/gradle/gradle/issues/847#issuecomment-1159365667>

Thanks for the feedback! My issue might be slightly different. We aren't running into circular dependencies on our builds

Hi Ivan, if the module is not reachable from the `app` module, what type of task makes run the isolated module?

Hey Iñaki! in our most recent case the build failed for task
```:app:processDebugResources```
no tasks for the new module ran

this task was executed from
```:app:assembleDebug```

Almost every time you use an explicit `dependsOn` (except if on the left-hand side is a lifecycle task) you are doing something wrong, and also if you configure paths manually.
Instead you should always wire outputs and inputs together to get implicit task dependencies where needed.
If you for example have a task that packages a source jar, it also needs the dependency on the generation task.
Or if you have some static code analysis task, it also needs the dependency on the generation task.
.....

```sourceSets {
    main {
        java {
            srcDir(generateDefaultVersion)
        }
    }
}```

Well, almost, you need to change the define outputs of your task a bit, so that the right directory is the task output

So in your case `"$buildDir/generated/src"` should be the output directory of the task

As long as only that task generates into that directory

Latest when you get multiple generation tasks, you should have one output directory per task

:+1: adding src dir using task output did the trick

got another "similar" failure though -&gt; now it complains about `publishPluginMavenPublicationToMavenLocal` and `generateMetadataFileForPluginMavenPublication` missing dependency on `publishPluginJar` :disappointed:

trying to find out why it wants to publish to maven local

my bad that was leftover of some old config

<@URQ797WRW> That’s really interesting to me what you said about `dependsOn` is in most cases wrong approach. In our code which, has seen very early versions of gradle, we have tons of custom tasks dependencies specified using `dependsOn` . Some of them were like that from the very begining, some dependOn have been added recently because of the Gradle complaining about implicit dependencies :grimacing:
These are mostly related to various code generators we have. For example
```tasks.all { task -&gt;
...
match = task.name =~ /^generate(.*)Sources$/
  if (match &amp;&amp; ['Debug', 'Release'].contains(match.group(1))) {
    task.dependsOn "many"
    task.dependsOn "different"
    task.dependsOn "code"
    task.dependsOn "generator"
    task.dependsOn "tasks"
    task.dependsOn "here"
    return
  }
}
...
}```
How to know what is the input for the `generateDebugSources` task so output of our custom task could be input there. Is there a way to inspect any gradle task to get some more insights on its inputs and outputs?

Yeah, in our project that started pre Gradle 1.0 and grew organically, it is similar and needs quite some cleanup. Back in the days you also needed the dependencies as no implicit ones existed.

Let's start with `tasks.all { ... }`. This innocent looking thing already completely disables [task configuration avoidance](<https://docs.gradle.org/current/userguide/task_configuration_avoidance.html|https://docs.gradle.org/current/userguide/task_configuration_avoidance.html>), as it requires each and every task to be realized and configured for each and every build.

If `generateDebugSources` has no task actions and only exists to trigger other tasks, for example that you can generate everything after a clean checkout, that is exactly what is called lifecycle task where explicit task dependencies are the way to go. If you on the other hand depend on `generateDebugSources` for example from `compileJava`, then it is bad, as then all other tasks needing sources do not have that dependency like source jar task, javadoc task, static code analysis tasks and so on. Instead the actual tasks that generate code should have their output properly defined and then configured as source directory for the source directory set, which then carries the task dependency automatically for all tasks requesting sources and so on.

Thanks, I am aware of the configuration task avoidance feeature and that’s another thing that I want to improve in our buildsystem, but I learned we will not benefit from it since we have a plugin that compiles native code as part of the build process, and as far as I am concerned c/c++ gradle plugins are not supporting configuration cache at all.

When it comes to to `generateDebugSources` I think is a task from Android Gradle Plugin. Not sure how to hook into it in any other way.

&gt; Thanks, I am aware of the configuration task avoidance feeature and that’s another thing that I want to improve in our buildsystem, but I learned we will not benefit from it since we have a plugin that compiles native code as part of the build process.
Don't confuse <https://docs.gradle.org/current/userguide/task_configuration_avoidance.html|task configuration avoidance> and <https://docs.gradle.org/current/userguide/configuration_cache.html|configuration cache>, those are completely different things.
The configuration cache can skip the whole configuration phase from init scripts, over settings scripts up to configuring tasks, if an entry in the cache is available and directly start with the execution phase.
The task configuration avoidance kicks in when the configuration phase actually has to be done and when used properly causes only the tasks that are going to be executed to be realized and configured, saving tremendous amount of time already in bigger builds with many projects and tasks.

&gt; and as far as I am concerned c/c++ gradle plugins are not supporting configuration cache at all
That's correct, CC is also not stable yet.
It will become stable with 8.1 hopefully.
Whether the native plugins are adapted until then I don't know.
But hopefully at least the Nokee plugins will adapt it: <https://github.com/nokeedev/gradle-native/issues/87>

&gt; When it comes to to `generateDebugSources` I think is a task from Android Gradle Plugin. Not sure how to hook into it in any other way.
I'm not an Android developer, so no idea, but <https://gradle-community.slack.com/archives/CA745PZHN/p1675803875548149?thread_ts=1675784054.931979&amp;cid=CA745PZHN> last post has some information about how to properly register a code generation task in Android builds.

I will give it a read, thanks for such a detailed explanation. You should be paid for the contribution you do here in this slack workspace :wink:

Here's my extension, with a polymorphic container:

```abstract class MyPluginExtension @Inject constructor(
  objects: ObjectFactory
) {
  val userProvidedConfig: ExtensiblePolymorphicDomainObjectContainer&lt;UserProvidedConfig&gt; =
    objects.polymorphicDomainObjectContainer(UserProvidedConfig::class)
}```
And here's some simple user-provided data that I want to capture. It's just a string property, that gets encoded later on. Users could provide their own subclass of `UserProvidedConfig` if they needed.

```abstract class UserProvidedConfig @Inject constructor (
  private val configId: String
): Named {
  abstract fun encode(): String

  override fun getName(): String = configId
}

abstract class StringConfig : UserProvidedConfig("string-config") {

  @get:Input
  abstract val someValue: Property&lt;String&gt;

  override fun encode(): String = someValue.orNull.toString()
}```
So I create the extension, and register the `NamedDomainObjectFactory` for the subclass

```val myExtension = extensions.create&lt;MyPluginExtension&gt;("myExtension").apply {
  userProvidedConfig.registerFactory(StringConfig::class.java) { objects.newInstance() }
}```
Now, I want to re-use this data in a task. It also has a `ExtensiblePolymorphicDomainObjectContainer`

```abstract class MyTask @Inject constructor(
  objects: ObjectFactory,
) : DefaultTask() {
  val userProvidedConfig: ExtensiblePolymorphicDomainObjectContainer&lt;UserProvidedConfig&gt; =
    objects.polymorphicDomainObjectContainer(UserProvidedConfig::class)
}```
But the problem is when I try to register the task, I want to re-use the values from the extension.

```val myTask = tasks.register&lt;MyTask&gt;("myTask") {

  // re-use the extension values
  userProvidedConfig.addAllLater(provider { myExtension.userProvidedConfig })
}```
But this leads to an error: `this type is not known to this container. Known types are: (None)`

I've tried accessing the createable types using the internal Gradle API, but I can't figure out how to do this type-safely

```  val createableTypes =
    (myExtension.userProvidedConfig as org.gradle.api.internal.DefaultPolymorphicDomainObjectContainer&lt;UserProvidedConfig&gt;).createableTypes

  createableTypes.forEach { createableType -&gt;
    userProvidedConfig.registerFactory( 
// ERROR None of the following functions can be called with the arguments supplied.
      createableType,
      NamedDomainObjectFactory { name -&gt;
        myExtension.userProvidedConfig
          .containerWithType(createableType)
          .create(name)
      }
    )
  }```


build.gradle.kts.kt

Couldn't you just set the container, instead of using `addAllLater`?

What do you mean, set the container? I dont think there's a `.set()` function

```abstract class MyTask @Inject constructor(
  objects: ObjectFactory,
  userProvidedConfig: ExtensiblePolymorphicDomainObjectContainer&lt;UserProvidedConfig&gt;
) : DefaultTask() {
  val userProvidedConfig = userProvidedConfig
}```


ahh in the constructor. That could work… I’ll have a look, thanks.

It would be more convenient to be able to copy a `ExtensiblePolymorphicDomainObjectContainer` instance, because then it means tasks could be registered arbitrarily, without needing the container. But that’s probably rare in my case.

my usecase is actually a little more complicated. I have a NamedDomainObjectContainer, and each element in that container also needs the ExtensiblePolymorphicDomainObjectContainer.

Do you know if a NamedDomainObjectContainer supports creating an element and passing in constructor arguments? I couldn't find anything.

Yes, you can give a factory to `ObjectFactory.domainObjectContainer()`

thanks! That helped. I'm pretty sure it's working, finally :sob:

<https://github.com/adamko-dev/dokkatoo/pull/36/files>

Hard to say without a crystal ball.
Look at the `--debug` log to see whether someone is going on or what happened last, and take a thread dump and analyze it to see where things are hanging if they really are.

Thanks.  Found a network connection problem there, today  I added a global repository in the `~/.gradel/init.gradle`.

```allprojects {
	buildscript {
		repositories {
			mavenLocal()
//			maven {
//				url "<https://myhost:5443/nexus/content/groups/public/>"
//				credentials {
//					username "hantsy"
//					password "XXX"
//				}
//			}
			mavenCentral()
			google()
		}
	}

    repositories {
		mavenLocal()
//		maven {
//			url "<https://myhost:5443/nexus/content/groups/public/>"
//			credentials {
//				username "hantsy"
//				password "XXX"
//			}
//	   }
		mavenCentral()
		google()
    }

}```

The private maven repository(with credentials) is always connection time out,  but the same credentials and url,  it is working well in the m2/settings.xml.

Actually this snippet is a very bad idea.
A build should configure the repositories it needs.
Otherwise the builds only work, or work differently, on other machines than on your machine.
Unless you always ever work with local projects that never see a different machine than yours.
Then it would just be bad practice.

Besides that `mavenLocal()` per-se, no matter where used, is a bad idea, especially as first repository and without content filter.
It is broken by design (by Maven, not by Gradle), it will make your builds slow, unreliable, and flaky, and might miss Gradle Module Metadata for dependencies which could provide better dependency resolution as it is a richer model than POMs.

The private  maven repository is just a mirror of the *central maven repo* and some open source repo, such as jboss, jakarta repos, only used to speed up download in a team and save time for work.

Always using Maven central is terrible, it could block my IP according to the maven principle. And in China, the download rate from maven central only 5k-10k,  it is every slow.

Gradle design is good, like a piece of art, but its slow build and deps download is terrible for Chinese, so most of time I have to use Maven even it is design is not ideal.

&gt; The private  maven repository is just a mirror of the *central maven repo* and some open source repo, such as jboss, jakarta repos, only used to speed up download in a team and save time for work.
Well, as I said, by that snippet you modify all builds to add those repositories first.
So you change the behavior of the builds as you change the repositories that are available and the order too. So unless all builds you ever use have these repositories anyway, you might easily get into trouble.

And with `mavenLocal()` anyway, especially if you also use Maven, because if you resolve a dependency with maven, it is stored to `mavenLocal()`, but only the files that were actually downloaded, so usually you now have incomplete and thus corrupt entries in that repository that is also misused as a cache directory by Maven. If you then try to use that dependency from Gradle, it sees the broken state but assumes it is good. This might go well if you are lucky, this might break in a loud and obvious way, or this might break in subtle ways you don't see right away and can spend a tremendous amount of time to find out why something is not working as expected.

Besides that it makes your builds slow and not only unreliable and flaky.
And to reuse the artifacts that are present in `mavenLocal()` to save the download time, you don't need that declaration, Gradle already automatically takes an artifact from there if it is present with the expected checksum as far as I remember.

&gt; Always using Maven central is terrible, it could block my IP according to the maven principle.
Why should it block your IP?
According to what principle?
Especially as Gradle does have a proper cache and does not re-download a dependency it already has in its cache.

&gt; And in China, the download rate from maven central only 5k-10k, it is every slow.
Yeah, it's fine to use a caching proxy that multiple people can use then.
But you shouldn't just prefix each and every build with a list of repositories it maybe shouldn't have with an init script.
You could for example add logic to your init script that checks whether Maven Central is present as repository and then replaces it by your proxy or something like that.

I think something like this should work:
```def replaceMavenCentralWithProxy = { repository -&gt;
    if (!(repository instanceof MavenArtifactRepository)) {
        return
    }
    if (repository.url.host == '<http://repo.maven.apache.org|repo.maven.apache.org>') {
        repository.tap {
            url = '<https://myhost:5443/nexus/content/groups/public/>'
            credentials {
                username "hantsy"
                password "XXX"
            }
        }
    }
}

def versionParts = gradle.gradleVersion.split('\\.')
def majorVersion = versionParts[0] as int
def minorVersion = versionParts[1].split('-')[0] as int

def handleSettings = { settings -&gt;
    if (majorVersion &gt; 3 || (majorVersion == 3 &amp;&amp; minorVersion &gt;= 5)) {
        settings.pluginManagement {
            if (majorVersion &gt; 4 || (majorVersion == 4 &amp;&amp; minorVersion &gt;= 9)) {
                repositories.configureEach(replaceMavenCentralWithProxy)
            } else {
                repositories.all(replaceMavenCentralWithProxy)
            }
        }
    }
    if (majorVersion &gt; 6 || (majorVersion == 6 &amp;&amp; minorVersion &gt;= 8)) {
        settings.dependencyResolutionManagement {
            repositories.configureEach(replaceMavenCentralWithProxy)
        }
    }
}

if (majorVersion &gt;= 6) {
    beforeSettings(handleSettings)
} else {
    settingsEvaluated(handleSettings)
}

allprojects {
    buildscript {
        if (majorVersion &gt; 4 || (majorVersion == 4 &amp;&amp; minorVersion &gt;= 9)) {
            repositories.configureEach(replaceMavenCentralWithProxy)
        } else {
            repositories.all(replaceMavenCentralWithProxy)
        }
    }
    if (majorVersion &gt; 4 || (majorVersion == 4 &amp;&amp; minorVersion &gt;= 9)) {
        repositories.configureEach(replaceMavenCentralWithProxy)
    } else {
        repositories.all(replaceMavenCentralWithProxy)
    }
}```


Hey <@U04NU3YN79Q>! :wave: The only official recommendations we have on project naming are here: <https://docs.gradle.org/current/userguide/multi_project_builds.html#naming_recommendations>

They do not address your specific concern, however.

For that, my *personal* recommendation would be to use the `projectName-x` convention as you described. In fact, we can look at a few open source projects that demonstrate the same convention:

• <https://github.com/spring-projects/spring-framework>
• <https://github.com/junit-team/junit5>
But there are also many projects that do not follow that convention.

It should be noted that you can always override the convention of using the project name only as the published artifactId. For example, you could override the convention to be `{rootProjectName}-{projectName}`.

See the Gradle documentation for more info on configuring the publishing plugin: <https://docs.gradle.org/current/userguide/publishing_maven.html>

But that would then make problem if the project is consumed in a composite build, wouldn't it? :confused:
An alternative would be, that you do not publish at `&lt;my-group&gt;:...` but at `&lt;my-group&gt;.projectName:...`.

&gt; But that would then make problem if the project is consumed in a composite build, wouldn't it?
Could you describe a scenario in which it would cause a problem? :thinking_face:

&gt; An alternative would be, that you do not publish at `&lt;my-group&gt;:...` but at `&lt;my-group&gt;.projectName:....`.
Yeah, that would work too!

&gt; Could you describe a scenario in which it would cause a problem?
With "problem" I mean you cannot simply include it, but need to use manual dependency substitution, which I personally see as work-around for badly configure builds. :smile:

Ah, yes. I understand now. :+1:

Indeed if you need to use composite builds it would not be a good option to deviate from Gradle's artifactId convention for published artifacts.

Even if you don't, someone else using your library might want to use latests code to test or fix something, so just don't mess with it. :smile: (it being changing artifact coordinates on publishing)

Yes. It's usually not a good idea to deviate from Gradle conventions. :sweat_smile:

…I might have to add the project name to the group then. At least that makes some semantic sense. Would that cause issues with Central? With their group verification policies?

No, if you own `foo.bar` you also own all `foo.bar.baz`, `foo.bar.baz.bam` and so on.

Thanks, that's reassuring. Sounds like it's the best solution, then. Thanks.

To clarify, in case the discussion on changing the published `artifactId` introduced any confusion, you could still use `&lt;my-group&gt;:projectName-x:&lt;version&gt;` if for some reason a consistent group is important to you. If not, then Vampire's suggestion is certainly less verbose.

I don't really care about the group as long as it doesn't cause problems with Central. Included builds are important for me though.

You could write your own test engine for JUnit Platform, then any tool supporting JUnit 5 also supports your tests. (It is easier for narrow cases than it sounds :smile:)

hmmmm okay…

So I could implement the TestEngine interface (example: <https://github.com/junit-team/junit5/blob/r5.9.2/documentation/src/test/java/example/CustomTestEngine.java>) and then throw a ‘test failed exception’

and then figure out a way to JavaExec launch a local class?

I assume at some point I would need to execute the TestEngine from within a Test task

That is what JUnit platform does for you.
If you for example have the custom test engine in your test source set and did not restrict which tests engines should run, it is automatically run like any other test engine.

ahhh, but I don’t want to define the TestEngine from within a source set. I want to have a Gradle task that does this

So I want to dynamically define a Gradle task that will produce a ‘TestDescriptor’ so any <https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/testing/TestListener.html|TestListener> will be able to pick it up

Make a dedicated source set with your test engine, make a dedicated `Test` task running that source set.
(Or actually use the JVM Test Suites plugin to let Gradle define all that for you, except for the actual test engine)

dynamic-test_gradle_kts.kt

Urgh, that's not what I meant.
But well, if it works for you and you are fine with using internal API. :ok_hand:

I understood what you mean, but it wasn’t what I was asking for :)

it’s nice to be able to dynamically define a test without needing JUnit or an additional source set, because perhaps the subproject I want to test isn’t a Java project

The Micronaut Gradle plugins is a collection of various plugins, `io.micronaut.application`, `io.micronaut.lbrary`, `io.micronaut.docker`, etc

those plugins react to the presence of other plugins. For example, they would configure differently if the shadow plugin is applied, or if the Kotlin plugin is applied

in the past, we've had issues because using TestKit, all classes are always on classpath, even for "optional" dependencies like those plugins

so what I've setup is a distinct project for functional tests, which does _not_ use `withPluginClasspath()`.

This project lets us check the behavior of our plugins if an optional plugin is not applied in a user build, for example.

to make this work, I need to have build scripts which use plugins as if they were downloaded from the plugin portal or Maven Central

so, as an _input_ of the functional test tasks, I have a _full repository_, a Maven repository containing the locally built Micronaut Gradle plugins

so, in order to run functional tests,  you can think of this, mentally:
1. publish all plugins to a temporary "build" repository, e.g `publishAllPublicationsToBuildRepository`
2. patch the build under test settings file to add a plugin repository pointing to that build repository

This works very well, except for _one little thing_

`publishAllPublicationsToBuildRepository` will _always_ publish, even if there were no changes to the project

as a consequence, the inputs of the functional test tasks always change, so they are always out-of-date

because it is annoying, I have put in place a very dirty trick, which I'd like to get rid of:

<https://github.com/micronaut-projects/micronaut-gradle-plugin/blob/d9c2ef00515aa6b0f4306c10666025212acf8772/buildSrc/src/main/groovy/io.micronaut.internal.build.gradle-plugin.gradle#L57-L72>

this is a monstruousity, inaccurate and all, but more annoying, this is incompatible with the configuration cache

does anyone else have a trick to make publishing _no-op_ if nothing changed?

Why do the inputs for the test change?
Is the publication timestamp somewhere written, or why?

it's a snapshot, so all the files have the timestamp in their names, and the Gradle module metadata also contains file names...

one thing I want to try is a dummy version number for publishing, which would be "fixed", but it greatly complicates things.

Ah, right. :confused:
Yeah, was about to suggest either that, or maybe use a NONE path sensitivity. But due to file names in GMM, you would also need to exclude GMM and then it might be wrong again.
Or you make a copy of GMM, normalize the file names in it, use that as additional input and exclude the original GMM.

and this alone is super complicated. In addition, I have to forcefully clean the repository before every publication, this is a waste of time...

is it possible (reasonable) to adjust the plugin classpath to reflect the presence/absence of optional plugins, sidestepping this publishing ‘fun’?

I tried already. The pb is that as soon as you use `withPluginClasspath`, you get a different classloader setup, and this alone breaks tests

everytime I think I'm smart, Gradle slaps me in the face

perhaps a mixture of the two approaches - don’t publish the plugins but build the logic to generate the correct buildscript/classpath entries in test scripts, reflecting the set of plugins that are available.

but how do I put the plugins on classpath if they are not published in a repository?

or you mean using `buildscript { ... }` instead of `plugins`?

right but then I'm not testing the same thing anymore, since the classloader setup will be different

maybe I could use an Ivy repo instead of Maven...

Are you sure?
Usually you have the publication timestamp in the Ivy file.
Not sure whether it is mandatory and whether Gradle puts it there though.

```  &lt;info organisation="io.micronaut.gradle" module="micronaut-minimal-plugin" revision="4.0.0-SNAPSHOT" status="integration" publication="20230317162336"/&gt;```

but if it's just a matter of replacing with a dummy timestamp _in_ a file, could work...

yea, no need to get a headstart on Monday-type stuff…

for reference, monkey-patching:
```publications.withType(IvyPublication).configureEach {
        descriptor.withXml {
           asNode().info.@publication = "20230101000000"
        }
    }```

<https://ge.micronaut.io/s/gnoskbnjhoe5e/timeline>

so basically, before testing we're always going to do an unnecessary "clean tmp repo", "publish to repo" dance, but then the test task, which takes several orders of magnitude longer to execute will be up-to-date!

`› :warning:cannot serialize object of type org.gradle.api.internal.artifacts.repositories.DefaultIvyArtifactRepository:clipboard:, a subtype of org.gradle.api.artifacts.repositories.ArtifactRepository:clipboard:, as these are not supported with the configuration cache. ?`

Why do you have the Ivy repo as CC input?

there was work made by the Gradle team to support this for Maven, but not Ivy

Try with 8.1 snapshot build: <https://github.com/gradle/gradle/issues/13469>

Not for production of course, I just meant to try it :slightly_smiling_face:

I am in the process of making it work with Maven by creating additional publications with a dummy version

&gt; Multiple publications with coordinates 'io.micronaut.gradle:micronaut-minimal-plugin:4.0.0-DUMMY' are published to repository 'project'. The publications 'localComponentPluginPluginMarkerMaven' in project ':micronaut-minimal-plugin' and 'localPluginMaven' in project ':micronaut-minimal-plugin' will overwrite each other!


apparently Gradle has special handling of the marker

so if I copy the marker publication, it's not published at the same coordinates :disappointed:

ah ah... good old:
```project.afterEvaluate {
                    groupId = pub.groupId
                    artifactId = pub.artifactId
                }```

the gift which keeps on giving: <https://ge.micronaut.io/s/nxehccut4e254/failure#1>

it just gets me one step further, because the marker which is now published on my local repo doesn't have the dependency on the plugin artifact

now it works! both with a Maven repo and configuration cache

sorry, I skimmed the whole thread so maybe I missed something, but I managed to get TestKit + a Maven publishing task + a SNAPSHOT version working.

the problem is that if you set up a ‘regular’ Maven repo to a local dir, but the version is a -SNAPSHOT, then Gradle will always publish a new version with a new timestamp. However, the same is not true for publishToMavenLocal! In this case, Gradle will _not_ generate a timestamp.

So the trick I used was to override the location of Maven Local via a system property

<https://github.com/adamko-dev/kotlin-binary-compatibility-validator-mu/blob/d9260379ba095992f753da02c19d1153d263738c/buildSrc/src/main/kotlin/buildsrc/conventions/maven-publish-test.gradle.kts>

interesting. Not that I like the idea of using Maven Local much, though, scares me too much :smile:

it uses the Maven Local task, but not the actual Maven Local `./m2/repository` directory - all artifacts get published to a project-local dir

The main problem is the broken-by-design `mavenLocal` that Maven rapes itself.
if you have a didicated one for such tests it is probably ok.

I’ve been rolling around the idea of publishing as a plugin. I thought it was a much more robust way of testing with TestKit than the classpath, since it also tests that the Maven metadata and Plugin Marker artifact gets published correctly.

I understand the override. However this only works because you override the system property _at runtime_, wihch is kind of fragile, but arguably all solutions we find are fragile

yeah indeed - it could get very messy if another process tries to set the same property to something else.

also I need to make sure that my solution works if the jar actually changes (because "fixed" versions are cached, so my tmp jar could end up in Gradle's cache)

Then publish to regular `mavenLocal` and have a post-processing or followup task that copies your exact artifacts out into a new directory. :slightly_smiling_face:

this would work if you knew exactly what artifacts are published where

but it's not that obvious with plugins (e.g markers, multple plugins per project ...)

they’re published in `$rootDir/build/test-maven-repo` :)

Yeah, if you override the property.
My suggestion was to not override the property, but then extract from Standard Maven Local the published artifacts into a dedicated test repo. :-)

oh sorry, I was skimming again. I gotcha :+1:

```buildscript.dependencies.components.all {
                throw new GradleException("Eh!")
                if (id.version.endsWith("-DUMMY")) {
                    throw new GradleException("\$id")
                    changing = true
                }
            }```
doesn't throw in `settings.gradle`...

using Maven Local wouldn’t be necessary if it was possible to tell Gradle not to append a timestamp for a publication with a SNAPSHOT version. I’m sure I saw an issue for it…

Hm, actually I have an idea how to do it without edge-cases and without relying on a system property, by combining the mentioned bits here

Ah, damn, why is the logic not in `DefaultMavenLocalArtifactRepository`?
Could have been so easy:
```publishing {
    repositories {
        mavenLocal {
            name = "foo"
            setUrl(layout.buildDirectory.dir("foo-repo"))
        }
    }
}```

looks like I managed to make it work, at least locally. testing on CI: <https://github.com/micronaut-projects/micronaut-gradle-plugin/pull/675>

ah, replace SNAPSHOT with DUMMY. Good idea.

except that the metadata files still have `SNAPSHOT`, was painful

and it works because I'm using a `file` repository, so the artifacts are not cached

Of course it is.
But some notes:
• use Kotlin DSL :smile:
• do not use the legacy `apply ...`, but use `plugins { ... }` block to apply plugins
• Is  `PluginName` the id or the class name?

If it is the ID, then better namespace it, plugins without dot should be reserved for built-in plugins, and more importantly put it in quotes, so that it is a string.
If it is the class name, does it really have no package / did you forget to import it?

But that’s the point my gradle file is still in groovy. It’s quite big and would require a lot of effort to migrate to kotlin dsl fully.
The plugin doesn’t have package name. At least yet, I am experimenting I do not bother with such details :smile: PluginName is the class name.

Hm, then I would expect it to work.
Maybe try with adding a class name?

Or it is my brain being almost in holidays already. :smile:

Tried it already, didn’t work :disappointed:

Really? Isn't that just for top-level declarations in a `PluginName.kt`, but not if you have `abstract class PluginName : Plugin&lt;Project&gt;`?

I wonder if the fact that my buildSrc is a now a mix of groovy and kotlin may be problematic :thinking_face:

Tried exposing the plugin id using `gradlePlugins { }` block. In such case I got
`Could not find implementation class 'PluginName' for plugin 'my.plugin'`

Now I wonder if the plugin class is on the classpath

is `:buildSrc:compileKotlin` actually running? is the `.class` file present in `buildSrc/build/classes`? is it present in the jar under `buildSrc/build/`?

I’ve moved the plugin from `groovy` to `kotlin` source set. It can now be correctly applied. I won’t be able though to use a Task written in groovy in the `groovy` source set right? :wink:

I don't remember what the default order and dependency between the languages is, but you can change it. There is a snippet of what you need to do in the docs somewhere. :slightly_smiling_face:

Yes, I just realized I will have to change it. Looking for the snippet as apparently by default kotlin takes precedence over groovy.

It's not so much about precedence, but order and dependencies.
I wouldn't wonder if there is even no relationship either way by default.

<https://docs.gradle.org/current/userguide/building_java_projects.html#sub:compile_deps_jvm_lang>

Forgot to thank both of you <@URQ797WRW> and <@UABCKNV3K>. :pray: Using your help I eventually managed to achieve what I was trying to do :wink: Although I had to apply this one: <https://github.com/gradle/gradle/issues/11333> to force the kotlin compiler to take groovy compiler output to the kotlin compiler classpath. The code snippet there is also a bit outdated as `classpath` property of the `KotlinCompile` task shouldn’t be used anymore in favour of a new `libraries` property.

Getting the base URL of the repository as shown in build scans would be sufficient:

I'm a bit confused.
You say your worker api code should call X (X being the 3rd party library),
but it should not have a runtime dependency on X
and the users of your plugin should also not declare a dependency on X.
So where should X come from at runtime?

there’s a custom Configuration with a default dependency on X that will be resolved when the Task is triggered

Ah, that you meant with custom configuration, stupid me.
My brain is already on vacation. :smile:

And where do you want to / need to hide X from then?

if you add a dependency on `implementation("dev.adamko.dokkatoo:dokkatoo-plugin:0.0.4")` in buildSrc (or included plugin build) you’ll get an error, because some of the classes extend from dokka-core (which is the 3rd party lib)

I thought I fixed all those problems in v1.0.0 by using <https://github.com/Kotlin/binary-compatibility-validator|Kotlin/binary-compatibility-validator>, but I missed a class because of generic type erasure, so a `Property&lt;SomeDokkaCoreClass&gt;` requires a dependency on dokka-core.

I really want to prevent this from happening again by segregating the source sets, so that one is private and has a dependency on dokka-core, but another is public and has no dokka-core dependency

Ah, now I get what you mean.
You actually added an X class to your Public API and want to avoid that happening again by separating the concerns.
How about simply splitting the worker code into an own project / artifact that gets published too and depended upon from your plugin?

it’s really convenient being to publish one artifact to Gradle Plugin Portal so yes, multiple projects would probably be best. But I wanted to try and squeeze it into one.

&gt; What error exactly?
```&gt; Task :buildSrc:compileKotlin FAILED
e: Supertypes of the following classes cannot be resolved. Please make sure you have the required dependencies in the classpath:
    class dev.adamko.dokkatoo.dokka.parameters.DokkaSourceSetSpec, unresolved supertypes: org.jetbrains.dokka.DokkaConfigurationBuilder
Adding -Xextended-compiler-checks argument might provide additional information.

e: /projects/x/buildSrc/src/main/kotlin/buildsrc/convention/dokkatoo.gradle.kts: (21, 5): Cannot access 'org.jetbrains.dokka.DokkaConfigurationBuilder' which is a supertype of 'dev.adamko.dokkatoo.dokka.parameters.DokkaSourceSetSpec'. Check your module classpath for missing or conflicting dependencies

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':buildSrc:compileKotlin'.
&gt; A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   &gt; Compilation error. See log for more details

* Try:
&gt; Run with --info or --debug option to get more log output.
&gt; Run with --scan to get full insights.```

Would it maybe be an option to cover it with tests?
If you have tests without X, you should hopefully also get such failures.

I do have integration tests, but I suspect the Plugin DSL is being clever. It doesn’t seem to be a problem when the plugin is applied via its plugin ID

but actually looking at it, I’ve added X as an implementation dependency…

Ah, well, now looking at your error, it is the `compileKotlin` that fails.
So yeah, applying with ID then of course does not have that problem.

If you separate things but then want to have it in one jar it is imho always a bit unclean.
I'm not aware of a built-in way to separate code, but then have it in one jar, you always have to fiddle around if you need something like that.

I think I just need to change `implementation("dokka-core")` to `compileOnly("dokka-core")`

Maybe another option with tests would be some "architecture testing", there are tools or libs for it.
So that a test can e.g. verify that only classes in package Y use classes from X.

&gt; I think I just need to change `implementation("dokka-core")` to `compileOnly("dokka-core"`
Didn't you say it is a compile only dependency?

&gt; ^^^ have used archunit for exactly that.
Ah, yeah, tried to remember that name, thanks :slightly_smiling_face:

&gt; I thought it was, but I was mistaken :slightly_smiling_face:
But it wouldn't change anything, would it?
As long as you don't make it `api`, it will not be in the compile classpath of the consumer and thus give that error on compilation

ArchUnit gives me nightmares of working with someone who took ‘defensive programming’ to it’s absolute extreme maximum :grimacing:

&gt; and thus give that error on compilation
I want an error on compilation! The X classes should _only_ be used by the Worker API at runtime, which has an isolated classpath which contains X

&gt; ArchUnit gives me nightmares of working with someone who took ‘defensive programming’ to it’s absolute extreme maximum :grimacing:
like any tool it can certainly be abused.  If used selectively to enforce constraints that’s fine; if it becomes pedantic, well…

&gt; I want an error on compilation!
or maybe I don’t… I’m getting lost

At least it would be a build-time error :slightly_smiling_face:

Another option that does not separation-but-then-combination-trickery - but of course needs more time - is to just have a second source set that compiles the files that should not need X.
They then get compiled twice, but you would get a compilation error. :smile:

If  archunit gives you nightmares you could also have a simple text search check, that verifies the package of X is used nowhere except for the approved files.
archunit-light so to say. :smile:

you can throw an exception if you see any `rc` version via *`dependencyResolutionManagement`*

Sorry, with `configurations`
<https://docs.gradle.org/current/userguide/resolution_rules.html>

<https://kotlinlang.org/docs/whatsnew18.html#obligatory-check-for-jvm-targets-of-related-kotlin-and-java-compile-tasks>

This says kotlin targets JVM 8 or are you referencing something else?

and adding this:
```java.toolchain.languageVersion.set(JavaLanguageVersion.of(8))```
with Java 8 on my system does not fix it

Gradle only downloads files when necessary. This could already be during configuration phase for some if they are used there, or it could be as late as during actual execution of a task.

Can i reuse downloaded dependencies/files to .tmp directory for next CI builds after 1st "preparing" build?

Here you find infos about caching things for ephemeral agents: <https://docs.gradle.org/current/userguide/dependency_resolution.html#sub:ephemeral-ci-cache|https://docs.gradle.org/current/userguide/dependency_resolution.html#sub:ephemeral-ci-cache>

Thanks, but i can`t understand some moments. We are already has some runners for example for two projects with same stack. Before job execution downloads cache (from S3) (with .caches contents). If i retry job cache downloads again. But all downloading files seems like not in .caches and downloads into .tmp directory. Can i match .tmp contents with .caches?

What your can share and cache safely is described at the link I gave you. I'm on mobile so not sure about .tmp, but I think it - as its name suggests - is only temporary used while downloading files before stuffing them elsewhere like in the cache.

I saw several approaches. One is to publish catalog alongside with bom at the same coordinates with `org.example.project:project-bom`, another is to use `org.example.project:project-catalog`/`org.example.project:catalog`.
But mostly it's matter of personal preferences and/or org naming policy

Ok. So if I determined that some of my past uploaded build scans may have had sensitive information and I no longer have access to the link, there is nothing I can do?

If you do no view the build scan, it will be removed after 3 months of creation.

I have not viewed the majority of my scanes, but I would like to delete all of them including the ones I have viewed.

Or if you don't like that, `tasks.configureEach { ... }` and in there compare the parent class name with your string

that (`Class.forName`) actually did not work! even though `Class.forName("..")` finds the class

```tasks.withType(
    // kotlinx.kover.gradle.plugin.tasks.KoverVerifyTask is internal
    Class.forName("kotlinx.kover.gradle.plugin.tasks.KoverVerifyTask").javaClass as Class&lt;Task&gt;
)
    .configureEach {
        error("this never gets hit")
    }```


Maybe class loader problem? For example the `Class.forName` one comes from a different class loader than the one the tasks are made of.

why don’t 

```plugins {
    `jvm-test-suite`
}```

It's better, but still I would prefer to have unified way  using all plugins