https://gradle.com/ logo
Join Slack
Powered by
This message was deleted.
# community-support
s
This message was deleted.
m
The Micronaut Gradle plugins is a collection of various plugins, 
io.micronaut.application
, 
io.micronaut.lbrary
, 
io.micronaut.docker
, etc
those plugins react to the presence of other plugins. For example, they would configure differently if the shadow plugin is applied, or if the Kotlin plugin is applied
in the past, we've had issues because using TestKit, all classes are always on classpath, even for "optional" dependencies like those plugins
so what I've setup is a distinct project for functional tests, which does not use 
withPluginClasspath()
.
This project lets us check the behavior of our plugins if an optional plugin is not applied in a user build, for example.
to make this work, I need to have build scripts which use plugins as if they were downloaded from the plugin portal or Maven Central
so, as an input of the functional test tasks, I have a full repository, a Maven repository containing the locally built Micronaut Gradle plugins
so, in order to run functional tests, you can think of this, mentally: 1. publish all plugins to a temporary "build" repository, e.g 
publishAllPublicationsToBuildRepository
2. patch the build under test settings file to add a plugin repository pointing to that build repository
3. profit!
This works very well, except for one little thing
publishAllPublicationsToBuildRepository
will always publish, even if there were no changes to the project
as a consequence, the inputs of the functional test tasks always change, so they are always out-of-date
because it is annoying, I have put in place a very dirty trick, which I'd like to get rid of:
this is a monstruousity, inaccurate and all, but more annoying, this is incompatible with the configuration cache
does anyone else have a trick to make publishing no-op if nothing changed?
v
Why do the inputs for the test change? Is the publication timestamp somewhere written, or why?
m
it's a snapshot, so all the files have the timestamp in their names, and the Gradle module metadata also contains file names...
one thing I want to try is a dummy version number for publishing, which would be "fixed", but it greatly complicates things.
v
Ah, right. 😕 Yeah, was about to suggest either that, or maybe use a NONE path sensitivity. But due to file names in GMM, you would also need to exclude GMM and then it might be wrong again. Or you make a copy of GMM, normalize the file names in it, use that as additional input and exclude the original GMM.
m
and this alone is super complicated. In addition, I have to forcefully clean the repository before every publication, this is a waste of time...
c
is it possible (reasonable) to adjust the plugin classpath to reflect the presence/absence of optional plugins, sidestepping this publishing ‘fun’?
m
I tried already. The pb is that as soon as you use 
withPluginClasspath
, you get a different classloader setup, and this alone breaks tests
c
oh ugh. 😢
m
everytime I think I'm smart, Gradle slaps me in the face
c
perhaps a mixture of the two approaches - don’t publish the plugins but build the logic to generate the correct buildscript/classpath entries in test scripts, reflecting the set of plugins that are available.
m
but how do I put the plugins on classpath if they are not published in a repository?
or you mean using 
buildscript { ... }
instead of 
plugins
?
c
correct
m
right but then I'm not testing the same thing anymore, since the classloader setup will be different
c
ugh yea. them darn classloaders.
m
maybe I could use an Ivy repo instead of Maven...
since there wouldn't be any timestamp
v
Are you sure? Usually you have the publication timestamp in the Ivy file. Not sure whether it is mandatory and whether Gradle puts it there though.
m
ah shit...
Copy code
<info organisation="io.micronaut.gradle" module="micronaut-minimal-plugin" revision="4.0.0-SNAPSHOT" status="integration" publication="20230317162336"/>
but if it's just a matter of replacing with a dummy timestamp in a file, could work...
v
But maybe you can normalize that?
yeah
m
why did I start this on a Friday?
c
yea, no need to get a headstart on Monday-type stuff…
m
for reference, monkey-patching:
Copy code
publications.withType(IvyPublication).configureEach {
        descriptor.withXml {
           asNode().info.@publication = "20230101000000"
        }
    }
👌 1
👍 1
it works 🎉
🙈 1
so basically, before testing we're always going to do an unnecessary "clean tmp repo", "publish to repo" dance, but then the test task, which takes several orders of magnitude longer to execute will be up-to-date!
🙌 1
noooooooooooooooooooooooooooooooooooooo
› :warning:cannot serialize object of type org.gradle.api.internal.artifacts.repositories.DefaultIvyArtifactRepository:clipboard:, a subtype of org.gradle.api.artifacts.repositories.ArtifactRepository:clipboard:, as these are not supported with the configuration cache. ?
slapped again
ffs
v
Why do you have the Ivy repo as CC input?
m
it is not
it's an input of the publication task
there was work made by the Gradle team to support this for Maven, but not Ivy
eternal sadness
😞 2
v
m
nope, won't use snapshots
v
Not for production of course, I just meant to try it 🙂
m
I am in the process of making it work with Maven by creating additional publications with a dummy version
👌 1
I'm really trying hard not to scream
Multiple publications with coordinates 'io.micronaut.gradlemicronaut minimal plugin4.0.0-DUMMY' are published to repository 'project'. The publications 'localComponentPluginPluginMarkerMaven' in project ':micronaut-minimal-plugin' and 'localPluginMaven' in project ':micronaut-minimal-plugin' will overwrite each other!
apparently Gradle has special handling of the marker
so if I copy the marker publication, it's not published at the same coordinates 😞
v
o_O
m
wait, it's probably me...
or not...
ah ah... good old:
Copy code
project.afterEvaluate {
                    groupId = pub.groupId
                    artifactId = pub.artifactId
                }
why is it so complicated? 😞
oh there's a magic flag, 
alias = true
v
On publication? Interesting
TIL
m
it just gets me one step further, because the marker which is now published on my local repo doesn't have the dependency on the plugin artifact
v
🙈
m
now it works! both with a Maven repo and configuration cache
that was not fun
a
sorry, I skimmed the whole thread so maybe I missed something, but I managed to get TestKit + a Maven publishing task + a SNAPSHOT version working. the problem is that if you set up a ‘regular’ Maven repo to a local dir, but the version is a -SNAPSHOT, then Gradle will always publish a new version with a new timestamp. However, the same is not true for publishToMavenLocal! In this case, Gradle will not generate a timestamp. So the trick I used was to override the location of Maven Local via a system property https://github.com/adamko-dev/kotlin-binary-compatibility-validator-mu/blob/d9260379ba095992f753da02c19d1153d263738c/buildSrc/src/main/kotlin/buildsrc/conventions/maven-publish-test.gradle.kts
❤️ 1
👍 2
v
Yay, nifty
m
interesting. Not that I like the idea of using Maven Local much, though, scares me too much 😄
a
it uses the Maven Local task, but not the actual Maven Local 
./m2/repository
directory - all artifacts get published to a project-local dir
v
The main problem is the broken-by-design 
mavenLocal
that Maven rapes itself. if you have a didicated one for such tests it is probably ok.
a
I’ve been rolling around the idea of publishing as a plugin. I thought it was a much more robust way of testing with TestKit than the classpath, since it also tests that the Maven metadata and Plugin Marker artifact gets published correctly.
👍 2
m
I understand the override. However this only works because you override the system property at runtime, wihch is kind of fragile, but arguably all solutions we find are fragile
a
yeah indeed - it could get very messy if another process tries to set the same property to something else.
m
also I need to make sure that my solution works if the jar actually changes (because "fixed" versions are cached, so my tmp jar could end up in Gradle's cache)
v
Then publish to regular 
mavenLocal
and have a post-processing or followup task that copies your exact artifacts out into a new directory. 🙂
m
this would work if you knew exactly what artifacts are published where
but it's not that obvious with plugins (e.g markers, multple plugins per project ...)
a
they’re published in 
$rootDir/build/test-maven-repo
:)
v
Yeah, if you override the property. My suggestion was to not override the property, but then extract from Standard Maven Local the published artifacts into a dedicated test repo. :-)
a
oh sorry, I was skimming again. I gotcha 👍
m
Copy code
buildscript.dependencies.components.all {
                throw new GradleException("Eh!")
                if (id.version.endsWith("-DUMMY")) {
                    throw new GradleException("\$id")
                    changing = true
                }
            }
doesn't throw in 
settings.gradle
...
a
using Maven Local wouldn’t be necessary if it was possible to tell Gradle not to append a timestamp for a publication with a SNAPSHOT version. I’m sure I saw an issue for it…
v
Hm, actually I have an idea how to do it without edge-cases and without relying on a system property, by combining the mentioned bits here
Let me try quickly
Ah, damn, why is the logic not in 
DefaultMavenLocalArtifactRepository
? Could have been so easy:
Copy code
publishing {
    repositories {
        mavenLocal {
            name = "foo"
            setUrl(layout.buildDirectory.dir("foo-repo"))
        }
    }
}
😭 1
m
looks like I managed to make it work, at least locally. testing on CI: https://github.com/micronaut-projects/micronaut-gradle-plugin/pull/675
⭐ 1
a
ah, replace SNAPSHOT with DUMMY. Good idea.
m
except that the metadata files still have 
SNAPSHOT
, was painful
and it works because I'm using a 
file
repository, so the artifacts are not cached
a
would it work if you created a specific MavenPublication and manually set the version to be DUMMY?
Copy code
publishing {
    publications.create<MavenPublication>("test") {
        this.version = "CUSTOM"
    }
}
m
nope, I think it's because in case of multiple publications Gradle doesn't know which one to use to replace dependencies in the generated pom/module files
a
here’s another attempt that seems to work in one of my Gradle Plugin projects. No MavenLocal hacks, but it does rename the version (similar to @melix’s example) it creates a separate ‘test’ publication, and then following Restricting publications to specific repositories restricts this test publication tasks to only be enabled for the test-repo, and the ‘main’ publication task is only enabled for the ‘main’ publishing repo (plus some other tweaks)
maven-test-publish_gradle_kts.kt
I think I’ve figured out a complicated, but technically non-hacky way to do this in a generic way. 1. define a file-based Maven repo that publishes to a temp dir 
build/tmp/.maven-publish-test
2. for each MavenPublication, create a task that creates a file containing the MavenPublication’s artifacts (
MavenPublication.getArtifacts()
), and compute an md5 checksum, and save it to disk 3. in any 
PublishToMavenRepository
task that publishes to 
build/tmp/.maven-publish-test
, again compute the checksum of the current artifacts, and inside an 
onlyIf {}
compare it against the checksums computed in step 2 (which won’t exist the first time round) 4. after the 
PublishToMavenRepository
task that publishes to 
build/test-publish-temp
has completed, • sync the files from 
build/tmp/.maven-publish-test
to 
build/maven-publish-test
, • re-compute the file-based checksums, • delete the contents of 
build/tmp/.maven-publish-test
(to prevent aggregating SNAPSHOT spam) And then in test projects, define the repo (the path can also be injected via a env/system/gradle property). It’s best to limit the group so Gradle doesn’t try to resolve a ‘live’ version from another repository.
Copy code
repositories {
  maven("/path/to/project/build/maven-publish-test/") {
    mavenContent {
      includeGroup("my.group")
    }
  }
}
The result is that even with a SNAPSHOT version the 
onlyIf {}
check will block publishing until the artifacts change, It doesn’t matter if Gradle appends a timestamp to the publication. This also works across subprojects, given a couple of consumer/provider Configurations.
Copy code
dependencies {
  mavenPublishTest(projects.core)
  mavenPublishTest(projects.plugins.base)
}

tasks.test {
  dependsOn(tasks.updateMavenTestRepo)

  environment(
    "MAVEN_PUBLISH_TEST_REPO" to mavenPublishTest.testMavenRepo.asFile.get().invariantSeparatorsPath
  )

}
I’ll publish and share the repo soon
13 Views
Open in Slack
PreviousNext
Powered by