https://gradle.com/ logo
Join Slack
Powered by
This message was deleted.
# community-support
s
This message was deleted.
Untitled
s
We use this code which works for us (just replace functional tests with integration tests): https://github.com/oss-review-toolkit/ort/blob/0bee05120ac5ad5acede533e1193c0caa786a5d2/build.gradle.kts#L219-L240
a
our integration tests sometimes skips running
Do you know why? Is this perhaps caused by Gradle noticing the inputs haven’t changed, so it skips the tests? (IntelliJ used to force the all tests to re-run, even if they hadn’t changed, but the latest version of IntelliJ will skip unchanged tests but it doesn’t display it in an intuitive manner)
e
Tests are run on Github actions with gradle caching, it simply said
> Task appintegrationTest FROM-CACHE
The inputs probably weren't changed, since 
integrationTest
doesn't depend on 
main
but I would like to tell gradle that such dependency exists
@Sebastian Schuberth I don't really see you doing anything to make functional tests depend on main
Copy code
// Associate the "funTest" compilation with the "main" compilation to be able to access "internal" objects from
    // functional tests.
    kotlin.target.compilations.apply {
        getByName("funTest").associateWith(getByName(KotlinCompilation.MAIN_COMPILATION_NAME))
    }
I don't want to do this, since my integration tests should not be able to access anything from the main sources
a
do you still get Task appintegrationTest FROM-CACHE even when a file in src/main changes?
e
This was the build for a commit where 
src/main
changed, yes 🙂
👍 1
a
how do the tests in src/integrationTest work? They don’t access files in src/main directly, but do the main sources get published to a Maven repo? Or do they get packaged into a Docker container?
e
packaged to docker The integration tests use REST / JMS to blackbox-test the application, and integration-tests depend on a 
dockerBuildImage
task which packages the application into a container image
a
are they packaged to Docker via a Gradle task?
💡 1
e
Yeah, I'm using 
com.bmuschko.docker-spring-boot-application version 9.3.1
, which must've falsely thought the docker image was up to date..
a
okay cool, that’s good. This is where it might get tricky though. The first step is to tell Gradle that the integration test task depends on the 
dockerBuildImage
task.
e
I had that, and the docker image was built as part of the build 🤔
a
where it might get tricky is if the 
dockerBuildImage
task is correctly wired up, so Gradle can fingerprint the inputs/outputs and skip the task if necessary
yeah so I guess the inputs/outputs aren’t registered properly
e
Copy code
@Suppress("UNUSED_VARIABLE") // Gradle determines name of test suite by the name of the variable
      val integrationTest by registering(JvmTestSuite::class) {
         testType.set(TestSuiteType.INTEGRATION_TEST)

         useJUnitJupiter()

         dependencies {
         }

         targets {
            all {
               testTask.configure {
                  dependsOn(tasks.dockerBuildImage)
                  shouldRunAfter(test)
               }
            }
         }
      }
Damn, okay.. 😞 so I guess that's an issue that needs to be filed in the plugin.. is there anything I can do on my end to always assume that input is updated then?
s
You probably need a dependency on the project as shown here: https://blog.gradle.org/introducing-test-suites#test-suites---a-better-way-forward
👀 1
a
in the GitHub action that showed
Task appintegrationTest FROM-CACHE
what was the result of the dockerBuildImage task? Was it also FROM-CACHE, or something else?
s
e
Copy code
> Task :app:dockerBuildImage
Building image using context '/actions-runner/_work/my-service/my-service/app/build/docker'.
Using images 'my-service'.

> Task :app:compileTestKotlin

> Task :app:dockerBuildImage
Step 1/8 : FROM openjdk:17-slim
 ---> 37cb44321d04
Step 2/8 : LABEL maintainer=root
 ---> Running in 2ac9bae4fc41
Removing intermediate container 2ac9bae4fc41
 ---> 4053d7984046
Step 3/8 : WORKDIR /app
 ---> Running in 7824e4d9d78b
Removing intermediate container 7824e4d9d78b
 ---> 66775799fb5c
Step 4/8 : COPY libs libs/
 ---> 4ccb83a13379
Step 5/8 : COPY resources resources/
 ---> 07594ab8e36a
Step 6/8 : COPY classes classes/
 ---> da4d15a0b2cc
Step 7/8 : ENTRYPOINT ["java", "-XX:+UseContainerSupport", "-cp", "/app/resources:/app/classes:/app/libs/*", "com.company.my.app.MyApplicationKt"]
 ---> Running in 6686bad67dc7
Removing intermediate container 6686bad67dc7
 ---> 23f0960863ea
Step 8/8 : EXPOSE 8080
 ---> Running in 3e91ee0851f6
Removing intermediate container 3e91ee0851f6
 ---> df1ac9790427
Successfully built df1ac9790427
Successfully tagged my-service:latest
Created image with ID 'df1ac9790427'.
a
Hmm, it’s strange that it ran 🤔 . If it ran, then because the integration test task has this 
dependsOn(tasks.dockerBuildImage)
then the intergration test task should also run Oh wait, Build Cache is enabled so if 
dockerBuildImage
doesn’t have an output file then Gradle does its magic
s
Since a while, Gradle also supports the per-task 
--rerun
option, maybe just use that explicitly.
a
there are two options I can think of 1. add an output file to 
dockerBuildImage
that contains the hash of the built docker image. If it has an output, then Gradle Build Cache will be able to kick in. You can use a 
doLast {}
block for this. It has to be an output file, because that’s the only type of Gradle task output considered by Build Cache. 2. compute and add the Docker image hash as an input to the integrationTest task, so Gradle will re-run the task if the image hash changes. Something like
Copy code
val integrationTest by registering(JvmTestSuite::class) {
targets.all {
  testTask.configure {
      val imageName = "my-image"
      inputs.property("dockerImageHash",
          providers.exec {
              executable("docker")
              args(parseSpaceSeparatedArgs("inspect --format='{{index .RepoDigests 0}}' $imageName"))
          }.standardOutput.asText)
  }
}
}
(Docker command from https://stackoverflow.com/q/32046334/4161471)
e
Thanks a lot guys, was stuck in meetings so need some time to catch up with the articles etc, but really appreciate the help 🤩
🚀 1
I ended up adding inputs to the test task:
Copy code
targets {
            all {
               testTask.configure {
                  dependsOn(tasks.dockerBuildImage)
                  shouldRunAfter(test)
                  inputs.files(fileTree("src/main")) // <-- Here
               }
            }
         }
The docker image hash you suggested @Adam exited with exit code 64 for some reason, even though the command Gradle was trying to use worked when I invoke it directly. Perhaps permission issues.. Either way, I think the above solution has the added benefit that it only runs the integrationTests if the main sources change, even if the docker image is rebuilt. It's good in my case, cause something is off with the caching of the docker image task so it always runs, which would make my tests always run as well..
v
Having the 
dependsOn
is not enough to have the docker image tasks output as input for up-to-date cache. Instead of using 
dependsOn
you would do 
inputs.files(tasks.dockerBuildImage)
. That way the outputfile of the task is considered when the fingerprint of the test task is calculated and you get the task dependency automatically. The 
dockerBuildImage
task also has the id of the built image in a file as output file, also for it's own up-to-dateness, so actually just using that should already have worked. Besides your problem that the docker image task is not up-to-date.
🦸 1
e
Nice, that should be even better.. It also seems that the task is not up-to-date because no docker image digest is generated.. not sure what to do about it yet but it should be solvable 🙂
👌 1
So 
dependsOn
only generate temporal dependency, without adding dependencies on the actual outputs from the task?
v
Yes, and thus practically any 
dependsOn
is a code-smell (except with a lifecycle task on the left-hand side)
til 1
You should always prefer wiring inputs to outputs and get implicit task dependencies
e
Makes sense 🙂
a
exited with exit code 64 for some reason
possibly 
docker
is on your user path, but not the system path? 🤔 But I’m just guessing.
What I’ve done before for building Docker images is to do it manually. I set up a separate 
./docker/
dir that contains the Dockerfile and docker-compose.yml (that’s only used for building the image) Then I define a Gradle Sync task 
prepareDockerBuild
to sync requisite files into 
./docker/build/
, so that it’s clear about what files are used for the building the Docker image. And then a 
dockerBuild
exec task that runs 
docker-compose build
, with working dir set to 
./docker/
. If the 
prepareDockerBuild
task doesn’t sync any files, then Gradle will skip 
dockerBuild
task - which is faster than invoking Docker and waiting for Docker to figure out that nothing has changed.
e
Nope
a
it’s a “why isn’t this enabled by default??” option
e
hehe 😄 yeah.. legacy reasons I suppose, like so many other things in Gradle..
Like why so many APIs are not lazy by default..
a
indeed. Hopefully in 9.0 it’s better… https://github.com/gradle/build-tool-roadmap/issues/28
e
Your solution sounds nice btw, but I'm in an org where I think I'm one of the persons with most Gradle knowledge, and I wouldn't know how to do that without spending a couple of days exploring I suppose.. I think there's a general resistance to having "code in gradle", rathern than just using Gradle to "declare builds", so it'd be a bit controversial to add more complex logic such as that for sure
a
mm yeah, for sure. I find it tough because a lot of the Gradle plugins just don’t follow the Gradle API well enough. Ideally they should just drop-in, and work… but when they don’t, they can be a real pain. But maybe your is fixed when you use 
inputs.files(tasks.dockerBuildImage)
as suggested by @Vampire and with reproducible archives 🤞
e
Yeah I got everything on my end working after those changes 🙂 Had to adjust it to 
inputs.files(tasks.dockerBuildImage.get().imageIdFile)
, and then do a small change for the dockerBuildImage task to cache properly (by adding explicit tag, was using :latest implicitly)
a
nice! Good to hear
e
Think I'll try to summarize today's Gradle adventure into a blogpost.. can be useful to share within my client's org at least.. 🙂
a
it’s best to use 
map {}
instead of 
.get()
though - so the task isn’t configured unless it’s required
Copy code
inputs.files(tasks.dockerBuildImage.map { it.imageIdFile})
e
ahh.. of course 😆
v
Why do you explicitly dereference the image id file property? This way you probably loose the implicit task dependency. And it should work with what I wrote I think.
Or well, you don't loose the task dependency, as it is an output property, so will work. But it should be unnecessary as it is the only output property.
e
Gradle complained about this:
Copy code
- Task `:app:integrationTest` of type `org.gradle.api.tasks.testing.Test`: cannot serialize object of type 'com.bmuschko.gradle.docker.tasks.image.DockerBuildImage', a subtype of 'org.gradle.api.Task', as these are not supported with the configuration cache.
  See <https://docs.gradle.org/7.6.1/userguide/configuration_cache.html#config_cache:requirements:task_access>
v
> it’s a “why isn’t this enabled by default??” option
hehe 😄 yeah.. legacy reasons I suppose, like so many other things in Gradle..
I'd say probably more what makes more sense. 😄 Many people would majorly complain if they loose their timestamps in the archive. And even without that option, the archive will be up-to-date if the inputs and outputs didn't change.
Gradle complained about this:
Ah, okay, was not aware about that CC problem
e
I'm also getting this when I try to upgrade to Gradle 8:
Copy code
Configuration cache entry discarded due to serialization error.
Configuration cache state could not be cached: field `commandProvider` of `com.bmuschko.gradle.docker.tasks.image.Dockerfile$EntryPointInstruction` bean found in field `instructions` of task `:app:dockerCreateDockerfile` of type `com.bmuschko.gradle.docker.tasks.image.Dockerfile`: error writing value of type 'org.gradle.api.internal.provider.DefaultProvider'
Does it mean the plugin needs updating? 😄
v
Maybe, I don't know. Didn't use it with a Gradle 8 build so far
4 Views
Open in Slack
PreviousNext
Powered by