Hi, I have a cacheable Gradle task which Gradle so...
# caching
r
Hi, I have a cacheable Gradle task which Gradle sometimes decides to rerun due to "The task was not up-to-date because there was no task history available.". Enabling
-Dorg.gradle.caching.debug=true
and comparing two builds' Build Scans shows that the only difference between the executions is the location of some of the dependencies. I.e., in one build, some dependencies where fetched from
/data/gradle-dependency-cache/gradle-home
while in the other, they were fetched from
/home/jenkins/.gradle
. The hashes for all dependencies are exactly the same between the builds, regardless of which folder they were fetched from. And yet, Gradle calculated a different input file fingerprint for jar list input. Can anyone explain what's going on here and how I might be able to solve this issue so that this task would be deterministically cached? I can share the task logs if that would be of any help. Thanks a lot.
c
Perhaps check the path normalization on your task(s) e.g.
@PathSensitive(RELATIVE)
. That becomes part of the cache key.
r
Hi Chris, the task's inputs are always at the same location. What seems to differ is the path of the dependency jars. I should mention that I'm using
GRADLE_RO_DEP_CACHE
.
I guess I would like to ask if the dependencies' location is also part of the cache key?
c
that’s what
@PathSensitive
does - it normalizes the path; generally you want
RELATIVE
to only store inputs relative to project directory. If you are storing absolute paths for input files that will invalidate the cache if the project is in a different absolute path. Dependencies are cached differently - those are relocatable and shouldn’t be a problem. From your message
"The task was not up-to-date because there was no task history available.".
the issue is not with dependencies, rather with the build cache / task input fingerprinting. Check what the task takes for inputs - are they normalized? Are they the same between builds?
r
Here's an example of what the logs are telling me: Build A:
Copy code
/tmp/jenkins/workspace/model9-lab/PrimeGradleCaches/model9-zos-catalog/build/classes/java/main/io/model9/catalog/TypedVolser.class='/tmp/jenkins/workspace/model9-lab/PrimeGradleCaches/model9-zos-catalog/build/classes/java/main/io/model9/catalog/TypedVolser.class' / bc2649c12ec53e8894f8f0c8bfc5d8aa
/home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/33.0.0-jre/161ba27964a62f241533807a46b8711b13c1d94b/guava-33.0.0-jre.jar='/home/jenkins/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/33.0.0-jre/161ba27964a62f241533807a46b8711b13c1d94b/guava-33.0.0-jre.jar' / ede2e3cac8e95ca2ab8e19df9e83419e
Build B:
Copy code
/tmp/jenkins/workspace/model9-lab/PrimeGradleCaches/model9-zos-catalog/build/classes/java/main/io/model9/catalog/TypedVolser.class='/tmp/jenkins/workspace/model9-lab/PrimeGradleCaches/model9-zos-catalog/build/classes/java/main/io/model9/catalog/TypedVolser.class' / bc2649c12ec53e8894f8f0c8bfc5d8aa
/data/gradle-dependency-cache/gradle-home/caches/modules-2/files-2.1/com.google.guava/guava/33.0.0-jre/161ba27964a62f241533807a46b8711b13c1d94b/guava-33.0.0-jre.jar='/data/gradle-dependency-cache/gradle-home/caches/modules-2/files-2.1/com.google.guava/guava/33.0.0-jre/161ba27964a62f241533807a46b8711b13c1d94b/guava-33.0.0-jre.jar' / ede2e3cac8e95ca2ab8e19df9e83419e
Notice how TypedVolser.class has the same path and fingerprint. It's fine. And notice how guava-33.0.0-jre.jar has a different path and also the same fingerprint. It's also fine. But even though, both inputs have the same fingerprints between the two builds, Gradle decides to calculate a different fingerprint for the task's $2 input fingerprint: Build A:
Copy code
Appending input file fingerprints for '$2' to build cache key: f593e94e83b42c1025a1e2075b88ced9
Build B:
Copy code
Appending input file fingerprints for '$2' to build cache key: 4c2ef49eacab6dcdc75581511c54cb43
It looks like the only differences between the two builds' inputs are the dependencies' (jars) locations.
c
Are those JARs inputs to your custom task? What are the custom task inputs?
r
Ohh, I think you hit the nail on the head there. And it might have just clicked for me
Copy code
inputs.files(testSourceSet.compileClasspath)
I guess that would cause this issue and PathSensitive would in fact solve it for me
c
Worth a try, iirc that’s `
Copy code
.withPathSensitivity(PathSensitivity.RELATIVE)
for the declarative task inputs. there’s also a classpath-specific task input, let me see if I can find that…
👀 1
There’s also
@Classpath
, which is similar to
PathSensitivity.Relative
but with special handling knowing its a classpath. https://docs.gradle.org/current/userguide/incremental_build.html#sec:task_input_output_annotations
doesn’t look like that one is available via
inputs.files
, only via the annotation.
r
OK. I'll give PathSensitive a try. Thank you very much Chris
👍 1
a
for classpath normalization, try this:
Copy code
inputs.files("...")
    .withNormalizer(org.gradle.api.tasks.ClasspathNormalizer::class)
r
Thanks Adam. Will try that
n
I just wanted to point out that the message
The task was not up-to-date because there was no task history available.
has nothing to do with caching. It only explains why the task was not up-to-date. This is a common source of confusion. You are likely seeing this message for every build for every task if this build ran on an ephemeral container (CI) where there is no previous workspace or build. [Here is](https://docs.gradle.org/current/userguide/incremental_build.html) more information on Gradle incremental build.
r
Thanks @no, understood.