I have a CI job that has a

build

job that compiles everything, and then a couple of downstream jobs that does some stuff with the built artifacts. The

build

job starts with restoring

~/.gradle/shared_cache/modules-2

and sets the

GRADLE_RO_DEP_CACHE

environment variable to this and saves that path to CI cache at the end after having rsync'd any delta. Downstream jobs restore that shared cache and maintains their own cache of

~/.gradle/caches/modules-2

with for example runtime dependencies that aren't available in the shared cache. While this works nice I noticed that it's still pretty slow and with

--info

I noticed what it looks like http lookups of manifests for the dependency verification feature. I was under the impression that this would be a no-op given that I have a warm cache with all my dependencies now, so I put that to the test and added

--offline

argument to the build and this made the build fail with

checksum is missing from verification metadata

. So how do I avoid dependency verification needing network access? If I search my local

~/.gradle

directory for

.asc

files, the

modules-2

directory is the only one that matches, so these should be available in the cache already. Is this a current limitation that

GRADLE_RO_DEP_CACHE

doesn't allow for resolving dependency verification signatures? I'm using a checked-in keyring, and have keyservers disabled in the

verification-metadata.xml

with the goal of not needing network access.

and saves that path to CI cache at the end

Why? You configure it as RO - i.e. read-only - cache, so it should not change during Gradle execution. The RO cache is more for cases where you map one directory into multiple concurrently build agents like into multiple Docker images at once or similar so that it can safely be used concurrently as it does not change but is read-only. To store int he end and restore at next beginning I wouldn't use RO cache.

after having rsync'd any delta

Where do you get a delta from? You might get additional things form the non-RO cache, but that does not clean up old unused ones. And as I said probably does not make much sense if you don't use it concurrently anyway.

1. Because downloading a cache from a CI-local server based on libs.versions.toml + gradle-wrapper hash is faster than having gradle download half the Internet from various places. This cache is likely to be identical for at least a whole day, perhaps a few days. 2. Delta will be in ~/.gradle/caches/modules-2 minus gc.properties and *.lock as per documentation. Append, ofc, which is fine as CI cache has a lifetime so at some point it will be rebuilt from empty, but that's many builds later. A delta can occur as CI can restore cache from a partial cache key if libs.versions.toml has changed, but partial key got a match of an older version. ...and downstream builds run in their own filesystems.

I did not ask why you use CI cache at all. I asked why you do use RO cache setup instead of simply CI-caching the RW cache. https://docs.gradle.org/current/userguide/dependency_caching.html#sec:cache-copy

To be able to have a separation between build job and downstream jobs caches. Don't want each job to have its own cache of everything, and want all jobs to hit cache. The problem before was that I cached ~/.gradle/caches which provided nice build times compared to what I see now (although I guess that cached way too much), but downstream jobs fetched their runtime dependencies. If it was possible to resolve all dependencies with one command in the build job, that would remove the need to maintain separate caches. From what I can tell, the

--refresh-dependencies

command would bypass the cache, and not sure if it would do what I wanted anyways.

To be able to have a separation between build job and downstream jobs caches. Don't want each job to have its own cache of everything, and want all jobs to hit cache.

You can still not store a cache in the downstream jobs but restore the build job's cache, can't you?

If it was possible to resolve all dependencies with one command in the build job, that would remove the need to maintain separate caches.

There could always be stuff downloaded at task execution time. But other than that, you could just resolve all resolvable configurations, or use the

--write-verification-metadata

which also resolves all resolvable configurations.

ooooh... so "If the checksum is available" == checksum in

verification-metadata.xml

? Well that's annoying. The reason I don't have that is to be able to approve DependaBot PRs from trusted projects. If having checksums in that file then each dependency bump needs manual intervention. It's a trade-off, while it would add something, having signature verification is at least a big step up from nothing.

ooooh... so "If the checksum is available" == checksum in

verification-metadata.xml

?

I guess so, never used verification myself yet.

You should. Internet is a hostile place.

I know 🙂

Well... you would sacrifice either build times or dep-bump-effort 😂

The reason I don't have that is to be able to approve DependaBot PRs from trusted projects

Maybe dump Dependabot and use Renovate instead. Renovate is like Dependabot on steroids. I generally dislike such bots in general, but if at all, then I would always use Renovate. Not sure whether it can handle the checksums, but worth a try.

It actually passed now with --offline! Well... not really passed, kotlin daemon crashed, but it failed in the beginning with --offline before iirc.