This message was deleted.
# caching
s
This message was deleted.
v
I would say yes, caching should work just fine.
j
if it’s not though (as it isn’t) what would be the usual suspects to check? 1. I’ve applied the android cache fix plugin 2. using same JVM major version (although openjdk on ci, corretto locally) 3. same goes for agp/gradle-versions (those are all defined in code and I’m building the same branch)? 4. what else?
v
You mentioned GE, so maybe compare two builds, one that reuses the cache and one that doesn't?
Also, maybe
-Dorg.gradle.caching.debug=true
can help, when set to true, individual input property hashes and the build cache key for each task are logged on the console
j
yeah I did a lot of “rinse-and-repeat” analysis… and I’m near certain this started happening after agp 7.3 update
I mean I have local builds reusing CI results from the 7.2.x branch (created just before bumping to 7.3) and the next commit (which is the version bump) has 0 remote hits
comparisons in gradle enterprise are hard to read though… showing 4k tasks with differences most of them differing on the value of “aapt2” property (along with bunch of others)
on other runs (with more changes than just agp 7.3.x) I have even weirder results: 1. one comparison that shows no common tasks between builds even though that’s clearly not the case 2. most of the comparisons with composite plugins show that the only different task inputs were related to the composite plugins idk this is a real pickle, spent all day trying to figure it out and came out with nothing… other than the possibility it has nothing to do with my changes (hence my previous questions of how to query GE for remote cache hit % over time)
n
If the comparison shows a lot of tasks with differences in implementation then it is likely an infrastructure difference (such as JDK version) or a buildscript classpath difference.
f
@Jan Duzinkiewicz Can you check the
/performance/build-cache
page of such build scan using composite builds for convention plugins? Due to a Gradle chicken-egg issue, it is possible that those early evaluated builds don’t share the build cache setup you declare on your root build. Do you see a warning here about multiple build cache configuration ? Can you also maybe check with the Export API the values of the
BuildCacheConfiguration
events ? sth like
/build-export/v2/build/<<YOUR BUILD SCAN ID>>/events?eventTypes=BuildCacheConfiguration
This is not going to explain the 4k differences, but it might explain cache misses for tasks coming from those early builds.
e
If I remember correctly AGP 7.4 requires Java 11 now
j
Do you see a warning here about multiple build cache configuration ?
I do… but I’ve also seen the remote cache being reused by CI nodes
Can you check the
/performance/build-cache
yeah, that’s my SOP - create a “unique build by adding an android buildconfig field with a unique value at a “root” of the dependency graph. start a build in CI, start a build locally once CI finishes, check remote cache hits and do a comparison
If the comparison shows a lot of tasks with differences in implementation
1. for agp 7.2 builds the tasks that are different all seem to fall into “jetifier transform” which is understandable. 2. for “bisect” all builds I’ve done to see that the issue started occuring when we moved to agp 7.3.1 (still using binary plugins)… it’s like 3. for “composite convention plugin builds”… it did show normalization issues? (need to dig those out… I did a lot of comparisons yesterday)… reverting to binary plugins did’t help. Here’s the weirdest thing - “reverting to binary plugins and removing dokka plugin (it was shown as a build-dependencies diff)… > that gets me to remove cache hits (🎉 )… but also build comparisons show no common tasks between builds (which is clearly false) lmk if you want screenshots/more details on any of those cases
I’ll just keep posting my theories/findings so back to case 3 - removing binary plugins and dokka, I’m certain the builds did run the same tasks I don’t understand why GE would tell me otherwise. this happens regardless of there being remote cache hits or not… but only on latest build tools (agp 8 beta, gradle 8, java 17)
If I remember correctly AGP 7.4 requires Java 11 now
Java 11 has been a requirement since AGP 7.x? one thing I’m not sure is if Java 17 is a requirement or just a recommendation for AGP 8.x (the release notes just say it’s bundles now, and 8.1 lint checks will ship with 17, but I’m on 8.0) - if someone could confirm, that would help me plan/communicate jump to java 17 with my team
But is not important I believe
Off topic - we (my team) are always on the latest JDK unless it makes us troubles
j
we’re at 11 since years, but I did jump to 17 for my latest builds when I discovered remote cache failures. but then I went back to old builds and remote cache wasn’t working on 11 either so.. and it’s working for 17 now (so maybe JVM is not part of the problem)
anyways, here’s a screenshot of build comparison task-inputs in the case of “composite convention plugins” that suggests those could break cache-ability? but idk, it does not say so explicitly
n
are both builds using the same git commit? can you share the differences in custom values and the differences to infrastructure?
j
yes, same commits.. for this particular comparison, infra diff is… I had the same results when I enforced corretto everywhere, I can double-check that
anyway, I want to finish my dokka theory first though
the reason why I suspect dokka is causing this (other than - removing it resulted in me seeing remote cache hits is):
n
can you share a screenshot of the
/performance/build-cache
page?
j
can you share a screenshot of the
/performance/build-cache
page?
for which build?
n
yes, if there is a difference to the buildscript classpath that is definitely the reason
for both builds
j
I’ve seen builds where the dokka difference was there but it would still hit remote cache… I can run more experiments with cherry picking just it’s removal though (I didn’t get to that yday… it was late)
my suspicion for why is dokka different (it’s the same version after all) - when I drill down - it shows one build selected runtimeElements variant and the other runtime… and my best guess right now, it’s because we use different repositories locally/in-ci (long story) and… maybe the mirror set between them is ignoring some files (gradle module metadata) that then lead to different variant selection?
n
even if it is the same version, if it is showing a difference in the task input comparison that means that there is a difference in the jar. if there is a difference in the jar on the buildscript classpath then it is definitely causing the "task implementation changed" issue you are seeing.
j
but the dokka theory doesn’t sound like a dead end/false lead?
I think I’ll run a few more experiments: 1. get back to composite build plugins, remove dokka 2. get back to builds using agp 7.3.x and also apply dokka fix to see if it fixes it for them it will take me a while, most of which will be waiting for CI maintenance to end 🙂and then maybe fixing what’s broken by that maintenance would still appreciate the explanation for the weird “no common tasks” screenshoot posted above
ok, so I’ve run both of those, and got remote cache hits in both cases, so it definitely looks like it’s dokka and not plugins in composite builds
is there a way to run dependencyInsight, but for buildEnvironment dependencies?