Roldan Galan
02/14/2024, 4:17 PMgradle-home-cache-cleanup: true
(and we can see in the logs that is working), but the results are a bit disappointing. It's true that the home cache entry dropped from 3.6 gb
to 2.3 gb
in our main project. But we verified that three months ago this was smaller than 1 gb
. So I wonder if everything not used is actually being cleaned up. ๐ค
I also tried in a sample tiny internal repo and after enabling this flag, we still see the cache growing after some minor changes that, according to the docs, should not happen (like after bumping gradle). More details in the thread. ๐งตRoldan Galan
02/14/2024, 4:17 PMgradle-version: current
in the action setup as it's a requirement for the cleanup. But all our builds are using the gradle wrapper defined in the project (this should be fine, but sharing just in case this impacts somehow).
After that I've added the debug flag and noticed something that catch my attention in the logs. The Gradle User Home, after extracting common artifacts that the action stores in separate entries, still lists two transforms-#
folders:
227M ./caches/transforms-3
493M ./caches/transforms-4
I don't have much knowledge about these internals of Gradle, but my understanding is that these folders in the cache increment the number whenever the artifacts are not compatible with the new gradle version. So are they still required after a gradle bump?? ๐คRoldan Galan
02/14/2024, 4:21 PM2 .gradle/.setup-gradle/
1 .gradle/caches/
210 .gradle/caches/8.6/
1 .gradle/caches/jars-9/
4 .gradle/caches/journal-1/
2422 .gradle/caches/modules-2/
18908 .gradle/caches/transforms-3/
8946 .gradle/caches/transforms-4/
1 .gradle/notifications/
2 .gradle/notifications/8.4/
2 .gradle/notifications/8.5/
2 .gradle/notifications/8.6/
As you can see, also the notifications folder keeps data from Gradle 8.4 and 8.5 even if we are running Gradle 8.6. The contents on this folder are insignificant, but I think they prove that some legacy data still remains in the cache.Daz DeBoer
02/14/2024, 8:32 PMDaz DeBoer
02/14/2024, 8:33 PMgradle-home-cache-cleanup
support relies entirely on Gradle's built-in cleanup mechanism, but the action will touch certain files and "age" certain other to force the cleanup to run more frequently.Daz DeBoer
02/14/2024, 8:34 PMnotifications
directories remain in place. Gradle doesn't do any cleanup of these files.Daz DeBoer
02/14/2024, 8:45 PMgradle-version: current
in your action, it would be preferable to code the same version that is used in your project. Fewer Gradle versions being used means retaining fewer files.
โข When you enable cache debug logging, the most interesting messages will be found in the "post-action step". Search for Gradle User Home (directories >5M): after extracting common artifacts
and you'll see the main directories that are contributing to the Gradle User Home cache entry size. It would be very interesting to see if these grow over time, or only when updating to a new Gradle version.Roldan Galan
02/15/2024, 10:17 AMInstead of usingUsually we adopt the latest Gradle version quite quickly after becoming current so they will be usually the same (also we have automation with Renovatebot, and this would require us changing this value manually after each bump). We never needed this parameter before in the gradle action, but is required for the cleanup. So when it was added in the main repo it was already the matching the one used in the project. Current is now 8.6 anyway, so it matches with the latests runs in the sample repo from my second message.in your action, it would be preferable to code the same version that is used in your project.gradle-version: current
โข When you enable cache debug logging, the most interesting messages will be found in the "post-action step". Search forYes, the information from my above messages about the size of the two differentand you'll see the main directories that are contributing to the Gradle User Home cache entry size. It would be very interesting to see if these grow over time, or only when updating to a new Gradle version.Gradle User Home (directories >5M): after extracting common artifacts
transforms-x
folders comes from that section after extracting common artifacts. So this is what is being currently stored by the action after cleanupRoldan Galan
02/15/2024, 11:18 AM3.6 gb
and was reduced to 2.3 gb
after enabling the cleanup three days ago). The new entry for the home is just 457 MB
.
Then these builds only produce/require 500mb. So there is a lot of legacy data that is being preserved from one run to the other that is not properly captured by the cleanup mechanism. ๐
Some other data for context:
โข We started using Github Actions 4 months ago, with the gradle action already integrated. We never looked into the caches it was producing until now.
โข 3 months ago the home cache had 1 gb
(eldest data I could find)
โข It was using Gradle 8.4 then. We bumped to 8.5 couple of months ago. And to Gradle 8.6 two weeks ago.
โข Besides Gradle version, something else must contribute to this increase, as we see a constant increase week over week, even without bumping gradle. But many engineers contribute to this repo, so is not a controlled environment and is hard to point what causes these increases.
โข Cleanup was enabled three days ago.
First screenshot: Last commit that used the already existing cache (Gradle User Home size: 2367 MB)
Second screenshot: First commit without cache (Gradle User Home size: 456 MB)Roldan Galan
02/15/2024, 11:19 AMDaz DeBoer
02/15/2024, 1:47 PMtransforms-*
directories? Or do you see other large + growing directories in Gradle User Home? It would be good to know what things I should investigate.
Is this an Android project? Kotlin? (I mostly tested cache-cleanup with standard java projects, and these don't make much use of Transforms).Roldan Galan
02/15/2024, 2:16 PMtransforms-*
directories are most likely the main reason for this increase. So it would be the main place to look in my opinion.
Let me know if you need any other data I can facilitate from our current setup ๐Daz DeBoer
02/15/2024, 2:24 PMRoldan Galan
02/15/2024, 2:26 PMRoldan Galan
02/16/2024, 9:55 AMDaz DeBoer
02/16/2024, 12:15 PM