This message was deleted.

Hi David, I think is a great idea. Following the GHA example https://github.com/gradle/gradle-enterprise-build-validation-scripts/tree/main/.github/actions/gradle it could be interesting to extend it with a GE Api calls that parses the results and post a comment in GH or updates the PR state, going to do some tests...

I don't see any obvious way to to have automation provide value. Sure, I can run this in CI and manually check the results, but it would be even better if the script can alert if there's an issue or if I could hit an API to extract the comparison data

The current value of the scripts is that you do not have to run manually the build caching experiments that reveal build caching deficiencies, meaning you do not have to manually run two builds for each experiment in the right configuration (and without touching the build itself), but instead the scripts take care of it. The scripts highly reduce the chance for human errors when running build caching experiments, something we have observed many times in practice. The scripts also add necessary metadata on the file system as well as in build scans to be able to navigate from the build scans back to the experiments on disk, and vice versa. In addition, the scripts run the experiment in a way that makes it very reproducible, even when running many experiments back to back. The scripts already provide a summary of how well cacheable the build is for the given set of tasks with which the build was invoked. Returning with an error from the scripts when the build is not fully cacheable (or some variation of this) is already on our roadmap and something we are working towards.

I did not mean to suggest that these scripts provide no value at all. It's just my definition of automation means that I don't need to manually verify that there is a problem with the caching. When I reviewed the README, it mentioned " automation of the validation scripts" which led me to believe I was missing something obvious. I'm glad to hear it's on the roadmap, as it would be great to shift build time validation as something that we can catch just in time, instead of after the fact. I'd like to suggest a similar approach to Android Lint where you can baseline the errors.

Thanks for the feedback.

as it would be great to shift build time validation as something that we can catch just in time, instead of after the fact.

I see the sequence as following: • optimize the build for full cacheability using the build validation scripts (run locally) • get notified when there is a regression in full build cacheability by triggering the validation scripts in CI on a regular base (on the roadmap) • use the build validation scripts to investigate the regressions (run locally) You could also run the local steps above in CI, but there is value in being able to inspect the two build runs on the file system to narrow down what exactly "broke" the caching.