Slackbot
01/06/2023, 3:46 PMEric Haag
01/06/2023, 4:39 PMabilitiy to fail on every experiment if something not working correctlyAs you pointed out, this is already possible for all caching experiments (Gradle 2-5 & Maven 1-4).
can we have some sort of the Baseline file to allowlist what we expect to be non-incrementalThe scripts themselves don't actually know which tasks were executed, only the number of tasks and the cumulative execution duration per outcome. Therefore, this is probably not something we will add to the scripts because it would be a pretty major refactor of how we process data coming from the API.
ability to configure http timeouts for fetching scansThis is a good idea! Please feel free to create an issue for this.
extension points for project provisioningSorry, but I don't quite understand this suggestion. Could you please elaborate? Keep in mind, you can always use `--project-dir`/`-p` to change the script's working directory. You can also leverage
:project:task notation to only execute against a single project. For example using --tasks ':subproject-a:build' instead of --tasks build.
What do you think about experiments 04-05 in my caseIndeed these are more difficult to automate as they involve running builds in CI using remote cache. We have the gradle/gradle-enterprise-oss-projects repository which has many examples of running experiments in a continuous fashion. In that repository we only run experiments Gradle 1-3 and Maven 1-2. We feel this is good enough to detect a majority of regressions in project cacheability. Please feel free to use the gradle/gradle-enterprise-oss-projects repository for inspiration on how to automate running the experiments. If you are using GitHub Actions we also have composite actions you can use which simplify the process of running experiments. Here they are for Gradle and Maven. Once again, thank you for your suggestions!
Dmitriy Voronin
01/06/2023, 4:51 PM