Hi! I've been thinking about replacing our Jenkins CI process with a Gradle build. That is, instead of using Jenkins pipeline scripting or introducing a new technology such as Earthly or Dagger I would define a Gradle task tree which is made up of regular Gradle build tasks,

Exec

tasks, tasks running ansible playbooks, etc. and just have Gradle handle everything. Gradle already handles task parallelization and caching. I think that I just need to define a decent DSL to define the task tree and find an elegant way to pass information between the tasks (e.g., one task could create AWS IAM credentials which another task would want to use). Build information could be passed back via a build listener or the like. Does anyone have any experience with anything like this? Is this a good idea? Thanks!

It sounds like you are going to try to use Gradle as some sort of general-purpose workload automation system and it's not really designed for that case so you may run into hard limitations. There are many other open-source projects that are probably be better suited for replacing Jenkins, e.g. Concourse CI. But then maybe you will manage to achieve something interesting with Gradle, so it's up to you.

Thanks for the comment @Julien Plissonneau Duquène. I'll have a look at Concourse CI. Just for me to get a clearer picture, if you can think of any such hard limitations I would love to hear them. Thanks

It really depends on what you are trying to achieve, it was not clear that you still wanted to keep Jenkins above it (that makes sense) and just replace pipeline scripting. The most obvious limitation I can think of is that with a build tool, you start with a set of tasks that you know you want to execute, and the build tool then builds a graph of tasks that need to be completed before the given tasks are started. If any of these tasks fail normally subsequent tasks won't be started. In workflow automation it mostly works the other way: you start with some trigger (e.g. output of a task that completed) and then the tool figures out which tasks can now be started with that input, eventually with rules to prevent choking on limited resources, or retry tasks that failed. While building a project a build tool does that as well, but with much more limited options to queue, delay, retry, work around tasks that failed, or monitor global limits on resource usage. For example on most CI platforms if a single test job fails you can usually manually retry that job alone without reevaluating the previous jobs at all. With a build tool you could request a single test task, but all the tasks it depends on will be reevaluated up to the source files, so if your sources changed in the meantime you can't retry the exact same test job anymore.

Interesting points Julien. Thank you.