Any Gitlab pipelines users here? Guess this questi...
# randoml
Lukasz K
05/18/2022, 9:13 AMAny Gitlab pipelines users here? Guess this question applies to GitHub jobs as well.
With the support for docker containers in Lambda I got to wonder if I could drop my previous (unfulfilled) plan of ec2 on-demand runner fleet in favor of spawning Lambdas that could be assigned my team's CI/CD jobs.
Anyone here tried going similar direction? Seems like all the building blocks are now available and this approach would allow for an instant runner spawning whenever one was needed without the overhead of managing ec2s.
Lukasz K
05/18/2022, 9:17 AMMy backend jobs are ran in Seed (bless you for that team!) but we've got a separate repo for the Flutter app which needs to validate PRs/main branch merges against three different build platforms and things can get messy when you spawn x*12 builds, where x = amt of PRs in queue + some of those are dependent on each other
e
Edward Asquith
05/18/2022, 9:34 AMWe use gitlab at my place, though we’re considering a transition to Seed.
Not sure your plan will work. Self-hosted build agents usually need to have a persistent connection to the build management system, and I’m fairly sure this is the case for GitLab too.
In general if your build needs are fairly simple my advice would be to use someone else’s hosted build containers and just pay for the build minutes. Overall tends to be cheaper than running your own infrastructure and maintaining your own build images.
l
Lukasz K
05/18/2022, 9:38 AMIt is possible to set up autoscaling (https://docs.gitlab.com/runner/executors/docker_machine.html) with one orchestrating runner and a fleet under it
Lukasz K
05/18/2022, 9:39 AMI was half way there with the config for auto-spawning EC2s but lambdas seem more cost-efficient in this regard and would start up faster
Lukasz K
05/18/2022, 9:41 AMThe biggest issue with our builds is the requirement for a macOS worker for iOS builds. Hence we keep a bare-metal that runs those + any random jobs that can be done in separate threads with what memory is left
e
Edward Asquith
05/18/2022, 9:41 AMOh that’s neat - didn’t know about that one
l
Lukasz K
05/18/2022, 9:58 AMI'll reply here if I find the PoC worthwhile