How the community packages were built This is probably far too verbose, but figured more details are better than less 🙂 Every bit I used except for the Overlook InfraTech private key is available in public repos in the overlookinfra github org. You'll notice I have a lot of

overlookinfra

nomenclature in here, but I'd like to move these all to more "puppet community" nomenclature instead. • puppet-agent ◦ Building off of the work Jeff Clark did to improve Docker support in Vanagon, I added a bunch more things to make building these packages with containers work better. You can see my choices of container images for each platform there. I also tried to standardize the platform defaults a bit since they've gotten rather divergent over the years. This also allows us to build for different architectures. I built all of this on my Ryzen 5 5950X, and aarch64 and ppc64le were run under emulation with qemu. ◦ In some cases, Perforce uses their own internal version of build tools (pl-build-tools) to build for older platforms who ship with build tools that are tool old. Rather than do this, I've moved utilizing publicly available updated tools instead. I think this was mostly just for el7 these days. ◦ In each of the component repos (puppet-runtime, pxp-agent-vanagon, puppet-agent), you'll see we have a

plubming

branch, which is used for syncing the branches from

puppetlabs

exactly as they are. I've also put my code for building these things in there. ▪︎ Each has an

overlookinfra.patch

which changes some of the component files in order to make them work correctly with containerized building with our vanagon changes. ▪︎ Rake tasks • Tag - This checks out the given

puppetlabs

tag we want to build for, then applies the

overlookinfra.patch

, and then retags in the form

<puppetlabs tag>-overlookinfra

. It pushes a branch with a name of the form

overlookinfra/<puppetlabs tag>

that contains the commit that is tagged. In

pxp-agent-vanagon

and

puppet-agent

, this also overwrites the component json file to point to the OSL

puppet-artifacts

S3 bucket. • Build - This utilizes the

build-vanagon.rb

script which is also found in the

plumbing

branch. This is a pretty hacky script, but what it does is create directories for however many platforms you want to build in parallel, and then run those builds in the container for each platform. You can certainly do 1 single thread to do one platform at a time. On my 32 hardware thread/32GB RAM system, 8 threads seems to be pretty reasonable. All of the packages end up back in this repo in the

output

directory like it would with a normal vanagon build. • Upload - This is written specifically for uploading to the OSL S3 buckets. You won't be able to use this without the secrets, but all the code is there. ◦ First, puppet-runtime is built and uploaded to the

puppet-runtime

S3 bucket. Then

pxp-agent

, which utilizes

puppet-runtime

, then

puppet-agent

which utilizes both. • puppetserver/puppetdb ◦ This has similar build and upload rake tasks as above. Right now, we aren't needing to make changes to the repo for our purposes, as we can use puppetlabs' ezbake for now, so no tag task. ◦ Build uses the

build-ezbake.rb

hacky Ruby script. This creates (or reuses) an image that is based on almalinux 9 that has Ruby, Lein, and Java installed. It then builds for all the platforms, since you don't actually need to build on the specific platform your are building a package for. • The code used for signing the packages can be found at https://github.com/overlookinfra/misc/tree/main/signing. This uses an alma 9 container to sign RPMs and debian 12 to sign DEBs. For debs, we use

aptly

for repo creation, since creating a deb repo by hand is not so fun. For the yum repo, it's a lot simpler. We sign with our GPG-KEY-overlook key, and the pub key is available in repos. • To create the repository packages (i.e. the rpm files at https://yum.overlookinfratech.com/ to set up the repo on your machine), I used the

overlook

branch of puppetlabs-release. This is modified to point to the OSL repos, and includes the pub key which gets placed and imported automatically. This is basically all spike code, as I was trying to get something up and going as fast as possible, and a whole lot of copied code between repos. I would love some help in turning this into proper tooling (or perhaps integrating it into existing tooling) and creating proper pipelines. I also don't have Windows or MacOS packages built out, as that's a bit of a heavier lift. Also ran into some issues with SLES 12, I wonder if anyone still uses that. For what we have now, I've built it all on my home computer, and you should be able to as well for reproducibility purposes except for the upload and signing part, since you'd need secrets for that.