https://blue-build.org/ logo
Build Cache for GitLab.com
# questions
i
I am currently using BlueBuild with a GitHub Action that utilizes the GitHub Actions (GHA) build cache. This feature significantly improves my build times, and I’m very pleased with the results. However, I am considering migrating my project to GitLab.com and have noticed that there doesn't seem to be a similar caching feature available on that platform. I believe it might be possible to implement a caching solution akin to the GHA cache using "Inline cache" or "Registry cache" for GitLab.com. Could anyone provide insights on whether such features are available or if there are plans to implement them in the future? Thank you for your assistance!
a
message has been deleted
g
That would be something that I would have to code up. I believe we've already got an issue for it.
r
With our project https://gitlab.com/eu-os/workspace-images/eu-os-base-demo we would also benefit from this caching. Buildah got recently the options
--layers --cache-from … --cache-to
. Man here: https://manpages.ubuntu.com/manpages/plucky/man1/buildah-build.1.html With Gitlab CI, this could look like:
podman build -t container-name:$VERSION --layers --cache-to $CI_REGISTRY_IMAGE/cache-image --cache-from $CI_REGISTRY_IMAGE/cache-image -f Containerfile .
The
cache-image
could possibly made configurable. @gmpinder I did not find the issue in the cli repo. If you tell me where it is, I can also add this info to the issue (and track the issue myself). šŸ™‚
g
It looks like we don't have one. I can try to remember to make one later, or you can write it up
l
Could you share this cache stuff? I'd love to use it myself.
r
I have created this issue here: https://github.com/blue-build/cli/issues/410
g
Cool, thanks for that!
i
I've been using a caching strategy similar to this example from the [GitLab documentation](https://docs.gitlab.com/ci/docker/docker_layer_caching/#docker-registry-caching-example) in other projects, and it works great:
Copy code
bash
docker buildx build --push -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
  --cache-to type=registry,ref=$CI_REGISTRY_IMAGE/cache-image,mode=max \
  --cache-from type=registry,ref=$CI_REGISTRY_IMAGE/cache-image .
I think we should follow a similar approach to
BB_BUILDKIT_CACHE_GHA
to support this caching mechanism by adding another
if
statement in the
build_tag_push
function in
docker_drivers.rs
. What do you all think?
g
Yeah that's the plan. Just haven't gotten to it
i
Great news! It works perfectly now! For anyone facing similar issues, @gmpinder has resolved issue https://github.com/blue-build/cli/issues/410. Starting from version 0.9.12, you can enable registry caching by setting the
BB_CACHE_LAYERS
environment variable. Thank you, @gmpinder, for your excellent work!
g
The GitHub action should also include the change. The use_cache option will turn this on
You can also use the --cache-layers arg
r
Dear @gmpinder , what value does `BB_CACHE_LAYERS`take? I tried with
Copy code
BB_CACHE_LAYERS: true
in my variables dictionary in the gitlab-ci yaml file, but I get now the error: "invalid reference format"
g
It needs to be a string
So
"true"
Gitlab variables in the yaml only except numbers and strings
r
Then it must be another reason. I still get the same error: https://gitlab.com/eu-os/workspace-images/eu-os-base-demo/-/jobs/10217540472
g
Oh am I not replacing
/
with
_
for tags?
Rookie mistake
I can get a fix out for that
r
I do not follow. For which string such replacement would be due? Can I set this manually while waiting for a fix?
Our hackathon starts in 13h šŸ™ˆ
g
The branch name
It will get used for generating tags
And
/
is not allowed in a tag
r
ok
g
Yeah I've just gotta add some string replacement logic
That's gonna be 4 patches this weekend lol
Whatever, gotta make it better
@robert_83982 you should be good now btw
Good luck on your hackathon!
r
Hi @gmpinder , for EU-OS we build now 5 recipes in the same gitlab repo branch wit the same container image name, but different tags: - latest - latest-ublue - latest-almalinux The pipeline builds them all in parallel. As I cannot see several cache images in the registry, I think that the pipelines will all overwrite the same cache. Is there a way to configure the cache image name? My repo: https://gitlab.com/eu-os/workspace-images/eu-os-base-demo
g
It should push to a cache based on the tag
Copy code
rust
        let image: Reference = format!("{image_name}:{}", tags.first().map_or("latest", |tag| tag))
            .parse()
            .into_diagnostic()?;
        let cache_image = (self.cache_layers && self.push).then(|| {
            let cache_image = Reference::with_tag(
                image.registry().to_string(),
                image.repository().to_string(),
                format!(
                    "{}-cache",
                    image.tag().expect("Reference should be built with tag")
                ),
            );
            debug!("Using {cache_image} for caching layers");
            cache_image
        });
r
So when I use
alt-tags
, it will use the first one in the list and add -cache? If so, I suggest to mention this logic somewhere.