https://blue-build.org/ logo
Powered by
ERROR: failed to push to ...: unknown: too many ma...
# questions
r
Hi all, Our EU OS image build based on ublue version 44 fails with the error:
Copy code
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #49 exporting to image
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #49 pushing layers 50.2s done
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #49 pushing manifest for registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:ublue-44@sha256:c28adf5789ed495ecc7c2d31ada7ebf896d6c2c09f762f0b241e75ab325e8987
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #49 pushing manifest for registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:ublue-44@sha256:c28adf5789ed495ecc7c2d31ada7ebf896d6c2c09f762f0b241e75ab325e8987 0.0s done
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #49 ERROR: failed to push registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:ublue-44: unknown: too many manifest references
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => 
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #51 exporting cache to registry
[08:29:52 r.g.c/e/w/e/eu-os-demo:ublue-44] => #51 sending cache export 46.0s done
Source: https://gitlab.com/eu-os/workspace-images/eu-os-base-demo/-/jobs/16610306184 Any idea what's wrong? Our recipe file is here: https://gitlab.com/eu-os/workspace-images/eu-os-base-demo/-/blob/main/recipes/ublue-44.yml?ref_type=heads
a
message has been deleted
g
That's a new error. It doesn't look like you're building more than the amd64 platform
I'm noticing that job of yours has been failing for over a week, but with a different error each time
r
I cleaned the cache this weekend. So this may have changed something. So far, ublue is not our prority and we may as well remove it from our test builds. However, I think that in the larger community ublue is quite popular and I have an intellectual interest to understand why it just does not work. 🙂
@gmpinder https://gitlab.com/eu-os/workspace-images/eu-os-base-demo/-/jobs/16759315193#L2729
Copy code
[02:36:06 r.g.c/e/w/e/eu-os-demo:ublue-44] => #49 ERROR: failed to push registry.gitlab.com/eu-os/workspace-images/eu-os-base-demo/eu-os-demo:ublue-44: unknown: too many manifest references
I try to find the problem with Claude AI.
g
Looks to be an error that derives from Gitlab's registry
At least none that got indexed
That's honestly fascinating
They eventually converge to [this function](https://gitlab.com/gitlab-org/container-registry/-/blob/9d554515efc468202320ccb954bc0c352da6ad49/registry/handlers/manifests.go#L810). The 
manifestHandler
is what's holding the value
After traversing the types to find 
Context
and then eventually get to [
App
](https://gitlab.com/gitlab-org/container-registry/-/blob/master/registry/handlers/app.go#L151) that has a 
manifestRefLimit
So, it's whatever GitLab set it to
Now as to why there's a single image of yours causing that error, I have no clue. The cache is pushed to a different tag
But it's erroring on the main image
Unless it's talking about layers?
Copy code
$> podman inspect --format='{{len .RootFS.Layers}}' ghcr.io/ublue-os/kinoite-main
259
This might be the reason why
GitLab may have limited the layer count to Docker's which is 
128
.
Ublue chunks their images, which over 128 is can only be read by podman/buildah
r
Claude AI has proposed:
Copy code
variables:
        BUILDX_NO_DEFAULT_ATTESTATIONS: "1"
        BB_CACHE_LAYERS: "false"
Didn't help.
g
Copy code
$> podman inspect --format='{{len .RootFS.Layers}}' quay.io/fedora/fedora-kinoite:44
65
The official fedora images however are much smaller
r
Did the ubule 43 image had less than 128 layers?
g
Checking that now
Copy code
$> podman inspect --format='{{len .RootFS.Layers}}' ghcr.io/ublue-os/kinoite-main:43
99
Yeah, I'm fully convinced now
You could do chunking on that image yourself and bring it down to the "legal" limit
r
How would I need to do this?
g
Using the 
--chunkah
(
BB_BUILD_CHUNKAH
) flag and setting 
--max-layers
(
BB_BUILD_MAX_LAYERS
) to 
128
or lower
It will need to use 
podman
to run
But, so long as you're using the official bluebuild CLI image, it should be good to go
r
That means I replace
Copy code
- if: $RECIPE == "ublue-44.yml"
      variables:
        BUILDX_NO_DEFAULT_ATTESTATIONS: "1"
        BB_CACHE_LAYERS: "false"
with
Copy code
- if: $RECIPE == "ublue-44.yml"
      variables:
        BB_BUILD_CHUNKAH: "true"
        BB_BUILD_MAX_LAYERS: "124"
?
g
Yeah, that should work
Just a heads up though, there won't be any caching for this type of build
That is to say, the build won't cache or pull from cache. The chunking process though should make it so that you pull fewer layers than normal on each update
a
message has been deleted
PreviousNext
Powered by