while doing a release, I’m running into
```4531e20...
# contributing-to-airbytej
Jared Rhizor (Airbyte)
05/03/2021, 7:00 PM
while doing a release, I’m running into
Copy code
4531e200ac8d: Layer already exists
ed3fe3f2b59f: Layer already exists
b2d5eeeaba3a: Layer already exists
0.23.0-alpha: digest: sha256:b0cd277cd03b0a90a7ffecda65ada9a1364918a2d48dcc41830ed18e1cc4cbca size: 1986
Pushing migration (airbyte/migration:0.23.0-alpha)...
The push refers to repository [<http://docker.io/airbyte/migration|docker.io/airbyte/migration>]
ERROR: An image does not exist locally with the tag: airbyte/migrationu
user
05/03/2021, 7:00 PMI don’t actually have an image for it
u
user
05/03/2021, 7:00 PM Copy code
→ docker image ls | grep airbyte
airbyte/webapp dev dc4cf95abacb 4 hours ago 44.2MB
airbyte/server dev a93d0b093015 4 hours ago 622MB
airbyte/scheduler dev d140f239445f 4 hours ago 1.01GB
airbyte/seed dev b36f11fdbb88 4 hours ago 4.83MB
airbyte/code-generator dev ca5862050276 3 days ago 189MB
airbyte/db dev 6158eb3471b0 6 weeks ago 160MBu
user
05/03/2021, 7:00 PMbut that’s surprising to me given that I ran it with
VERSION=$ENV_VERSION GIT_REVISION=$GIT_REVISION docker-compose -f docker-compose.build.yaml buildu
user
05/03/2021, 7:01 PM@charles does this look familiar to you? only asking since you added the change to push migrate app as part of airbyte core (#1748)
u
user
05/03/2021, 7:01 PMno need to look into it if it isn’t familiar
u
user
05/03/2021, 7:02 PMhuh. no i don't think i've seen that before.
u
user
05/03/2021, 7:02 PMi think the migration module should build as part of composeBuild
u
user
05/03/2021, 7:02 PMyeah it is
u
user
05/03/2021, 7:02 PMso i'm a bit surprised it doesn't exist.
u
user
05/03/2021, 7:03 PMdo you happen to know if you are using buildkit locally for this?
u
user
05/03/2021, 7:04 PMI am seeing
Copy code
Building migration
[+] Building 0.3s (9/9) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for <http://docker.io/library/openjdk:14.0.2-slim|docker.io/library/openjdk:14.0.2-slim> 0.2s
=> [1/4] FROM <http://docker.io/library/openjdk:14.0.2-slim@sha256:95bd8202bd64d4e36ffdf824588c0551b35e8fc3f279492c561e8dfb4d404c70|docker.io/library/openjdk:14.0.2-slim@sha256:95bd8202bd64d4e36ffdf824588c0551b35e8fc3f279492c561e8dfb4d404c70> 0.0s
=> => resolve <http://docker.io/library/openjdk:14.0.2-slim@sha256:95bd8202bd64d4e36ffdf824588c0551b35e8fc3f279492c561e8dfb4d404c70|docker.io/library/openjdk:14.0.2-slim@sha256:95bd8202bd64d4e36ffdf824588c0551b35e8fc3f279492c561e8dfb4d404c70> 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 142B 0.0s
=> CACHED [2/4] WORKDIR /app 0.0s
=> CACHED [3/4] COPY build/distributions/airbyte-migration*.tar airbyte-migration.tar 0.0s
=> CACHED [4/4] RUN tar xf airbyte-migration.tar --strip-components=1 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:93bd27c5bffda19193f7dafd68245a4155b67a7e2e0d96f71f0d694ad8a7c690 0.0s
=> => naming to <http://docker.io/airbyte/migration:0.23.0-alpha|docker.io/airbyte/migration:0.23.0-alpha> 0.0su
user
05/03/2021, 7:04 PMi don't think so. i just use
./tools/bin/release_version.sh patchu
user
05/03/2021, 7:14 PMafter switching off buildkit I got
u
user
05/03/2021, 7:14 PM Copy code
e9559562f36a: Layer already exists
fee20f1b745d: Layer already exists
d0fe97fa8b8c: Layer already exists
0.23.0-alpha: digest: sha256:7ee993d71334704084b3ea05ea8038b1e408d80497cf43bd1bc3ba0a6b19882f size: 2211
Pushing webapp (airbyte/webapp:0.23.0-alpha)...
The push refers to repository [<http://docker.io/airbyte/webapp|docker.io/airbyte/webapp>]
ERROR: tag does not exist: airbyte/webapp:0.23.0-alphau
user
05/03/2021, 7:14 PMso looks like it isn’t related to that change
u
user
05/03/2021, 8:26 PMStill can’t figure this out…
u
user
05/03/2021, 9:51 PMcan i help?
u
user
05/04/2021, 4:36 AMI finally figured this out.
u
user
05/04/2021, 4:37 AMEven though I had ~10gb disk space free according to my Docker resources tab, it was auto sweeping these images in the middle of the build…
u
user
05/04/2021, 4:37 AMBumping up disk space for Docker significantly fixed the problem.
u
user
05/04/2021, 5:51 AMvom. what a crappy problem.
u
user
05/04/2021, 5:52 AMthe logs didn't mention anything?
u
user
05/04/2021, 6:08 AMthe logs for the builds didn’t. I didn’t check docker daemon logs.
u
user
05/04/2021, 7:51 AMthe logs for the builds didn’t. I didn’t check docker daemon logs.how did you figure this out then?
u
user
05/04/2021, 7:52 AMit was auto sweeping these images in the middle of the buildit's taunting you Jared
u
user
05/04/2021, 4:10 PMThe first thing I did was check space, and I saw I had several gigs left, so I tried looking for other solutions. Couldn’t find anything comparable online, spent time poring over the logs, re-running, etc. But I was only checking if the images existed at the end and not during which is why I was struggling.
u
user
05/04/2021, 4:11 PMThe solution was to give up and work on other things.
u
user
05/04/2021, 4:11 PMBecause while I worked on destination-postgres, that individual image was also being deleted out from under me (and it was easy to reproduce).