Muhammad Bilal
02/06/2025, 12:25 PMsource-postgres-check
pod are in pending state with the following event:
"Failed to pull image "airbyte/source-postgres:3.6.23": failed to pull and unpack image "docker.io/airbyte/source-postgres:3.6.23": failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/airbyte/source-postgres/manifests/sha256:f775686e83dac80200a6957949b6fb2488c75b6eae9b1f55615f9ed1304a3cca: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
Even though I am using external database.kapa.ai
02/06/2025, 12:26 PMdocker login
3. Using Image Pull Secrets: If you're running Airbyte on Kubernetes, you can set up image pull secrets to authenticate with Docker Hub. This was mentioned in the same Github issue: yaml - name: JOB_KUBE_MAIN_CONTAINER_IMAGE_PULL_SECRET value: dockerhub-secret
4. Using a Custom Registry: Several users have suggested using a custom registry or a pull-through cache to mitigate this issue. For example, in [Github Issue #21123](https://github.com/airbytehq/airbyte/issues/21123), users discuss using ECR pull-through cache or other private registries. 5. Prefixing Docker Repository: As a workaround, some users have modified the database to prefix the docker repository with their custom registry. This was mentioned in a comment in [Github Issue #21123](https://github.com/airbytehq/airbyte/issues/21123): sql UPDATE actor_definition_version SET docker_repository = '<your_registry>' || docker_repository;
6. Restarting Airbyte: In some cases, as mentioned in [Github Issue #39424](https://github.com/airbytehq/airbyte/issues/39424), restarting Airbyte (via docker compose) has temporarily resolved the issue for some users. It's important to note that while you mentioned using an external database, this doesn't affect the Docker image pulling process, which is why you're still encountering this error. To resolve this issue, I would recommend first trying to authenticate with Docker Hub using docker login
. If the problem persists, consider implementing one of the other solutions mentioned above, such as using image pull secrets or setting up a custom registry.