Is this your first time deploying Airbyte: Yes OS Version / Instance: Ubuntu 20.04 Memory / Disk: 16Gb / 1Tb SSD Deployment: Docker Airbyte Version: 0.35.32-alpha Source name/version: - Destination name/version: - Step: Starting docker-compose.yaml Description: Cannot start docker-compose. Seems like issue with airbyte/temporal - perhaps file permissions. Excerpt from docker logs airbyte-temporal

touch: CONTAINER_ALREADY_STARTED_PLACEHOLDER: Permission denied
...
ERROR   Unable to create SQL database.  {"error": "pq: database \"temporal\" already exists", "logging-call-at": "handler.go:97"}

I'm running docker-compose as the same user that the repository files are owned by, in one of /home subdirs. Thanks in advance :)

@Augustin Lafanechere (Airbyte) thanks for quick response. I did that and the problem still occurs. airbyte-temporal keeps failing.

CONTAINER ID   IMAGE                                  COMMAND                  CREATED              STATUS                   PORTS                                                                                         NAMES
460ea08fe266   airbyte/db:0.35.32-alpha               "docker-entrypoint.s…"   About a minute ago   Up About a minute        5432/tcp                                                                                      airbyte-db
3dec2a2545a8   airbyte/worker:0.35.32-alpha           "/bin/bash -c ${APPL…"   About a minute ago   Up About a minute                                                                                                      airbyte-worker
c32eeeb6abed   airbyte/temporal:0.35.32-alpha         "./update-and-start-…"   About a minute ago   Up Less than a second    6933-6935/tcp, 6939/tcp, 7234-7235/tcp, 7239/tcp, 0.0.0.0:7233->7233/tcp, :::7233->7233/tcp   airbyte-temporal
02ee6ad9e950   airbyte/scheduler:0.35.32-alpha        "/bin/bash -c ${APPL…"   About a minute ago   Up About a minute                                                                                                      airbyte-scheduler
c744159be131   airbyte/webapp:0.35.32-alpha           "/docker-entrypoint.…"   About a minute ago   Up About a minute        0.0.0.0:8000->80/tcp, :::8000->80/tcp                                                         airbyte-webapp
928723df9e57   airbyte/server:0.35.32-alpha           "/bin/bash -c ${APPL…"   About a minute ago   Up About a minute        8000/tcp, 0.0.0.0:8001->8001/tcp, :::8001->8001/tcp                                           airbyte-server

This seems to be the failing part (from airbyte-temporal logs)

PostgreSQL started.                                                                                                                                                                                                                          + update_postgres_schema
+ CONTAINER_ALREADY_STARTED=CONTAINER_ALREADY_STARTED_PLACEHOLDER                                                                                                                                                                            + '[' '!' -e CONTAINER_ALREADY_STARTED_PLACEHOLDER ']'
+ touch CONTAINER_ALREADY_STARTED_PLACEHOLDER                                                                                                                                                                                                touch: CONTAINER_ALREADY_STARTED_PLACEHOLDER: Permission denied
+ temporal-sql-tool --plugin postgres --ep db -u docker -p 5432 create --db temporal                                                                                                                                                         2022-02-21T18:46:29.059Z        ERROR   Unable to create SQL database.  {"error": "pq: database \"temporal\" already exists", "logging-call-at": "handler.go:97"}

Seems so: Docker version 20.10.12, build e91ed57 docker-compose version 1.29.2, build unknown I've restarted docker service just to be sure - no changes

Or this may be the culprit

{
  "level": "fatal",
  "ts": "2022-02-22T14:50:29.871Z",
  "msg": "error starting scanner",
  "service": "worker",
  "error": "context deadline exceeded",
  "logging-call-at": "service.go:233",
  "stacktrace": "<http://go.temporal.io/server/common/log.(*zapLogger).Fatal\n\t/temporal/common/log/zap_logger.go:150\ngo.temporal.io/server/service/worker.(*Service).startScanner\n\t/temporal/service/worker/service.go:233\ngo.temporal.io/server/service/worker.(*Service).Start\n\t/temporal/service/worker/service.go:153\ngo.temporal.io/server/service/worker.ServiceLifetimeHooks.func1.1\n\t/temporal/service/worker/fx.go:80|go.temporal.io/server/common/log.(*zapLogger).Fatal\n\t/temporal/common/log/zap_logger.go:150\ngo.temporal.io/server/service/worker.(*Service).startScanner\n\t/temporal/service/worker/service.go:233\ngo.temporal.io/server/service/worker.(*Service).Start\n\t/temporal/service/worker/service.go:153\ngo.temporal.io/server/service/worker.ServiceLifetimeHooks.func1.1\n\t/temporal/service/worker/fx.go:80>"
}

@Krzysztof Sazon are you able to run

docker ps

without sudo user?

yes, the terminal output I pasted was captured without sudo (although user is in sudoers group)

• running

docker-compose

as

sudo

results in problem not occurring • adding

USER root

in

airbyte-temporal

Dockerfile

changes nothing • user I'm starting

docker-compose

with belongs to the

docker

group

If you copy the airbyte files to another folder and try to run from there, works?

I've copied the whole repo dir from

/home/<me>/airbyte

to

/tmp/airbyte

and it behaves the same - error occurs

Hey do mind deleting the images and volumes and rerun from scratch or is this something you have already done?

Hey @Krzysztof Sazon is your user able to run docker as non-root user? (following this guide)

Just to let you know what what the actual problem: It was docker's proxy settings in

~/.docker/config.json

I had to use proxy again and this time, I've guessed and added some IP ranges to

noProxy

that could be potentially used by airbyte and it's working 🙂