Hi team, I am encountering some issues in deploying helm chart version 0.40.18 in our eks. The first time postgres pod initialez I can see the Logs: The files belonging to this database system will be owned by user “postgres”. This user must also own the server process. The database cluster will be initialized with locale “en_US.utf8". The default database encoding has accordingly been set to “UTF8”. The default text search configuration will be set to “english”. Data page checksums are disabled. fixing permissions on existing directory /var/lib/postgresql/data/pgdata ... ok creating subdirectories ... ok selecting dynamic shared memory implementation ... posix selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting default time zone ... UTC creating configuration files ... ok running bootstrap script ... ok sh: locale: not found 2022-10-18 060717.073 UTC [31] WARNING: no usable system locales were found performing post-bootstrap initialization ... ok initdb: warning: enabling “trust” authentication for local connections You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. syncing data to disk ... ok Success. You can now start the database server using: pg_ctl -D /var/lib/postgresql/data/pgdata -l logfile start waiting for server to start....2022-10-18 060717.697 UTC [37] LOG: starting PostgreSQL 13.8 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit 2022-10-18 060717.698 UTC [37] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432" 2022-10-18 060717.704 UTC [38] LOG: database system was shut down at 2022-10-18 060717 UTC 2022-10-18 060717.708 UTC [37] LOG: database system is ready to accept connections done server started CREATE DATABASE /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/000_init.sql ALTER SYSTEM CREATE DATABASE You are now connected to database “airbyte” as user “airbyte”. 2022-10-18 060717.933 UTC [52] ERROR: role “docker” does not exist 2022-10-18 060717.933 UTC [52] STATEMENT: GRANT ALL ON DATABASE airbyte TO docker; psql/docker entrypoint initdb.d/000 init.sql12: ERROR: role “docker” does not exist The bootloader pod starts and hangs with message: 2022-10-18 060732,982 main INFO Loading mask data from ’/seed/specs_secrets_mask.yaml 2022-10-18 060734 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):1091 - Using default value for environment variable CONFIG_DATABASE_USER: ‘airbyte’ 2022-10-18 060734 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):1091 - Using default value for environment variable CONFIG_DATABASE_PASSWORD: ‘*****’ 2022-10-18 060734 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):1091 - Using default value for environment variable CONFIG_DATABASE_URL: ‘jdbc:postgresql://airbyte-db-svc:5432/db-airbyte’ 2022-10-18 060735 [32mINFO[m c.z.h.HikariDataSource(<init>):80 - HikariPool-1 - Starting... 2022-10-18 060735 [32mINFO[m c.z.h.HikariDataSource(<init>):82 - HikariPool-1 - Start completed. 2022-10-18 060735 [32mINFO[m c.z.h.HikariDataSource(<init>):80 - HikariPool-2 - Starting... 2022-10-18 060735 [32mINFO[m c.z.h.HikariDataSource(<init>):82 - HikariPool-2 - Start completed. 2022-10-18 060736 [32mINFO[m i.a.c.EnvConfigs(getEnvOrDefault):1091 - Using default value for environment variable SECRET_PERSISTENCE: ‘TESTING_CONFIG_DB_TABLE’

For airbyte-db it’s ok to restart a few times in a row before starting accepting incoming connections

It restarts only once…giving the upper message, the pb is that bootloader remains in running, without completing its status

Redeploy chart with

--set airbyte-bootloader.env_vars.LOG_LEVEL=DEBUG

Yes, you were right the connection to the db was failing. Now it is fixed and everything is deployed with success except to webapp which fails with:

2
2022/10/18 11:30:16 [emerg] 1#1: host not found in upstream "airbyte-gd-euc1-airbyte-airbyte-server-svc:8001" in /etc/nginx/conf.d/default.conf:2

1
nginx: [emerg] host not found in upstream "airbyte-gd-euc1-airbyte-airbyte-server-svc:8001" in /etc/nginx/conf.d/default.conf:2

The chart is being deployed using argo so pb I need to provide more info to webapp?

@[DEPRECATED] Marcos Marx turned this thread into Zendesk ticket 2889 to ensure timely resolution!

Hmm, can you describe services?

kubectl describe svc airbyte-gd-euc1-airbyte-airbyte-webapp-svc -n airbyte Name: airbyte-gd-euc1-airbyte-airbyte-webapp-svc Namespace: airbyte Labels: app.kubernetes.io/instance=airbyte-gd-euc1-airbyte app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=webapp app.kubernetes.io/version=0.40.14 helm.sh/chart=webapp-0.40.18 Annotations: <none> Selector: app.kubernetes.io/instance=airbyte-gd-euc1-airbyte,app.kubernetes.io/name=webapp Type: ClusterIP IP Family Policy: SingleStack IP Families: IPv4 IP: 172.20.60.36 IPs: 172.20.60.36 Port: http 80/TCP TargetPort: http/TCP Endpoints: Session Affinity: None Events: <none>

Would be great if you shared the issue here :) Who knows maybe it’ll help someone with similar issue

Yes, nodeSelector for the webapp pod was different than the one for server pod so they were unable to communicate.

@Dana Vatavu Facing the same issue.. The bootloader was running and on debugging the logs found that the connection was failing. Would be great if you can share what did you have to change the database host to?