Hi! I am wondering how upgrading airbyte to v0.40....
# replication-troubleshooting
h
Hi! I am wondering how upgrading airbyte to v0.40.26 is working for anyone as it seems that the temporalio/auto-setup:1.7.0 move to airbyte/temporal-auto-setup:1.13.0 has broken airbyte upgrade. I have tested this both using helm charts deployment to GKE and using stable overlay deploying to local minikube cluster. Error:
Copy code
+ temporal-sql-tool --plugin postgres --ep airbyte-db-svc -u docker -p 5432 create --db temporal
2023-01-03T17:19:34.826Z ERROR Unable to create SQL database. {"error": "pq: database \"temporal\" already exists", "logging-call-at": "handler.go:97"}
2023/01/03 17:19:35 Loading config; env=docker,zone=,configDir=config
2023/01/03 17:19:35 Loading config files=[config/docker.yaml]
{"level":"info","ts":"2023-01-03T17:19:35.468Z","msg":"Updated dynamic config","logging-call-at":"file_based_client.go:143"}
{"level":"info","ts":"2023-01-03T17:19:35.469Z","msg":"Starting server for services","value":["history","matching","frontend","worker"],"logging-call-at":"server.go:123"}
Unable to start server. Error: sql schema version compatibility check failed: version mismatch for keyspace/database: "temporal". Expected version: 1.6 cannot be greater than Actual version: 1.4
The problem seems to be in the
update-and-start-temporal.sh
:
Copy code
update_postgres_schema() {
  { export SQL_PASSWORD=${POSTGRES_PWD}; } 2> /dev/null

  CONTAINER_ALREADY_STARTED="CONTAINER_ALREADY_STARTED_PLACEHOLDER"
  if [ ! -e $CONTAINER_ALREADY_STARTED ]; then
      touch $CONTAINER_ALREADY_STARTED
      temporal-sql-tool --plugin postgres --ep "${POSTGRES_SEEDS}" -u "${POSTGRES_USER}" -p "${DB_PORT}" create --db "${DBNAME}"
      ...
  fi
Steps to reproduce: (kubernetes deployment, for example minikube) 1. git checkout v0.40.25 2. kubectl apply -k kube/overlays/stable 3. git checkout v0.40.26 4. kubectl apply -k kube/overlays/stable This will result in airbyte-temporal pod crashing Fix (either works): • edit the deployment for airbyte-temoral and update the image used to temporalio/auto-setup:1.7.0 • revert
kube/resources/temporal.yaml
to previous version (basically changing the spec container image) and deploy
r
Hi, there is these issues on github : https://github.com/airbytehq/airbyte/issues/20870 https://github.com/airbytehq/airbyte/issues/16437 The workaround is to drop tables temporal and temporal_visibility... I'm not yet in production so it's fine for me but honestly it's not a great workaround because we lose temporal states, history etc. I don't get all the flaws of this solution, so it could be great to have a temporal db migration working fine for those who can't simply delete tables
g
@Remi Nonnon, we did the same...