@kapa.ai I am planning to migrate Airbyte from the old cluster to a new one. Besides the configuration settings in Airbyte, are there any other factors I need to consider, such as migrating the PostgreSQL database or anything else?

Yes, when migrating Airbyte from one cluster to another, you need to consider not only the configuration settings but also the PostgreSQL database that Airbyte uses to store its data. Here are the steps you can follow: 1. Export the configuration from your old Airbyte instance. 2. Disable the old Airbyte instance. 3. Import the configuration to the new Airbyte instance, pointing to the database. To migrate the PostgreSQL database, you can follow these steps: 1. Stop all services and only start the database to prevent any new data/update in it. You can use the following commands:

docker-compose down    docker-compose up -d db

2. Create a backup of the database using the following command:

docker exec airbyte-db pg_dump -U docker airbyte > airbyte_backup.sql

3. Stop the database again with

docker-compose down

. 4. Copy/transfer the SQL file to your new server. 5. On your new server, start Airbyte using

docker-compose up -d

. After the service is running, stop it to rebuild the database with

docker-compose down

. 6. Start only the database with

docker-compose up -d db

. 7. Run the following commands to drop the new database, create a new one, and regenerate your database:

docker exec airbyte-db psql -U docker -c 'drop database airbyte';    docker exec airbyte-db psql -U docker -c 'create database airbyte with owner docker;'    cat airbyte.sql| docker exec -i airbyte-db psql -U docker -d airbyte

8. Restart Airbyte with your backup using

docker-compose up -d

. Please note that you must trigger all connections again to create the temporal workflow for them again. You can use the Airbyte API to do it. For more details, you can refer to these discussions: [Migrating between Airbyte instances](https://discuss.airbyte.io/t/1768), [How to import/export Airbyte to a new instance](https://discuss.airbyte.io/t/3514), and [Connecting to an External Postgres database](https://discuss.airbyte.io/t/2903).

@kapa.ai if i do not migrate database, only migrate the configuration. is it any problem when i sync the stream in new cluster? especially for incremental table.

A message from kapa.ai

If you only migrate the configuration and not the database, you might encounter issues with incremental tables. This is because the state of the incremental sync is stored in the Airbyte's internal database. If you don't migrate the database, the new Airbyte instance won't have the necessary state information to continue the incremental sync from where it left off. Instead, it will start the sync from the beginning, which might lead to data duplication or missing data. In the case of incremental sync, Airbyte keeps track of the last record it successfully replicated (based on the cursor field), and starts replicating from that point in the next sync job. This information is stored in the Airbyte's internal database. If this database is not migrated, the new Airbyte instance won't know from where to start the incremental sync. Therefore, it's recommended to migrate both the configuration and the database when moving Airbyte to a new cluster to ensure the continuity of incremental syncs. For more details, you can refer to these discussions: [Incremental sync failing for Postgres source connector on M1 Mac Docker deployment](https://discuss.airbyte.io/t/824), [Error when incremental sync from postgres to redshift](https://discuss.airbyte.io/t/1342), and [Dbt failure with tabular normalization PostgreSQL -> Snowflake](https://discuss.airbyte.io/t/2552). Also, you can check out the [Airbyte's incremental data synchronization tutorial](https://airbyte.com/tutorials/incremental-data-synchronization#step-2).