Hello everyone, new here, really love the postgres...
# ask-community-for-troubleshooting
y
Hello everyone, new here, really love the postgres<>postgres connection for replication and will soon use that in production. One question before that, what is the meaning of non-breaking schema changes ?
k
A message from kapa.ai
New message text here
y
@kapa.ai for a new column in source table which you told as non-breaking schema, I want that column to be added and populate for all existing rows in destination also. I don't want to ignore this neither disable connection. I want it to run and fix the issue
k
A message from kapa.ai
New message text here
y
@kapa.ai I will not be using cdc, I will use incremental deduped + history mode. So I will follow below steps: 1. add new column in source table 2. in connection, click on refresh source schema 3. click on sync now This should now add new column in destination and also populate its values in history _scd and final tables in destination
k
A message from kapa.ai
New message text here
y
@kapa.ai when there is breaking schema changes like renaming column, what are the steps for this ? Do i have to create new connection ? I mean, from where I have to trigger a full-refresh overwrite
k
A message from kapa.ai
Sorry @Yusuf Mirkar, I seem to be having a bit of trouble. Don't worry though, I've pinged my support folks and they will try to fix me up! 🧑‍💻 Perhaps try again in 5mins?
y
@kapa.ai up now ?
k
A message from kapa.ai
I'm sorry, but I'm not sure what you're asking. If you have a question related to Airbyte, please provide more context or details, and I'll do my best to help you based on the information provided in the extracts above.
y
@kapa.ai please check previous conversation in same thread. tldr; when there is breaking schema changes like renaming column, what are the steps for this ? Do i have to create new connection ? I mean, from where I have to trigger a full-refresh overwrite
k
A message from kapa.ai
A message from kapa.ai
When there are breaking schema changes like renaming a column, you can follow these steps: 1. Make the schema changes in your source table (e.g., renaming a column). 2. In Airbyte, go to the connection settings and click on "Refresh Source Schema" [source](https://airbytehq.slack.com/archives/C021JANJ6TY/p1675703730453619). 3. Since the Incremental Deduped + History mode is not able to handle all types of source schema changes, it is recommended to trigger a Full Refresh - Overwrite sync to correctly replicate the data with the new schema changes [source](https://docs.airbyte.com/understanding-airbyte/connections/incremental-deduped-history#known-limitations). To trigger a Full Refresh - Overwrite sync, you can create a new connection with the Full Refresh - Overwrite mode: 1. Go to the Connections tab in Airbyte and click on "New Connection." 2. Select your source and destination. 3. Set the name of the connection and the destination stream prefix as needed. 4. Choose the "Full Refresh - Overwrite" sync mode [source](https://airbyte.com/tutorials/full-data-synchronization#database-synchronization-with-full-refresh-overwrite). 5. Click on "Set up Connection" and then "Sync Now" to start the sync process. This will overwrite the existing data in the destination with the new data, reflecting the schema changes you made in the source table.
When there are breaking schema changes like renaming a column, you can follow these steps: 1. Make the necessary schema changes in your source table (e.g., renaming a column). 2. In Airbyte, go to the connection settings and click on "Refresh Source Schema" [source](https://airbytehq.slack.com/archives/C021JANJ6TY/p1675703730453619). 3. Since Incremental Deduped + History mode doesn't handle schema changes like renaming columns well, it's recommended to trigger a Full Refresh - Overwrite sync to correctly replicate the data with the new schema changes [source](https://docs.airbyte.com/understanding-airbyte/connections/incremental-deduped-history#known-limitations). To trigger a Full Refresh - Overwrite sync, you can create a new connection with the Full Refresh - Overwrite mode: 1. Go to the Airbyte dashboard and click on "New Connection." 2. Select your source and destination. 3. Set the name of the connection and choose the "Full Refresh - Overwrite" sync mode. 4. Configure any other necessary settings and click "Save Changes." After setting up the new connection, a sync should automatically start, and the data will be replicated to the destination with the new schema changes. For more information on creating a Full Refresh - Overwrite connection, you can refer to this tutorial: [Explore Airbyte's full refresh data synchronization](https://airbyte.com/tutorials/full-data-synchronization).
y
@kapa.ai and then from next sync onwards if i want an incremental deduped+history mode, I just have to edited the same connection or again create new connection ?
k
A message from kapa.ai
@Yusuf Mirkar That's a great question, but unfortunately I am only capable of responding to a maximum of five questions per thread. Don't hesitate to create a new thread for any further questions you have though. I'm here to help!