We're switching our postgres connector to cdc, and...
# replication-ask-ai
y
We're switching our postgres connector to cdc, and it's working great on a test schema/dataset. What do you suggest we do to safely deploy this in production? I was thinking we could just point to the prod dataset and do the sycn from scratch, or we copy over tables from the cdc test dataset that's working great, as well as copying over the states and switch the destination dataset in airbyte to prod dataset
k
Deploying Postgres CDC to Production Both approaches you're considering are valid, and Airbyte's documentation actually supports a dual-write / state-copy strategy similar to your second option. Here's a breakdown: Option 1: Fresh sync from scratch on prod dataset This is the simplest approach but means re-syncing all historical data from your production Postgres source. For large databases, this can be time-consuming and resource-intensive. Key consideration: For Postgres CDC specifically, you must create a new, dedicated replication slot for the production connection. A single replication slot can only be used by one Airbyte destination — sharing it will result in only partial data being received. [Postgres CDC limitations] Option 2: Copy tables + copy state (recommended for large datasets) This is closer to Airbyte's recommended dual-write upgrade path and avoids re-syncing historical data. The steps would be: 1. Copy the raw data from your test/staging destination to your production destination schema for every stream using an incremental sync mode. [Incremental sync steps] 2. Copy the connection state from your working test connection: ◦ Go to your test connection → Settings tab → Advanced settings ◦ Copy the connection state (which tracks CDC position/incremental sync progress) ◦ On your new production connection, paste the state and click Update State ◦ This ensures historical data is not re-replicated [Incremental sync steps] 1. Create a new Postgres source pointing to your production database with its own dedicated replication slot — do not reuse the test replication slot. [Postgres CDC limitations] 2. Enable the production connection and verify syncs are working before decommissioning the test connection. Important CDC-specific warnings • One replication slot per destination — never share a replication slot between two Airbyte connections/instances. • Schema changes are not automatically supported in CDC mode. If your prod schema differs from your test schema, you'll need to refresh the source schema in the Airbyte UI. [Schema changes forum] • For production environments, it's recommended to set Invalid CDC Position Behavior to "Fail Sync" rather than auto-resync, to avoid unexpected large re-snapshots. [CDC best practices] Summary: Option 2 (copy data + copy state) is the safer path for production if you want to avoid a full historical re-sync, but make sure you set up a fresh replication slot for the production source connector. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
y
@kapa.ai I was thinking more on the line of: 1. Getting a backup of existing tables in the prod dataset 2. Delete them all 3. Create tables from scratch copying from the cdc test dataset 4. in the airbyte connection, change the destination to the production dataset than the cdc test one
k
Your Proposed Approach Your plan is essentially a variation of the dual-write/state-copy strategy and is reasonable. Here's how it maps to what Airbyte supports: Steps aligned with Airbyte guidance 1. Backup existing prod tables — good practice before any migration. 2. Delete prod tables and recreate from CDC test dataset — this is equivalent to copying raw data to the new destination schema, which Airbyte recommends for incremental/CDC connections to avoid re-syncing history. [Incremental sync steps] 3. Switch the Airbyte connection destination to prod dataset — before enabling syncs, you should also copy the connection state from your test connection to the new one: ◦ Go to your test connection → Settings → Advanced ◦ Copy the state and paste it into the new/updated connection via Update State ◦ This prevents Airbyte from re-syncing historical data from scratch [Incremental sync steps] Key concern: Replication slot The knowledge sources do not explicitly cover changing a destination mid-connection, so I want to flag one important CDC-specific risk: if your production source requires its own dedicated replication slot, make sure it is separate from the one used in your test setup. Sharing a replication slot between two connections will result in only partial data being replicated. [Postgres CDC limitations] Summary Your plan is sound, but don't skip copying the connection state before re-enabling syncs — that's the critical step to avoid a full historical re-snapshot on your production source. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.