My team is planning for the initial CDC snapshot with Aurora Postgres. They are also wondering about the setting of max_slot_wal_keep_size and monitoring/alerting best practices during the initial bootstrap and then ongoing incremental operation. What are the best resources to read up on best practices for these, and/or what are the most relevant posts on these topics?

For Aurora Postgres CDC with Airbyte, the most relevant “best practices” and background resources in the docs and community are: 1. General CDC behavior and initial snapshot with Postgres/Aurora ◦ Airbyte’s core CDC overview explains how the initial snapshot works (full

SELECT

snapshot first, then WAL-based incremental) and what metadata columns you’ll see. [CDC overview] ◦ The Cloud SQL for Postgres CDC setup guide is still useful conceptually for Aurora because it describes how Airbyte uses logical replication, replication slots, and WAL, and when CDC is recommended (large DBs, need deletes, no good cursor). [Cloud SQL CDC] 2. Initial snapshot / bootstrap tuning ◦ Initial Waiting Time (per-source setting): governs how long Airbyte waits during snapshot and CDC phases; guidance on when to increase it (low-activity DBs, many schemas/tables) vs keep default. [CDC best practices – source config] ◦ Initial Load Timeout (snapshot max duration, up to 24h): recommendations by database size (8h default, 12–24h for large/very large DBs) and advice to monitor the first sync to tune it. [CDC best practices – source config] 3. WAL retention and

max_slot_wal_keep_size

-type concerns ◦ Airbyte’s CDC best practices talk about WAL retention period at the database level and how it must be longer than your sync frequency, with concrete examples (e.g., daily sync → 2–3 days minimum, 7 days recommended). This is the key conceptual guidance that maps to Aurora’s WAL retention /

max_slot_wal_keep_size

decisions. [CDC best practices – DB config] ◦ Postgres troubleshooting explains what happens when WAL needed for CDC is purged and how to mitigate (sync more frequently, increase

wal_keep_size

), which is directly analogous to sizing

max_slot_wal_keep_size

and retention in Aurora. [Postgres troubleshooting] 4. Behavior when WAL/LSN is lost (important for alerting strategy) ◦ The Invalid CDC Position Behavior setting describes what happens when the CDC position becomes invalid (e.g., WAL recycled): either automatically re-sync (full snapshot) or fail the sync. It also gives recommendations (production/large DBs → “Fail Sync”). This is central to how you’ll want to alert on WAL/slot issues. [CDC best practices – source config] 5. Postgres replication slot / WAL growth edge cases (good for operational playbooks) ◦ The Postgres troubleshooting page has a section “Under CDC incremental mode, there are still full refresh syncs” that walks through WAL removal scenarios and mitigation (sync frequency,

wal_keep_size

). [Postgres troubleshooting] ◦ A community thread specifically about replication slot not advancing when there are no changes (safe_wal_size shrinking until slot is lost) is very relevant to ongoing operations and monitoring. The workaround described is to ensure at least one change per sync (e.g., a “heartbeat” table) so the slot advances and WAL is released. [Forum: replication slot safe_wal_size] 6. Sync frequency vs WAL retention (for ongoing incremental) ◦ The CDC best practices “General Airbyte configuration” section gives concrete guidance on sync frequency vs WAL retention (e.g., near real-time every 1–4h with 7-day retention, daily reporting with 7-day retention, etc.), which you can map to Aurora’s WAL retention and

max_slot_wal_keep_size

so that WAL isn’t recycled before Airbyte reads it. [CDC best practices – general config; CDC best practices – DB config] 7. Initial CDC waiting time (for large change backlogs) ◦ Postgres troubleshooting has an “(Advanced) Setting up initial CDC waiting time” section that explains the initial wait (default 300s, range 120–1200s) and when to increase/decrease it depending on backlog vs no changes. This is useful when bootstrapping a large Aurora instance or when you expect a big WAL backlog. [Initial CDC wait] ------------------------- How to use these for planning & monitoring While the docs don’t give Aurora-specific

max_slot_wal_keep_size

numbers or concrete alert rules, the pieces above are what you’ll want to combine: • Use the WAL retention vs sync frequency guidance to size Aurora’s WAL retention and

max_slot_wal_keep_size

. [CDC best practices – DB config] • Use Invalid CDC Position Behavior + Postgres troubleshooting to define alerts on: ◦ Sync failures due to invalid CDC position. ◦ Unexpected full refreshes after CDC was already running. [CDC best practices – source config; Postgres troubleshooting] • Use the forum thread to understand the “no changes → slot not advancing → WAL growth” pattern and consider a heartbeat table or similar if your workload is bursty. [Forum: replication slot safe_wal_size] If you’d like, I can help you turn these into a concrete checklist for your Aurora environment (e.g., suggested sync frequency, which Airbyte settings to start with, and what to watch during the first snapshot). 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.