<@U04S8PLJWMT> issue of postgres to postgres cdc r...
# ask-ai
s
@kapa.ai issue of postgres to postgres cdc replication is it resolved ?
k
It’s partially resolved, but not universally “fixed once and for all.” From the documents: - Several specific Postgres→Postgres CDC bugs have been fixed in later Airbyte / connector versions. Examples: - Missing
deleted_at
/ deletes not reflected correctly in BigQuery and Postgres CDC were fixed by upgrading Airbyte to ≥ 0.39.20 and/or using newer Postgres connectors.[[CDC BigQuery deletes](https://discuss.airbyte.io/t/461)] - A regression where incremental CDC state wasn’t updated (leading to WAL bloat / resyncs) was addressed by upgrading the Postgres source connector to 0.4.31+.[[CDC not consuming slot](https://discuss.airbyte.io/t/1791)] - Other CDC problems are still open or require workarounds: - There are ongoing issues with Debezium / CDC throwing errors (e.g.,
IndexOutOfBoundsException
after long runs) on newer connectors; this is still an open GitHub issue.[[Debezium crash](https://github.com/airbytehq/airbyte/issues/65065)] - Some CDC bugs around LSN / state handling and PgJDBC are open; users report needing to pin the Postgres source connector to 3.6.18 instead of newer 3.6.2x/3.6.30 versions to avoid offset / LSN problems.[[CDC LSN bug](https://github.com/airbytehq/airbyte/issues/55846); [PgJDBC issue](https://github.com/airbytehq/airbyte/issues/49802)] - Partitioned-table CDC and some deployment-specific cases (Aurora caching layer, AlloyDB, etc.) still have limitations or open issues.[[Partitioned table CDC](https://discuss.airbyte.io/t/1147); [Postgres troubleshooting](https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#vendor-specific-connector-limitations)] Because of this, whether your Postgres→Postgres CDC issue is “resolved” depends on: 1. Your Airbyte version 2. Postgres source connector version 3. Exact symptom (e.g., deletes not showing, WAL slot not advancing, connection always full-resyncing, etc.) If you share: - Airbyte version - Postgres source & destination connector versions - Brief description of the current problem I can map it to the closest known issue/fix from the docs and suggest a specific upgrade or workaround (including when pinning to an older connector is recommended).
s
@kapa.ai, i am talking about the numeric column issue
k
A message from kapa.ai
For numeric columns in Postgres→Postgres CDC, the issues in the sources are not described as fully resolved, and in some cases only workarounds are documented. What we do know from the provided material: - There are CDC issues in Postgres source connectors that affect numeric handling / state in general, and several users had to downgrade the Postgres source connector to 3.6.18 to get CDC working reliably again.[[CDC LSN bug](https://github.com/airbytehq/airbyte/issues/55846); [PgJDBC CDC issue](https://github.com/airbytehq/airbyte/issues/49802)] - For other databases, we see concrete numeric-conversion problems in CDC (e.g. MSSQL CDC to S3 failing to convert numeric to double during Avro conversion), which were only worked around by changing destination type (Databricks) and upgrading, not by a specific numeric-type bugfix.[[MSSQL numeric CDC](https://discuss.airbyte.io/t/2901)] - The general requirement for partner destinations is to preserve numeric fidelity (use large
DECIMAL
/
DOUBLE
, avoid lossy float for decimals), but this is a guideline, not a confirmation that all Postgres CDC numeric issues are fixed in code.[[Data fidelity](https://docs.airbyte.com/platform/connector-development/partner-certified-destinations#data-fidelity-and-data-types)] The sources you provided do not explicitly state that a specific “numeric column CDC bug” for Postgres→Postgres has been fixed in a certain version, nor do they reference a dedicated GitHub issue for that exact symptom. Because of that, the safest stance based on these documents is: - If you are seeing numeric-column problems with Postgres CDC, the only clearly documented mitigation is: - ensure you are on a recent Airbyte version, and - if you hit CDC regressions, try pinning the Postgres source connector to 3.6.18, which users report as stable for CDC.[[CDC LSN bug](https://github.com/airbytehq/airbyte/issues/55846)] - Beyond this, the knowledge sources don’t provide enough detail to say “the numeric column CDC issue is definitively resolved in version X”. If you can share: - Exact behavior (e.g. wrong numeric values, type cast errors, sync failures) - Your Airbyte + Postgres connector versions I can compare that to the known issues/workarounds in these docs and point you to the closest documented path (upgrade vs. downgrade recommendations).