https://linen.dev logo
Join Slack
Powered by
Are there any issues with a Postgres version 17 da...
# replication-ask-ai
d
Are there any issues with a Postgres version 17 database as a source in Airbyte?
k
Based on the knowledge sources, there is one reported issue specifically involving PostgreSQL 17 as a source in Airbyte: CDC Incremental Sync Failure (Debezium Type Conversion Error) A user running self-hosted Airbyte v2.0.1 with PostgreSQL 17.4 and source-postgres connector v3.6.18 reported that incremental CDC syncs were failing across all tables with a Debezium type conversion error (a timestamp column being received as a 
Long
epoch value when the schema expected a 
STRING
type). Full refresh syncs worked correctly, but incremental CDC syncs showed 0 records loaded after the initial sync. [Postgres CDC issue] The suggested temporary workaround was to switch affected streams to Full Refresh sync mode until the issue is resolved. General CDC Requirements to Keep in Mind Regardless of Postgres version, the documentation notes several CDC requirements that apply: [CDC requirements] ā€¢ CDC requires Postgres v10 or above ā€¢ Incremental CDC sync is only supported for tables with primary keys ā€¢ Data must be in tables, not views ā€¢ Schema changes are not automatically supported in CDC mode Beyond the CDC issue above, the knowledge sources do not contain additional information specifically about known incompatibilities or bugs unique to PostgreSQL 17 as a source. If you are experiencing issues, it may be worth checking the latest connector version and the Airbyte GitHub issues tracker for the most up-to-date information. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
Based on the knowledge sources, there are a couple of relevant points regarding PostgreSQL 17 as a source in Airbyte: 1. Version Requirements: The Airbyte Postgres source connector supports Postgres v9.3.x or above for non-CDC workflows and Postgres v10 or above for CDC workflows, so PostgreSQL 17 falls within the supported range. [Postgres Troubleshooting] 2. Reported CDC Issue with PostgreSQL 17: One GitHub issue reports a CDC incremental sync failure with the Postgres source connector (v3.6.18) on PostgreSQL 17.4. The error involves a Debezium type conversion problem where a 
createdAt
timestamp column is received as a 
Long
(epoch timestamp) when the schema expects a 
STRING
type. This caused incremental CDC syncs to show 0 records loaded after the initial sync, while full refresh syncs worked correctly. The issue was escalated to Airbyte's engineering team. [CDC Sync Fails Issue] 3. PgJDBC Compatibility Issue (CDC): There is also a known issue with source-postgres versions 3.6.19 and later where the PgJDBC driver version (
42.7.4
) causes problems with Debezium CDC, leading to repeated pipeline failures or full re-syncs. The proposed workaround is to downgrade to source-postgres 3.6.18 or downgrade the PgJDBC version to below 
42.7.0
. This issue is not specific to PostgreSQL 17 but would affect it as well. [CDC Not Working 3.6.22] In summary, while PostgreSQL 17 is technically within the supported version range, there are active known issues with CDC syncs that may affect it. If you're experiencing problems, switching to Full Refresh mode is a suggested temporary workaround, or downgrading the source-postgres connector to v3.6.18. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
m
@Daniel Nilsson I'm not sure what the bot is on about. Source-postgres is one of the most widely used connectors, and postgres 17 is well supported. Many CDC bugs have been fixed since the version it's talking about. Just use 3.7.2 or the upcoming 3.8.0 and you'll be in good shape.
šŸ‘ 1
6 Views
Open in Slack
PreviousNext
Powered by