@kapa.ai, airbyte silenlty stops replication of incremental tables

Silent Replication Stops in Airbyte Incremental Syncs There are several known causes for Airbyte silently stopping or dropping records during incremental syncs. Here's a summary of the known issues and their fixes: 1. Cursor Boundary / Precision Issues Records sitting exactly at the cursor's upper bound can be silently dropped. This was specifically identified in

source-snowflake

, where timestamps with sub-microsecond precision caused rows to be excluded from the

WHERE

clause. [Snowflake cursor fix] The underlying query looks like:

SELECT * FROM table WHERE cursor_field >= 'last_sync_max_cursor_field_value'

If a record's cursor value doesn't exceed the stored max (e.g., due to rounding or same-day updates), it won't be picked up. [Incremental Append docs] 2. Cursor Field Not Updated on Source Records If records are modified in the source without updating the cursor field, they will be silently skipped. Airbyte only picks up records where

cursor_field >= last_sync_max_cursor_value

. [Incremental Append docs] 3. Large JSONB Tables / Silent 0-Row Syncs (Postgres) A known issue with

source-postgres

(v3.7.0) on large JSONB tables using

ctid

-based initial sync: the sync appears to complete successfully but loads 0 rows with no errors logged. [Postgres ctid issue] 4. Incremental Mode Not Available on Some Tables (Postgres) In certain versions of

source-postgres

, incremental mode was hidden for tables with nullable cursor columns or views. This was resolved in version 1.0.11 (or by downgrading to 1.0.2). [Postgres incremental option] [Postgres views issue] 5. Timezone-Related Silent Record Loss (NetSuite) In

source-netsuite

, records modified between the sync run time and local midnight in the account's timezone are permanently lost due to date truncation in cursor slices. [NetSuite bug] 6. Duplicate Rows in Incremental | Append Mode

Incremental | Append

does not deduplicate — it provides at-least-once delivery, so records at the inclusive cursor boundary may be re-emitted. If deduplication is needed, use Incremental | Append + Deduped mode (which requires a primary key). [Duplicate rows issue] General Recommendations • Upgrade connectors to the latest version, as many silent failures are version-specific bugs. • Use CDC (Change Data Capture / WAL) if your source supports it, to avoid cursor-based limitations entirely. [Incremental Append docs] • If you suspect silent data loss, check job logs under Job History in the Airbyte UI and look for record counts. Could you share more details about your specific source/destination connectors and versions? That would help narrow down the exact cause. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.