@kapa.ai I am having intermittent problems with a Salesforce to Snowflake connection. The majority of my streams are setup as incremental. In some occasions (I am still not sure about the root cause) I register large data losses in the target tables (>80% records missing, corresponding to several millions of missing records). When I refresh the individual streams, usually the problems disappears. What can I verify to diagnose the problem? What can I do to prevent it?

Diagnosing & Preventing Data Loss in Salesforce → Snowflake Incremental Syncs There is a known root cause that closely matches your symptoms. Here's what to check and how to prevent it. Known Root Cause: Sliding start_date Window When

start_date

is not explicitly configured, the Salesforce connector historically defaulted to

now - 2 years

. This creates a sliding window that shifts forward over time. The data loss sequence is: 1. Initial sync runs at time T₀ with

start_date = T₀ - 2 years

2. Incremental syncs continue normally using cursor state 3. A schema change (e.g., a new column added in Salesforce) triggers a truncate refresh 4. State is cleared → connector restarts from

start_date = T₁ - 2 years

5. Records with

SystemModstamp

between

(T₀ - 2 years)

and

(T₁ - 2 years)

are permanently lost This is exactly the pattern reported in a confirmed bug, and a fix was merged in connector version 2.7.23 that replaces the dynamic default with a fixed date (

2006-01-01T00:00:00Z

). [PR #77664] What to Verify 1. Connector version: Check if you are running source-salesforce < 2.7.23. If so, upgrading to 2.7.23+ is the primary fix. [PR #77664] 2. Explicit

start_date

configuration: Check whether you have a

start_date

explicitly set in your connector config. If not, you were affected by the sliding window bug. Setting an explicit

start_date

(e.g.,

2006-01-01T00:00:00Z

) protects you regardless of connector version. [PR #77664] 3. Schema change history: Correlate your data loss events with schema changes in Salesforce (new columns, renamed fields, etc.). If losses coincide with schema propagation events, this confirms the root cause. [GitHub Issue #77627] 4. Schema propagation setting: If your connection is set to "Propagate field changes only", a schema change can trigger a truncate refresh on affected streams, clearing the cursor state. [GitHub Issue #77627] 5. Eventual consistency / missing recent records: Separately, Salesforce's API does not guarantee immediate availability of recently created/updated records. If you see missing records near the trailing edge of sync windows, consider increasing the Lookback Window in the connector config (e.g., from

PT10M

to

PT30M

or

PT1H

). [Salesforce docs] How to Prevent It 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements