@kapa.ai Added 2 new streams to a connection and forgot to make it schemaless. The connection was for GCS to Snowflake. I cleared only those 2 streams, and on clearing now it is giving error - Warning from destination: sourceStats must be set with recordCount for state message id=1 What is wrong?

Thinking completed Error: sourceStats must be set with recordCount for state message id=1 This error originates from the Airbyte bulk load CDK's

StateStore

. Looking at the relevant code:

val stats =
    requireNotNull(msg.sourceStats) {
        "sourceStats must be set with recordCount for state message id=${key.id}"
    }
histogramStore.acceptExpectedCounts(key, stats.recordCount)

The

StateStore

strictly requires that every state (checkpoint) message includes a

sourceStats.recordCount

field. If this field is missing or null, the error is thrown. [StateStore source] What's likely happening When you cleared those 2 streams and triggered a re-sync, the source (GCS) emitted a state message for those streams that did not include the

sourceStats.recordCount

field. This is particularly common with empty streams — streams that have no records to sync. The state message is generated, but the record count metadata is not properly populated. This is a known issue in the CDK when handling empty or newly-added streams in Socket/speed mode (which is the default for Snowflake destination v4.0.0+). A related GitHub issue confirms: > "Empty streams appear to trigger a code path where the

partition_id

is not properly included" and state messages lack required fields. [GitHub Issue] What you can do 1. There is no user-side workaround for this specific error — it requires a fix in the CDK or destination connector, as confirmed by the Airbyte engineering team. [GitHub Issue] 2. Report or track the issue: Check if GitHub issue #72385 covers your case (it's currently open). You can add your reproduction details there. 3. In the meantime, you could try manually dropping the tables for those 2 streams in Snowflake and re-triggering the sync, as the Airbyte docs suggest for failed clears: "To resolve a failed clearing of data, you should manually drop the tables in the destination so that Airbyte can continue syncing accurately." [Clear Data Docs] The root cause is a CDK-level bug, not a misconfiguration on your end. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.