<@U04S8PLJWMT> is there a way to backfill only one...
# replication-ask-ai
a
@kapa.ai is there a way to backfill only one stream within a connection of many? running a mongodb (MongoDb v2.0.7) -> bigquery connection, sync is based on cdc
k
The knowledge sources don't provide explicit documentation on selectively backfilling a single stream in a CDC-based MongoDB → BigQuery connection, but here's what can be gathered: Single-Stream Clear/Backfill for MongoDB CDC Using the Clear feature (Airbyte v1.0+): You can trigger a clear (formerly "reset") for a single stream without affecting others: 1. Navigate to your connection's Status page. 2. Click the three grey dots next to the specific stream you want to backfill. 3. Select "Clear data" — this will only clear that stream's data in the destination. 4. Then run a sync to reload data for that stream. [Clearing Data Docs] Important caveats for MongoDB CDC: • The MongoDB V2 connector uses Global state (shared CDC state across streams), not per-stream isolated state. This means that clearing/resetting one stream's state may interact with the shared CDC resume token. [State Types] • When a stream's initial snapshot state is cleared, the connector will re-run the full initial snapshot for that stream before resuming CDC. The
InitialSnapshotHandler
starts from the beginning of the collection if no existing state is found for that stream. [MongoDbCdcInitializer] • There is no built-in way to backfill only a partial date range (e.g., last 1 month) for a single stream — the initial snapshot always reads the full collection. [Forum: Partial Data Fetching] In summary: You can clear and re-sync a single stream via the UI, but it will perform a full collection snapshot for that stream, not a partial/date-filtered backfill. If you need to limit the data, the suggested workaround is to create a MongoDB view that filters the data you want, and sync that view instead. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.