Hey <@U04S8PLJWMT>, I noticed there are some missi...
# ask-ai
l
Hey @kapa.ai, I noticed there are some missing data in one of my tables in the airbyte_raw schema two days ago. Is it possible for me to do some kind of partial refresh by deleting the records in the past two days from the schema? Would airbyte's cursor be 'resetted' to the previous two days?
k
You can “rewind” an incremental stream, but not by directly deleting rows from the
airbyte_raw
table and expecting Airbyte’s cursor to follow automatically. ### 1. What Airbyte supports Airbyte provides Refresh Syncs that explicitly reset the cursor for incremental streams, effectively starting them over “from the beginning.”[[Refreshing data](https://docs.airbyte.com/platform/operator-guides/refreshes)] - A Refresh Sync erases the current cursor for the selected incremental stream(s).[[Refreshing data](https://docs.airbyte.com/platform/operator-guides/refreshes)] - It will then re-import all records again for those streams (as long as they are still present in the source).[[Refreshing data](https://docs.airbyte.com/platform/operator-guides/refreshes)] This is not limited to the last two days; it replays the full history of that stream from the source. Separately, Airbyte may also drop and re-create final tables (“soft reset”) as part of typing & deduping, but that is driven by Airbyte itself for schema/refresh reasons, not by you manually deleting raw data.[[Final table re-creation](https://docs.airbyte.com/platform/using-airbyte/core-concepts/typing-deduping#on-final-table-re-creation)] There is a manual procedure documented to force a full re-build of a final table from raw data (drop final table, set
_airbyte_loaded_at
to
NULL
, then run a sync), but that assumes you still have the raw rows; it doesn’t support selectively reprocessing just a date range.[[Final table re-creation](https://docs.airbyte.com/platform/using-airbyte/core-concepts/typing-deduping#on-final-table-re-creation)] ### 2. About “partial refresh” for the last 2 days The documentation and forum content do not describe a supported way to: - Delete only the last N days of rows from
airbyte_raw
, and - Have Airbyte automatically adjust its cursor to that point and re-sync just that range. In fact, one forum answer explicitly notes that partial initial loads (starting from a later point instead of from the beginning) are not supported – the first sync is always full for incremental streams, and the suggested workaround is to adjust the source (e.g., sync from a view with filtered data).[[Partial data fetching](https://discuss.airbyte.io/t/3381)] So: - If you manually delete rows in
airbyte_raw
, Airbyte’s stored cursor/state will not automatically be reset to two days ago. - The documented “refresh” functionality is all-or-nothing per stream (rewind the cursor entirely), not “only the last X days.”[[Refreshing data](https://docs.airbyte.com/platform/operator-guides/refreshes)] ### 3. Practical options Based on the docs: 1. If you can afford a full re-read of the stream: Use a Refresh Sync for that stream (“refresh and retain” or “refresh and remove” depending on your sync mode and needs). This will reset the cursor and re-import all data from the source, fixing missing rows if they’re still present there.[[Refreshing data](https://docs.airbyte.com/platform/operator-guides/refreshes)] 2. If you truly need partial backfill only (last 2 days): The knowledge sources do not document an official, cursor-aware way to do this on a running connection. The recommended patterns in similar situations are: - Use a separate connection that does a one-time full re-ingestion of a defined period (e.g., last 12 months / last month) and then stop it, while your main incremental connection continues.[[One-time partial resync](https://discuss.airbyte.io/t/1992)] - Or adjust the source-side data (e.g., a view that only exposes the date range to re-ingest) and use full refresh on that auxiliary connection.[[Partial data fetching](https://discuss.airbyte.io/t/3381)] Given what’s in the docs, there is no documented, supported mechanism where simply deleting two days of data from
airbyte_raw
will cause Airbyte to reset its cursor to that point and backfill exactly those days.