<@U04S8PLJWMT> Good morning! I am using the Square...
# replication-ask-ai
j
@kapa.ai Good morning! I am using the Square connector. I have 1/1/2024 entered as the replication start date, and I can see data in the Square UI for sales in 2024. However, I'm not getting any sales from 2024 landing in my table. My sync is running successfully, and I do see more recent sales showing up correctly. How can I get the 2024 sales into my destination?
k
Good morning! This is a common situation with incremental syncs in Airbyte. Here's what's likely happening: Why You're Missing 2024 Data When an incremental sync runs successfully, Airbyte saves the latest cursor value (e.g., the most recent
updated_at
timestamp it encountered) as the state for the next sync. On subsequent syncs, Airbyte uses that saved cursor value — not your configured start date — as the new starting point. [Incremental sync] So if your connection was first set up or synced at some point after 1/1/2024, the cursor may have already advanced past that date, meaning older 2024 records are no longer being requested. How to Get the 2024 Data To backfill the missing 2024 sales, you have a couple of options: 1. Reset the connection: In Airbyte, you can perform a Reset on the connection (or specific streams). This clears the saved cursor state, forcing the next sync to start fresh from your configured start date (1/1/2024). After the reset, run a sync and it should pull all data from 1/1/2024 onward. 2. Check your sync mode: If you're using Full Refresh - Overwrite or Full Refresh - Append, all data should be pulled every time regardless of the cursor. If you're using an incremental mode, the cursor state is what controls what gets synced. The knowledge sources do not contain specific troubleshooting steps for the Square connector's start date behavior beyond what's described above, but resetting the stream state is the standard Airbyte approach to backfilling data when the cursor has moved past your desired start date. [Incremental sync] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
@kapa.ai is resetting the connection the same as clicking "Refresh your data"? If not, can you tell me the clickpath to get to the reset connection option?
k
Great follow-up question! These are actually different operations in Airbyte. Here's a breakdown: Refresh vs. Clear (formerly "Reset") • Refresh your data — A Refresh Sync erases the current cursor for incremental streams and re-syncs the data from the beginning. Importantly, it does not cause data downtime — your existing data remains queryable in the destination until the refresh completes successfully, at which point the new data is swapped in. [Refreshing your data] • Clear (previously called "Reset") — A Clear deletes all records from your destination tables and resets the cursor state. Unlike a Refresh, it does cause data downtime — your tables will be empty once the Clear completes. [Clearing your data] For your use case (backfilling 2024 sales), Refresh your data is actually the better option since it avoids data downtime. Click Path for Each Option To Refresh your data (recommended): • Navigate to your connection's Status page → click the three grey dots next to the relevant stream → select Refresh data. [Refreshing your data] To Clear data (all streams): • Navigate to your connection's Settings tab → click Clear data → confirm the selection. [Clearing your data] To Clear data (single stream): • Navigate to your connection's Status page → click the three grey dots next to the stream → select Clear data. [Clearing your data] Given your sync mode, the available refresh options may vary — for example, Refresh and Remove Records and Refresh and Retain Records are only available for Incremental | Append + Dedupe mode. [Refreshing your data] šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
j
I did a refresh of my data, but am still not seeing the complete set of 2024 data I expect. It looks like some, but not all, of the 2024 data is populating.
@kapa.ai I did a refresh of my data, but am still not seeing the complete set of 2024 data I expect. It looks like some, but not all, of the 2024 data is populating.
k
That's a frustrating situation. Based on the knowledge sources available, here are a few things to consider: Possible Reasons for Incomplete 2024 Data 1. The source may not retain all historical records. The documentation notes that not all sources keep their history forever. If you performed a Refresh and Remove Records sync, and your Square source does not retain all records from 2024, this could lead to data loss — the refresh can only bring back what the source currently has available. [Refreshing your data] 2. The refresh type matters. If you used Refresh and Remove Records, only data newly synced since the refresh was initiated will appear in the destination's final tables. If some 2024 records were missing from the source at the time of the refresh, they won't appear. [Refreshing your data] What to Check • Verify the data exists in the source: Confirm in the Square UI that the missing 2024 records are still accessible via the API (not just the UI). Sometimes data visible in a UI is not fully accessible via API depending on filters or API limitations. • Check your sync mode: If you're on Incremental | Append + Dedupe, you could try a Refresh and Retain Records to preserve existing data while re-syncing, which would help you compare what came back vs. what was already there. Unfortunately, the knowledge sources don't contain specific details about the Square connector's API behavior or limitations that might explain why only partial 2024 data is returning. I'd recommend checking the Square connector documentation directly or reaching out to Airbyte support for connector-specific guidance. šŸ‘ 0 button šŸ‘Ž 0 button ā„¹ļø Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.