any tips on how to speed up a stripe historical in...
# replication-troubleshootingr
Robert Put
10/28/2022, 8:53 PMany tips on how to speed up a stripe historical initial sync? its taking over 72 hours.... at about 1000 records a minute
Robert Put
10/28/2022, 9:05 PMlookback window, 0 days
data request time increments, 365 days
m
Marcos Marx (Airbyte)
10/28/2022, 9:39 PMHello Robert, in most cases connectors are configured to use the maximum page size allow by the API service to each request to speed up the sync. Are you seeing a lot of backoff messages during the sync? Other reason could be the time to normalize the data. Could you share the sync log to analyze the step took longer to finish?
r
Robert Put
10/28/2022, 9:42 PMno, no backoff messages
and no normalization steps, was just extraction
i actually don't have the old logs, didn;t save them during an upgrade
but have these from the current sync that start 5 hours ago
Robert Put
10/28/2022, 9:43 PMd5778ff6_aa12_4502_b422_20323ade5c1e_logs_1028_txt.txt
Robert Put
10/28/2022, 9:47 PMand its just incremental append mode
Robert Put
10/28/2022, 9:48 PMbut like i said the logs just showed the above growing the entire time
Robert Put
10/28/2022, 9:49 PM72 hours it stopped
i increased that to 7 days vs 3 days now
but looking for other ways to speed it up or if i need to in the future to re-sync
m
Marcos Marx (Airbyte)
10/28/2022, 10:09 PMI didn’t see any issue in Github about this topic for Stripe connector. @Robert Put do you mind opening a Github issue, probably we can discuss more technically there.
r
Robert Put
10/28/2022, 10:28 PMRobert Put
10/28/2022, 10:28 PMcreated!
✅ 1
2 Views