Hello everyone, i have one question:
when a sync fails and a new attempt is started, will the new attempt start syncing from where the previous one left? (in case of an incremental load for example)
the fact that no state file is present in the folders made me doubt this 😕
c
Chris (deprecated profile)
03/03/2021, 10:22 AM
ah it depends if your previous one was successful or not
if you refer to the failed attempt then that state shouldn’t be saved
I guess if it’s the first time you make a sync (even in incremental load) it still needs to perform at least a full refresh so no state file so far?
a
Alberto Costa
03/03/2021, 10:28 AM
indeed, i’m talking about a first sync. the tables it needs to sync are big and the process fails halfway through. then a new attempt starts, but it starts from the very beginning, making all the previous work useless.
Alberto Costa
03/03/2021, 10:29 AM
is this the normal, expected behavior or am i missing something?
c
Chris (deprecated profile)
03/03/2021, 10:33 AM
yes this is the simple implementation for the moment
we do have discussions about this but haven’t made improvements to this behavior yet
Chris (deprecated profile)
03/03/2021, 10:37 AM
There was also ideas about considering some checkpointing feature during long replications so that retrying won’t start from scratch