Hi, trying out Airbyte for our data pipelines and ...
# ask-community-for-troubleshooting
d
Hi, trying out Airbyte for our data pipelines and I like what I am seeing so far, this is the product that I've always wanted! I do have a question -- one of the MySQL tables am trying to sync is quite big, with transactions going back 5 years. Syncing invariably fails because of timeout/ closed connection. I don't really need to go back to the beginning, so is it possible to set the cursor to some other value for the first sync? Target is GCS CSV file.
1
u
Hi @Dom Cimafranca, glad you like our product! The easiest method for your problem would be to create a view of your source table and only select a subset of the rows. If you cannot do this, I can suggest another hacky workaround that will require a bit of manipulation within the Airbyte DB.
d
Thank you @[DEPRECATED] Augustin Lafanechere! I will try the view approach first. I am curious about the workaround, though, so maybe a bit of a pointer sometime if you have the time. Might be a good intro for me into the internals of Airbyte.
u
If you are using an incremental load, Airbyte stores the state of the current cursor value in its DB; changing the cursor value to a more "recent" one could trick Airbyte into thinking it already loaded older data.
d
Thank you, I will also have a look in that direction.