https://linen.dev logo
e

Elias Djurfeldt

01/18/2022, 5:32 PM
Hi there - what a great project you have here 👌 Got a quick question: are there any plans for being able to filter data in the source already before it leaves the source environment? Or like an offset perhaps? For example if I have some kind of SQL source and I don’t want to sync all of the data when first setting up my connection? I only want to sync data from now and onwards? Cheers!
Not possible now, there is an issue requesting for JDBC connectors to add this feature: https://github.com/airbytehq/airbyte/issues/5541
For API’s most cases they allow you to choose the start date
r

Ronny Ritongadi

01/19/2022, 7:44 AM
Great - thanks for answering! I don’t know enough about Airbyte but would this also apply to e.g. a BigQuery source then? And how much work do you think it would be (roughly ofc 🙂 ). If you could point me in the right direction I could perhaps take a look, although my java skills are a bit lacking
3 Views