Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hi there - what a great project you have here :ok_hand: Got a quick question: are there any plans for being able to filter data in the source already _before_ it leaves the source environment? Or like an offset perhaps? For example if I have some kind of SQL source and I don’t want to sync *all* of the data when first setting up my connection? I only want to sync data from now and onwards?

Cheers!

Not possible now, there is an issue requesting for JDBC connectors to add this feature: <https://github.com/airbytehq/airbyte/issues/5541>

For API’s most cases they allow you to choose the start date

Great - thanks for answering! I don’t know enough about Airbyte but would this also apply to e.g. a BigQuery source then?

And how much work do you think it would be (roughly ofc :slightly_smiling_face: ). If you could point me in the right direction I could perhaps take a look, although my java skills are a bit lacking