Lokesh Pathak
07/31/2023, 10:15 AMuser
07/31/2023, 10:16 AMwhere
clause. This is possible because the connector has a /search
or allows this possibility and the custom query will create a new stream and doesn’t change the default ones. However, in general, Airbyte is an ELT (Extract, Load, Transform) tool, meaning it extracts data from the source, loads it into the destination, and then performs transformations. The transformations are performed after loading into the destination. You can create connections and run syncs using the web UI, octavia-cli or by calling the public API. Once the sync is complete, it's possible to run transformations and even integrate them into Airbyte. If you want to filter data before it is loaded into the destination, you would need to build a custom connector for your API and use it internally. You can learn more about connector development [here](https://docs.airbyte.com/connector-development/). Please note that if you want to change the filter, you would need to retrieve all data again. Therefore, it is often best practice to first transfer and store all data and start applying your logic to the result. If you have a specific use case or connector in mind, I would recommend reaching out to the Airbyte community on [Slack](https://airbytehq.slack.com/archives/C021JANJ6TY) for more specific guidance.