hello, is there actually any way of filtering the ...
# contributing-to-airbytec
Cristiano Sarmento
08/05/2021, 1:35 PMhello, is there actually any way of filtering the data from a source database to a destination? Let’s suppose I have a table X on my source and I want to filter something ( for example: where X.id in (1,2,3) ), is there some “Airbyte way” of doing this kind of filters or the better way to to hat may be my just building a view (filtered) in the source database and import the entire view?
Cristiano Sarmento
08/05/2021, 2:16 PMHi we offer custom dbt transformations today, so it's possible to do so. Please see https://docs.airbyte.io/operator-guides/transformation-and-normalization/transformations-with-airbyte for more info
u
user
08/05/2021, 2:16 PMIf you aren't familiar with DBT, your best bet is to have a view like you suggested!
u
user
08/05/2021, 2:20 PMand if you want to filter pre-loading the recomendation is create a view in your source database
table has date from 2010 to 2020
create view airbyte_view ... where date > 2016
this way you can remove PII and bring only data you want
u
user
08/05/2021, 2:55 PMNice, thanks for the help!
u
user
08/05/2021, 2:55 PMAbout dbt I really need to learn more about it, the “view approach” is the easier to me, by now