Hello! I have a question regarding adding custom dbt transformations to AirByte. I have set some sources as Incremental Dedup + Basic Normalization in Airbyte. My questions is, knowing that my source uses incremental dedup, do I need to configure my custom dbt models as incremental? For example, I have orders source as Incremental Dedup. In my custom dbt project, I have it as a source. Then, orders go though each layer in dbt (base/staging/marts). In the base layer, I have set the materialization = "ephemeral", staging = "view". Do I need to set materialization in marts as "incremental"? Or do I need to set all layers to incremental materialization, since Airbyte source is incremental dedup?

loading...

your custom dbt models does not depend on the materialization of the source tables produced by airbyte. You can read more about incremental here: https://docs.getdbt.com/docs/building-a-dbt-project/building-models/configuring-incremental-models You can have both

table

or

incremental

for your marts table, You can even chain 4 models with

incremental

->

table

->

incremental

->

table

materializations one after the other too if you want to do that. it depends on what you want to do and how to run updates… that pipeline is totally different from normalization pipeline inside airbyte

@Chris Duong [Airbyte] Thanks for clarification!