https://linen.dev logo
#ask-community-for-troubleshooting
Title
# ask-community-for-troubleshooting
d

Donovan Maree

01/04/2022, 2:31 PM
Hi all and happy new year. I'm looking for a way to split the _airbyte tables into their own dataset separately from the basic normalization tables as they enter bigquery, is there a way to do this currently? The use case would be to make navigation simpler
👀 1
u

[DEPRECATED] Augustin Lafanechere

01/04/2022, 3:49 PM
Hi @Donovan Maree, do you mean keeping raw tables in a different dataset than the table outputted by the basic normalization?
d

Donovan Maree

01/04/2022, 3:51 PM
yes thats correct
u

[DEPRECATED] Augustin Lafanechere

01/04/2022, 4:10 PM
It's something you can achieve by running a custom transformation with DBT. You can retrieve the generated models for normalization and adapt them to change the destination dataset. We have a guide here about custom transformations.
c

Chris (deprecated profile)

01/04/2022, 4:11 PM
you can configure airbyte to replicate into some “staging” dataset as a destination Then, you should disable basic normalization option. In one of your previous sync, you can download the generated normalization/dbt project and add to github repository while changing the target destination of normalized tables as Augustin linked so you can have your custom results
👍 1
In each of the generated .sql files, you can change the value of
schema = "final_dataset",
in the config header to point to a different dataset https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-schemas
2 Views