hi everyone! I have two questions: 1. Is it possi...
# ask-community-for-troubleshooting
d
hi everyone! I have two questions: 1. Is it possible to use a local DBT project as a transformation? If so, how can I do that? Which path should I provide? 2. When we have multiple data sources, what is the best practice for transformations? Each data source should have it's own DBT project? Or a single DBT project that is triggered for every data source?
c
is this because you dont want to have a remote git repository?
d
just to make it easier to develop πŸ™‚
I know that I can run the DBT locally, but I want to run some manual e2e tests too (with Airbyte + DBT)
also, don't know if you saw @Chris (deprecated profile), but I've added a second question there too xD
c
for your second questions, both approach should be doable! BTW have you read this tutorial? https://docs.airbyte.io/operator-guides/transformation-and-normalization/transformations-with-airbyte#refresh-models-partially
πŸ‘ 1
d
about the second question, cool!! that's awesome. I didn't saw that tutorial, but I'm going to use that approach. Thanks!
c
for first question, it might be helpful to create a github issue to implement that properly otherwise i haven’t tried so i am not sure if it works, but maybe as a shortcut workaround for the moment, you could leverage the fact that there is a local data folder always mounted to docker containers as described here: https://docs.airbyte.io/operator-guides/browsing-output-logs#csv-or-json-local-destinations-check-local-data-folder So if your git repo is inside that folder
/tmp/airbyte_local/git_repo
, you could maybe specify a git url pointing there (
/local/git_repo/
) to avoid passing by a remote git repo
πŸ‘ 1
cc @abhi / @[DEPRECATED] Marcos Marx it might be a cool thing to show people on what to do with custom dbt project from a local repository
πŸ‘πŸ½ 1
πŸ‘ 2
d
cool thanks for the info πŸ™‚