Hey guys, I’m relatively new to the DBT custom transformation world and I was wondering why can’t I only specify a docker image which contains my DBT project? In our DBT project we would have a Dockerfile which will contain the DBT runtime cli. Is there something I’m not understanding or Airbyte didn’t think about this use case? My DBT custom transformation project would then reside INSIDE the docker image so that I wouldn’t need to specify a git project and branch.

Are you rebuilding/publishing your docker image with a new version everytime you make changes to your dbt project repository though? in that case, you are right, we wouldn’t need to clone anything if the files are already somewhere in your custom docker image

Are you rebuilding/publishing your docker image with a new version everytime you make changes to your dbt project repository though?

that would be the idea, yes i’m not saying the git repo url is a bad idea, but an alternative

I'd like to chime in and also suggest that the git repo url could be optional. If I have a dbt project with dependencies, then the project will only work if I build the image with the dependencies pre-installed during build or via entrypoint script (I don't think I can run two dbt commands otherwise?). As I'm building the image with the dbt project's specific dependencies I'll probably have the versioned dbt project itself already in the versioned image. It still works as-is, but it would be nice not to have the system do extra work in pulling and setting up a project that is already in the image.

Thank you for the feedback @Gary K and @Philippe Boyd! We have an open issue about using custom transformation without a git repo here. I think that what @Gary K suggests is also close to this one. This also broadly relates to a post replication hook feature described here. Feel free to subscribe to these issues to receive updates and please share your idea by commenting. 🙏