Title
#advice-data-transformation
Christopher Wu

Christopher Wu

04/21/2022, 7:01 PM
Hello @Augustin Lafanechere (Airbyte) @Chris Duong [Airbyte] , we want to follow up on a previous thread regarding support for DBT transformations on custom destinations: https://airbytehq.slack.com/archives/C01VDDEGL7M/p1645641827127489 Currently, we cannot run our DBT transformations on Airbyte because we use a custom destination, for which there is no normalization mapping that enables Airbyte to read the destination config to create the DBT project.yml file: https://github.com/airbytehq/airbyte/issues/7229 Will this issue be addressed in your short-term roadmap, allowing destinations to specify their own normalization mapping for DBT transformations via their spec.json files or some other method, or will we need to add a mapping for our destination to the Airbyte source code? If the latter, or if both options require the use of a normalization docker image, how can we utilize an existing image like
airbyte/normalization
? Do we need to copy exactly the same destination config parameters that the Postgres destination uses? We might be willing to contribute an implementation for https://github.com/airbytehq/airbyte/issues/7229, provided some necessary supporting logic is already in there. E.g., does the normalization runner already have access to destination specs?
abhi

abhi

04/23/2022, 6:09 PM
@Marcos Marx (Airbyte) small ping here
Thomas Gerber

Thomas Gerber

04/26/2022, 5:57 PM
friendly ping @Marcos Marx (Airbyte) 🙂
Marcos Marx (Airbyte)

Marcos Marx (Airbyte)

04/26/2022, 5:58 PM
sorry was OOO last week, later today ill take a look
Thomas Gerber

Thomas Gerber

04/26/2022, 5:59 PM
no worries, no rush, this is a non-trivial topic; but we are willing to contribute 🙂
7:00 PM
@Marcos Marx (Airbyte) friendly ping 🙂