Hello Airbyte team, I was wondering about Airbyte and DBT Transformation. Airbyte current does not have Quickstart Data Models like Fivetran, so I ended up writing all of my custom transformations for sources like Stripe and Google Ads. would there be any interest in having community sourced Quickstart data models? It would be nice to reuse all of the DBT code I wrote. The only problem I considered for Airbyte keeping transformations downstream compatible with changes in the source connectors, like schema changes.

Definitely YES! Actually I’m working right now doing a PoC to have these transformation/source/analytics models. I’m doing for Zendesk Support connector. You can join the #C06RLAXDQAX channel and let’s discuss more about the topic!

I second this! I think it is a great idea!!

Agreed. I've put tremendous effort into several models as well, and have wondered how many have done the same. As for keeping transformations compatible with upstream changes, I've done this by reading the available fields in my staging models and bubbling them up to the final marts. I've also added some macros to handle things like columns that are always null (giving an option to eliminate them in the staging models to reduce noise). This has been especially helpful in sources like Salesforce, which can go to crazy-town with the number of both objects and fields really quickly. (There are costs to things like this in terms of table IO, but having them configurable I think is a nice step). So now when Airbyte adds a non-breaking change, the model just picks it up from the source through a mix of reading the source fields from the adapter and excluding fields that don't need to be present in the final model (e.g.

SELECT * EXCEPT(...)

. For Salesforce specifically I've also done a lot to surface variable relationships as nested objects (since we're a SaaS/Agency use case and not every Salesforce account is configured the same, and it changes where some data is sourced). Maybe not a replacement for per-stream-models as a default, but maybe a good alternate mart for common things like Accounts and Contacts (+ Individuals depending on settings)