Airbyte is an open-source data integration engine that helps you consolidate your data in your data warehouses, lakes and databases.

Airbyte

Hello! With removal of custom transformations from Airbyte, what is the proposed solution to run dbt after completion of the sync job?

The <https://github.com/airbytehq/airbyte/discussions/34860#discussion-6183373|proposed solution> is to use an external orchestrator to handle this operation.
&gt; • Your use case is still supported! Airbyte integrates with leading orchestration tools. Our <https://airbyte.com/blog/integrating-airbyte-with-data-orchestrators-airflow-dagster-and-prefect|tutorials and quickstarts for Airflow, Dagster, and Prefect> show you how to set this up quickly and easily. Additionally, we are surveying the community on which tools they use to trigger dbt jobs. We’d love your survey feedback in slack <https://airbytehq.slack.com/archives/C01A4CAP81L/p1707152364371989|here>! We will use this survey to identify which orchestration tools are critical to have tutorials &amp; documentation for.


I think the survey is misrepresenting. Yes most of people in Data Analytics do use various orchestrators already. 
But for many adopting just Airbyte is already a challenge. Having dbt core as a command line tool doesn’t seem big problem. But asking an additional orchestrator to be installed is too much. 
It makes selling Airbyte quite difficult. 

People very conservative and asking to adopt 3 different products is too much 

To push Airbyte as an alternative to Fivetran as a open source product you can run on prem is hard 

Unless there is a transformation element 

Thanks for the feedback Andrey :octavia-thanks: <@U02TNBMDUMQ> fyi, maybe has more information

Our data team at Airbyte triggers dbt to run periodically on a set cadence rather than after Airbyte completes. We adapt the timing of our Airbyte syncs so that when dbt picks up the latest data whenever it runs and so that the dbt run finishes when data needs to be available downstream for analytics.  Would this be a potential solution for you?

tbh previous implementation was quite good when you can just triddger dbt run as part of airbyte pipeline

basic triggering mechanism would be quite handy

I know it’s probably not Airbyte’s primary scope, but it would certainly lower the adoption of modern data stack

That's good feedback for us to hear, thank you for sharing it! While transformations and orchestration capabilities are not our primary focus at this time, this helps us form our future roadmap.

We currently are working on enabling our sync success &amp; failure notifications for <https://github.com/airbytehq/airbyte/issues/37670|ingestion by webhook services>. You could potentially use this to ultimately trigger a dbt run through the dbt API.

Great! Feel free to subscribe to the issue I linked above - we'll update it when the functionality is available for OSS.