What are your thoughts on dbt <newly announced roa...
# advice-data-transformation
s
What are your thoughts on dbt newly announced roadmap where they try to: 1. Rewrite the way we test dbt-core and adapters and sustainable maintainer-ship 2. Adapter ergonomics, modular programmatic interfaces, and python language dbt models! 3. Later they want to unify lineage (metrics and semantic layer).
šŸ‘€ 2
šŸ‘‹ 1
There are some significant additions to achieve in a short time. I'm particularly curious about the python integration as we have matured data orchestration tools (see #advice-data-orchestration). It will not be an easy task on top of tackling the metrics and the semantic layer. It's going to be interesting :)
a
I like it! I think you can already run Python models on dbt with https://github.com/fal-ai/fal dbt supporting other languages other than SQL, makes me think that they want to get closer in terms of functionality to orchestrators like Airflow, Prefect and Dagster.
s
Thanks for sharing, Ari! Hmm that is very interesting! I wonder to what extent you should use that if you use it vastly and have lots of repeated resources and secrets for the connection if that scales with it. Or if it is instead thought of as one time, you really can't avoid python šŸ¤” . Yes, that's for sure (with getting closer to other orchestrators). And that is where I see the tricky part, as it wasn't planned from the beginning, which will be very hard to patch on top. But even better if they succeed. But I'm pretty sure for a long time, and you are much better at using perfect/dagster and integrating with dbt. But that's why I'm curious to see how that evolves.
šŸ‘ 1