Josh
06/04/2024, 12:07 PM430 of 878 OK created sql incremental model curve_ci_98830f92_elementary.dbt_columns [CREATE TABLE (0.0 rows, 0 processed) in 2667.63s]
This model is now taking 45 minutes to run which seems slightly crazy..?
Just curious what could be causing this now thanks?Josh
06/04/2024, 12:17 PMJosh
06/04/2024, 12:22 PMDenis Kokorin
06/04/2024, 1:04 PMdbt_columns model was in Elementary 0.14, in 0.15 it was optimized to upload only documented columns (at least it's what I got analyzing source diff between versions). What is more interesting Elementary finds out if there are no changes in a project and then dbt_columns takes a couple of seconds.
How big is your project? We have 200+ models and it takes 2-3 minutes to run all Elementary specific models with dbt run -s package:elementary .
But what is more important is how much CPU and RAM you have on CI server. Elementary models are filled with post-hooks, which means all the calculations take place on client's side. So if you have limited CPU or/and MEM, it could take longer.Brian Waligorski
06/13/2024, 2:21 PMdisable_dbt_columns_autoupload: true, but we’re still getting ``Resources exceeded during query execution: Not enough resources for query planning - too many subqueries or query is too complex.` for our larger jobs (~500 models)Anil
06/28/2024, 10:53 AMJosh
07/03/2024, 4:22 PM