Hi, using latest Elementary and dbt, taking a minute to upload the run results of a single model:

04:02:20  Running with dbt=1.7.4
04:02:21  Registered adapter: snowflake=1.7.1
04:02:24  Found 768 models, 5 analyses, 25 seeds, 2 operations, 630 tests, 235 sources, 10 exposures, 0 metrics, 1478 macros, 4 groups, 0 semantic models
04:02:24  
04:02:27  
04:02:27  Running 1 on-run-start hook
04:02:27  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
04:02:27  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
04:02:27  
04:02:27  Concurrency: 16 threads (target='prod')
04:02:27  
04:02:27  1 of 1 START sql table model crm.contact ....................................... [RUN]
04:02:33  1 of 1 OK created sql table model crm.contact .................................. [SUCCESS 1 in 6.21s]
04:02:33  
04:02:33  Running 1 on-run-end hook
04:03:31  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
04:03:31  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
04:03:31  
04:03:31  
04:03:31  Finished running 1 table model, 2 hooks in 0 hours 1 minutes and 7.45 seconds (67.45s).
04:03:32  
04:03:32  Completed successfully

Are there any workarounds to improve on-run-end performance?

Hi @Alex Mirkin At-Bay, Even when running dbt on a single model, we still upload artifacts for all models. This is cached, but checking each model against the cache to see if it changed is not very performant. If you are running models by groups, you can disable your on-run-end hooks and run them manually once a day or in any cadence you like. You can find instructions for this here

We are suffering the same issue here, the models & tests run pretty quickly but the on run end takes forever compared to them.

The workaround @Ella Katz mentioned worked for us perfectly. With the

disable_dbt_artifacts_autoupload: true

flag the

on-run-end

hook finishes much faster - few seconds. I replaced the

true

with jinja code that returns

false

if I'm deploying changes (CD) and

true

in our CI or scheduled DWH runs. This way the cache will be used for the unchanged resources and it will be much faster then running than running _`dbt run --select elementary.edr.dbt_artifacts`_ which takes 5 minutes to run.

Hi @Alex Mirkin At-Bay and @Ella Katz, this is the thread I was looking for! ✨ One thing that gets me confused is: does

disable_dbt_artifacts_autoupload: true

only disable metadata models from being updated? And run_results / test results would still be refreshed?

I've set

disable_dbt_artifacts_autoupload: true

and still getting the run and test results.