Hello! We recently upgraded our elementary to 0.12, and our on-run-end hooks are starting to take a very long time before they respond. The total models or tests will run in seconds, however when it comes to the

on-run-end.0

hook it takes upwards of 20 minutes despite showing only a short duration. Prior to upgrade we were on 0.7.5 and the end run hooks would take less than a minute. Any recommendations?

Hey @Karl Duane! Care to share with me your dbt.log? It will help me understand what might be the issue 🙏

Digging in, we see the problem go away when we set

disable_dbt_artifacts_auto_upload: true

Seconding - the post hooks never complete on our dbt prod runs

Hey @Karl Duane, can I ask you to try and run elementary 2 times? we changed the mechanism of uploading artifacts between the versions and we have a caching mechanism that suppose to reduce the timing after the first run. Care to try?

@Idan Shavit I’m game to give it a try. We’re currently exploring disabling the auto-upload and rerunning the full artifacts upload with a workflow every time we merge a pr, would this caching method allow us to only work with the artifacts for the table changed in the PR instead of the entire 2k+model project?

Seconding this as I'm having the same issues

The ‘caching mechanism’ Idan referenced seems not to be picked up either in dev or production. If we disable the dbt artifacts upload or remove the dbt_columns macro call in the on-end-run hooks, we speed up, however we then have to find another method/cadence for uploading the required artifacts

same issue here 😕 on-run-end hook takes now 7 minutes to respond

@Karl Duane @Jenna Jordan @Robert Riley @Julia Masche If you are using dbt in a version lower than 1.3, cache won't work, so the solution of disabling artifacts and running on merge should be the solution. Otherwise - we are releasing a new version with better logs on the hooks. If you will be willing to share logs after upgrading we could address this.

Hi, using latest Elementary and dbt, taking a minute to upload the run results of a single model:

04:02:20  Running with dbt=1.7.4
04:02:21  Registered adapter: snowflake=1.7.1
04:02:24  Found 768 models, 5 analyses, 25 seeds, 2 operations, 630 tests, 235 sources, 10 exposures, 0 metrics, 1478 macros, 4 groups, 0 semantic models
04:02:24  
04:02:27  
04:02:27  Running 1 on-run-start hook
04:02:27  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
04:02:27  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
04:02:27  
04:02:27  Concurrency: 16 threads (target='prod')
04:02:27  
04:02:27  1 of 1 START sql table model crm.contact ....................................... [RUN]
04:02:33  1 of 1 OK created sql table model crm.contact .................................. [SUCCESS 1 in 6.21s]
04:02:33  
04:02:33  Running 1 on-run-end hook
04:03:31  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
04:03:31  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
04:03:31  
04:03:31  
04:03:31  Finished running 1 table model, 2 hooks in 0 hours 1 minutes and 7.45 seconds (67.45s).
04:03:32  
04:03:32  Completed successfully

Are there any workarounds to improve on-run-end performance?

Hi @Alex Mirkin At-Bay, the amount of models should not impact the

on-run-end

runtime, since elementary uploads all of your dbt artifacts (models, sources, snapshots, etc), regardless of your selector. you can try disabling this behavior using

disable_dbt_artifacts_auto_upload: true

variable, and only enabling it for bigger jobs

Hi @Ofek Weiss, in this example only the run results were uploaded.

hmm... 🤔 can you share your

dbt.log

with me? DM is good if you prefer