Hi all, I have a pretty newbie question, once the elementary package is installed (part of our github workflow), I can see the hooks running but the table

dbt_run_results

only contains the test results which we are running using

dbt run --target elementary

. We can't use that target in the generation of our models in production though, so it seems that I am missing something here because I thought that those hooks are used to collect the run results by default and append them to

dbt_run_results

? Thank you so much!

Hi @Yassmine Chebaro , The artifacts and test results are added to the tables in the Elementary schema during the on-run-end hooks (more about that here). So they should be uploaded whenever you run your production dbt models. If they are not, please try to check if the on-run-end hooks are running and succeeding.

Hi @Ella Katz, Yassmine and I work together. From what I can understand, our on-run-end hooks are running without failure, but they are not updating the tables. Could I share our most recent log files with you?

Hi @Will Seaton, Can you please check if your elementary hooks are disabled? That can be the case if any of the following vars are defined in your command or dbt project yml:

disable_run_results
disable_tests_results
disable_dbt_invocation_autoupload
disable_dbt_artifacts_autoupload

I don't see any of the variables defined in the commands or the project yml. We see Elementary hooks running after our models in production now, like below:

Yes they should, but the 0.0 seconds runtime makes it look like they are not doing anything 🤔 Do you have rows in elementary_test_results without having to run the additional command? Or are those empty too?

We also thought the 0.0 seconds was strange, but we get the same runtime for our tests and those load correctly. So, yes, we do have results in elementary_test_results, but we do run the Elementary GitHub Action and generate our report directly after the tests if that counts for anything.

OK just tested it and I get the 0.0 seconds too. In your dbt.log of your production models run, can you find the following log:

Elementary: Uploading run results.

? If you do, the rows in the log after that should be the queries uploading the run results. let me know if this helps

I searched with Cmd + F, but didn't find the log unfortunately.

did you find any logs from other on-run-end hooks? artifact upload? you can send me the log and I'll try to take a look as well. My suspicion is that the different targets are somehow causing the run results to be written to another location, but I can't be sure. Can you tell me a little about your setup? How are you uploading test results vs. run results?

Our set up is a large number of automated jobs via GitHub Actions. These jobs install Elementary as part of dbt deps and so the hooks are triggered on all of them. However, the only workflow that seems work as intended is our workflow to create and host the Elementary report. This is a dedicated workflow that runs the Elementary models, runs all of our repos tests, then runs Elementary and generates the report and logs which are sent to a different repo to host the report via GitHub pages. The test results from the dedicated workflow are successfully written to the tables and populate the report as expected, but we do not have any data on model runtime or models success/failure being saved. I managed to save model run metadata when running locally, but this isn't a sustainable option for us.

yeah can you send me the github workflow you're using to run your models?

Oh no, I think that's definitely it. We forgot to add

dbt run -s elementary

to the other jobs...🤦‍♂️

Got it. So it's working now?

I was out of office last week so I'm only testing it out now, but I'll let you know the outcome.