I kept debugging and understood a couple of things : • We are using a on-run-end hook to manage our tags, this on-run-end hook runs before the one defined in elementary. • I try running

dbt build

to rebuild all models and run all tests. It looks like when the first on-run-end hooks fails, the second (the one defined by elementary) isn't triggered. I tried running

dbt build

while selecting a subset of models, I got no errors in both on-run-end hooks and elementary models are successfully updated with the related data. I don't know if the cause of the issue in the first

dbt build

is the failure of the first on-run-end hooks or just the number of models (around 250). • It isn't clear to me if I should exclude elementary models from selection when running

dbt build

/

dbt run

in my data pipeline and let only the on-run-hook handle the inserts, or are elementary models supposed to be run every time we run

dbt build

in our data pipelines. Here (link to elementary doc) it says that we should run

dbt run --select elementary

every time models are added/updated so we plan to add this step in our CI but I don't know if they are also supposed to run in our data pipelines. • Even though the second

dbt build

(the one selecting a subset of models) worked and successfully inserted records in elementary tables, when I try to regenerate the report, it's still empty. I get a warning about version incompatibility between elementary dbt package and elementary CLI (respectively 0.15.2 and 0.16), I don't know if that's the cause, will try to fix version and retry.

Hello everyone, After upgrading to latest versions of elementary dbt package & elementary CLI, I run a

dbt run --select elementary --full-refresh

and finally a

dbt build --select a_model_with_tests

. I can see the elementary tables are updated and contains data related to the dbt run and tests. The report generation also works now and contains the previously mentioned dbt run and tests. The only blurry point left is when running a

dbt build

command without selecting models ; that still doesn't update elementary tables and show up in reports. We need to understand if it's caused by • the fact that a failure in our on-run-end hook causes the failure of elementary on-run-end hook. I think theres an open issue in the dbt-core repo related to that (see https://github.com/dbt-labs/dbt-core/issues/7387) • the volume of data to be inserted in elementary tables at once Anyway thank you for the help @Larissa Mendes Hermogenes Rocha ! 🙏

so, I don't think the volume of data should be the issue in this case, if I were you I would check the on-run-end hook failure, since it's how Elementary log resuts to the tables. https://docs.elementary-data.com/dbt/on-run-end_hooks