Hey all! When running dbt build and then dbt sourc...
# community-support
c
Hey all! When running dbt build and then dbt source freshness, Elementary runs as a on-run-end hook for both. For our project it takes close to half an hour per run! So we would like to only run it once. Is the solution to: vars: disable_dbt_artifacts_autoupload: true disable_dbt_invocation_autoupload: true in dbt_project.yml and then to dbt run --select elementary manually after dbt build and dbt source freshness, or is there a better way?
n
Hey Christian! First, this sounds like a long running time. I'd recommend trying to disable the
dbt_columns
upload which is usually quite heavy -
Copy code
vars:
  columns_upload_strategy: none
Other than that, that's a great solution 🙂
c
Thanks! The culprit seems to be schema change tests actually, as it creates and deletes a temp table per test (and we have 1000+ of them...). So we'll probably see if disabling those will help.
🙌 1
It did not help. Neither did columns_upload_strategy: none. I'm a bit at a loss here, contemplating leaving Elementary behind for good, but we really like the result report though. Any other ideas?
To be more specific, in the Databricks logs I see almost 20 minutes between two Elementary queries: 0118 select artifacts_model, metadata_hash from
elementary
.
dbt_artifacts_hashes
order by metadata_hash (no queries in-between here) 01.37: DESCRIBE TABLE
elementary
.
dbt_run_results
n
Let us take another look. Can you please share your logs? 🙂