Hi all, We have an issue with Elementary not uploa...
# community-support
k
Hi all, We have an issue with Elementary not uploading all run results. We tried 0.11.2 and 0.11.1, same observation for both, we are running our incremental run:
Copy code
Finished running 276 view models, 78 table models, 29 incremental models
However for that invocation in
dbt_run_results
we can see only 177 views, 12 tables. All models are present in
dbt_models
- we just don't see complete runs in
dbt_run_results
. We are using dbt on bigquery (1.5.6). Any ideas what we are doing wrong?
👀 1
e
Hi @Krzysztof Ławecki, we're looking into that now
👍 1
k
@Erik Zaadi let me know if you would need anything from our side to help pinpoint the issue
🙏 1
m
Hey @Krzysztof Ławecki, it was a bug on our end. Just released
0.12.1
that should fix it. Please change the version on
packages.yml
, and run
dbt deps
&&
dbt run --select elementary
k
Thank you - will try that tomorrow and let you know
🙏 1
k
@Erik Zaadi @Maayan Salom the fix worked - thank you! One observation though - now the on-run-end hook takes over 7minutes (which makes our 10 min-long incremental run almost 25 mins long as we are also running tests). Any performance optimisation tips for BigQuery setup? Is there a way to run the on-run-end only after dbt test (but also include results of dbt run then?)? Ideally we would like to fit Elementary max within 5 mins, as we plan to have more models and test and it looks very long at the moment
🙏 1
m
@Krzysztof Ławecki is this consistent between runs? And are you using dbt 1.4.0 or above?
k
Yes, consistent - we are using dbt 1.5.6
m
Could you please share a
dbt.log
of a full run on DM? So we could see which part is performance heavy
k
Ok - I will run it on our dev env as we are not capturing
dbt.log
on our CloudComposer runs - I will will send it over today 🙂
🙏 1
FYI - I sent
dbt.log
to @Maayan Salom We upgraded dbt to
1.6.9
and issue still persists. Even simple
dbt seed
takes 2 minutes, while we only have only few seeds - elementary data takes 1.5 minutes here (and on dbt run and dbt test it takes much more) Also it looks like it utilises only only Main Thread - maybe thats part of the issue? We have ~8 threads on our prod job