https://linen.dev logo
Join Slack
Powered by
Hey. We have been running Elementary successfully ...
# community-support
a
Hey. We have been running Elementary successfully since April last year, but since 11 Jan this year there are a few models that no longer appears in the model runs part of the report, and I see they are no longer recorded in the DBT_RUN_RESULT table, but they are indeed executed with success multiple times per day. Any idea what may cause models to go missing from the report, or how I can debug? There are no changes to the missing models themselves. We're running the dbt package 0.11.2.
n
Hey @Anders Enger Can you please check if you see these models in the 
dbt_models
table? Also, please check in 
model_run_results
what is the max 
generated_at
value for each one of these models?
a
Hi. Yes they are in the dbt_models table and the max generated at is 2024-01-11 in the model_run_result. But Dbt is executing these models multiple times daily.
in this folder I have three models, but only one shows in the model runs:
n
can you share the dbt command you are running (the one that executes these models)?
a
yes, it is simply "dbt build"
it builds all models, and runs the elementary on-run-start-hook and on-run-end-hook
we have about 203 models in total, and these two are the ones I have detected to go missing, as they have usually been on top of the execution time overview
can also add that the missing models appear in the test run overview, but not in the model run
n
If there is not any “new” (depends on the days-back of your report) in the 
model_run_results
, you won’t see any run of these models. Can you please send here the log file of the dbt command that should run these models? And what are the models’ names as well 🙏🏽
a
Hi again. Had to leave yesterday, but yes, the model executes by Dbt but Elementary does not populate the dbt_run_results table anymore. In this log you can look for the model tmix_instrument_data
n
can you please check if you see new records (from the past week) in 
dbt_run_results
and in 
model_run_results
for this model?
a
Hi. The last record is from the 2024-01-11.
n
What is the latest record you have in this table (in general)? When did you upgrade the dbt package version? I see you are using dbt 1.7.4 and dbt-snowflake 1.7.1.
a
Upgraded Dbt package and Cli on Nov 22
n
I am asking about dbt version and not Elementary’s dbt package
a
I understand. They were all upgraded on the same day
The curious thing is that it is only some models that stopped getting run results, but I can't see that there is anything special about them, nor were there any changes to them in this timeframe
But, I see in Dbt Cloud we do not specify minor version for the environment, just 1.7(latest) so there could bumps up on minor versions without us actively making any changes.
n
I want to make sure, this is not a snapshot or a seed, right? Can you please share the configuration of this model? Can you please run also 
dbt build -s tmix_instrument_data
, and see if you see a new record in the DWH?
✅ 1
a
This is a regular model materialized as table yes. I'll provide the log from the individual build shortly. this is the config block: {{ config(materialized='table' ,persist_docs={"relation": true, "columns": true} ,tags=['financing'] ) }}
I can confirm when I build this model separately with the --select model syntax as you suggested, it generated a new entry in dbt_run_results . But not when the whole project is built. This applies to some other models also. Are we reaching a cap here with 230+ models?
So here are the full debug logs for both the single model build which generated a record, and the full project build which omits some models, including tmix_instrument_data
d
@Anders Enger, i went over the log files. Can you share the contents of 
ELEMNTARY_TEST_RESULTS
after running the same operations like when you've generated the logs ?
a
This is about the model runs though, test results looks to work fine
But here as can be see in the test result, the model produces results multiple times per day, as per schedule. But in the dbt_run_results it stopped on the 11th of Jan. The entries you see for 25,29 are manual runs of the single model and the entry today 30.01 is running with elementary package downgraded to 0.10.0
dbt_run_results.csvelementary_test_results.csv
👀 1
d
mm the log you attached before contains only the things from the 25, where everything looks fine
i'd like to see a log of running multiple models, that eventually results with missing model runs
a
This the one I can get from Dbt cloud console.
d
ok, this run should contains also 
tmix_instrument_data
?
a
yes. 2024-01-30 100700.807693 (Thread-8): 100700 Began running node model.financing_dbt_snowflake.tmix_instrument_data 2024-01-30 100700.809607 (Thread-8): 100700 548 of 560 START sql table model financing.tmix_instrument_data ................ [RUN]
d
@Anders Enger, question - did you try version 0.12.2 of elementary's package ?
I think that till version 0.12.0 we had a bug in this area (that caused to missing models) This fixed it, and was released in version 0.12.1
a
Hi. We were running 0.11.2 as that was the latest version with documentation on the site. But can test with 12.2. It looks like models with long queries were capped off?
d
The bug wasn't about single model long query but many models that accumulated into long query
✅ 1
a
Good to know. Thanks for your effort looking into it.
d
No problem, let me know if it didn't solve the issue
31 Views
Open in Slack
PreviousNext
Powered by