Hey, has anyone setup elementary OSS with dagster and dbt core? 😊 How often do you generate the observability report? At the end of every dagster run? 🤔

Hi Lukas! Without knowing too much about your set up I would say have it run as your normal dbt pipeline. So if you run dbt daily, I would suggest generating the report daily too.

Hi Joost, thanks for the reply! We have some models refreshing intraday and for downstream models we make use of dagster’s Auto Materialize feature. So we have quite a few intra-day runs 🤔 I guess I will give it a try after every run and see how that behaves 🤞

Interesting! Yes so you could also think of a strategy to have a separate job which just runs

edr report

end-of-day or start-of-day based on your needs etc. It is computationally not heavy to generate the report, so for costs I don't think there will be any difference. However, it might be cleaner to not generate it multiple times per day. I would suggest coming up with the best deployment strategy based on your set up 🙂