Noy Arie
06/01/2023, 12:00 PM0.8.0, dbt package 0.8.0)!
As this is a minor version bump, you need to run dbt run -s elementary
elementary What's New? elementary
⢠š Jobs info from Orchestrator dbtAGDP
⦠Elementary now supports collecting metadata about your jobs from your orchestration tool!
⦠The goal is to provide context that is useful to triage and resolve data issues:
⦠As a first step, you could filter the lineage by job in the Elementary report. filter
āŖļø More orchestrator related features are coming soon š
⦠Here is the guide for enabling jobs info collection..
⢠Anomaly detection configuration on all levels (test/table/project) š¤Æš
⦠You can now configure all test params in the project / model / test level.
⦠Why is it useful?
āŖļø It enables you to tailor the tests to the dataset and get higher level of accuracy!
āŖļø You can leverage inheritance, configure at a higher level (like folder of models) and save the need to configure by test.
⦠Some examples:
āŖļø You can set days_back: 90 to tests with time_bucket: period: week, and days_back: 7 to tests with time_bucket: period: hour .
āŖļø You can set timestamp_column: updated_at in your dbt_project.yml if this is your convention, and override it for models where it's different.
āŖļø You can set seasonality, time_bucket and timestamp_column at the source level, and it will apply for all the tests you add to tables of this source.
⦠We also upgraded our documentation of the tests configuration and how the tests work, to make it clearer š
elementary Other Changes elementary
⢠Added materialization field to models run results, thank you @Aril Mavinkere for contributing! š¤©
⢠Removed env from report summary.
As always - We are looking forward to talk to you here on Slack (#C02CTC89LAX, #C03LPJ5EYU9), and happy to schedule calls!Jovan Sakovic
06/01/2023, 12:06 PM: after https )Maayan Salom
06/01/2023, 12:09 PM