Colin
08/07/2025, 8:18 PMkapa.ai
08/07/2025, 8:18 PMkapa.ai
08/07/2025, 8:19 PMvolume_anomalies all the way to your earliest data, you can indeed use the training_period parameter. By setting a large training_period (e.g., covering the full historical range you want to backfill), Elementary will collect metrics for that entire period, as long as you have a timestamp_column configured. This is the recommended approach for timestamp-based tests, as the training_period defines how far back Elementary looks to build its training set for anomaly detection. For example: yaml models: - name: your_model tests: - elementary.volume_anomalies: timestamp_column: your_timestamp training_period: period: day count: <number_of_days_to_cover_all_history> [training_period documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/training-period) Process: 1. Set training_period to cover your full historical range. 2. Run the test to collect metrics for that period. 3. After the initial backfill, you can reduce training_period to a more typical value for ongoing monitoring. Important: - For regular tables/views, the full training_period is recalculated on each run. - For incremental models/sources, the full training_period is calculated on the first run and on full refresh; subsequent runs only calculate the detection_period unless you do a full refresh. - If you change training_period after running tests, you need to do a full refresh of the metrics to update the collected data ([Anomaly troubleshooting guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)). So, yes, you can backfill by running the test once with a large training_period, then set it back to a normal range for future runs. If you change the period later, remember to do a full refresh to update the metrics accordingly.Colin
08/07/2025, 8:23 PMkapa.ai
08/07/2025, 8:23 PMkapa.ai
08/07/2025, 8:23 PMdata_monitoring_metrics), Elementary will recalculate metrics for the entire training_period you have configured. Any data in the recorded metrics that is prior to the new training_period will be dropped—only data within the current training_period (plus detection period, if relevant) will be retained after the refresh. This behavior is explicitly described in the documentation: > If you change var training_period in your dbt_project.yml, you will need to run a full refresh to the metrics collected. This will make the next tests collect data for the new training_period timeframe. > 1. Change var training_period in your dbt_project.yml. > 2. Full refresh of the model ‘data_monitoring_metrics’ by running dbt run --select data_monitoring_metrics --full-refresh. > 3. Running the elementary tests again. > > [Source: Troubleshooting](https://docs.elementary-data.com/oss/general/troubleshooting) Additionally, for incremental models and sources: > The values of the full training_period period is calculated on the first test run, and on full refresh. The following test runs will only calculate the values of the detection_period period. > > [Source: training_period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/training-period) So, after a full refresh, only the data within the current training_period (and detection period) will be present in the metrics, and older data will be removed.Colin
08/07/2025, 8:24 PMkapa.ai
08/07/2025, 8:24 PMkapa.ai
08/07/2025, 8:25 PMvars section of your dbt_project.yml. However, it is important to note that while some parameters can be set globally, there are known limitations and bugs regarding which parameters are actually respected at the global level. Specifically, training_period is documented as a variable that can be set globally, but in practice, it may only be supported at the test level due to current implementation limitations and bugs. Here is an example of how you would configure training_period in your dbt_project.yml under the vars section: yaml vars: training_period: period: day count: 90 However, according to recent support discussions, training_period and detection_period are only reliably supported on a test level at this time, and setting them globally in dbt_project.yml may not have the intended effect. This is acknowledged as a bug and is being worked on by the Elementary team. For now, you should set training_period directly in your test configuration for consistent results: yaml models: - name: your_model tests: - elementary.volume_anomalies: training_period: period: day count: 90 [See Slack discussion for details](https://slack.com/archives/C02CTC89LAX/p1706795086121929) Summary: You can try to set training_period globally in dbt_project.yml as shown above, but due to current limitations, it is recommended to set it at the test level for now. If you want to track progress on this feature, you may follow updates from the Elementary team in their community channels.Colin
08/07/2025, 8:34 PMforce_metrics_backfill and old configuration variable?kapa.ai
08/07/2025, 8:34 PMkapa.ai
08/07/2025, 8:34 PMforce_metrics_backfill is not an old configuration variable. It is a current variable that you can use when running dbt tests with Elementary to force a full refresh of cached metrics. This is especially useful for incremental models or sources, where metrics are cached and not always recalculated unless explicitly requested. By running your test with --vars '{"force_metrics_backfill": true}', you ensure that all metrics are recalculated, bypassing any caching mechanisms. This variable is not deprecated and is still recommended for situations where you need to refresh all historical metrics or after significant data changes [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1705323587448999) [another example](https://slack.com/archives/C02CTC89LAX/p1698154501770259).Colin
08/07/2025, 8:35 PMforce_metrics_backfill apply to the training period, or will it ignore the training period and go all the way back to the earliest data?kapa.ai
08/07/2025, 8:35 PMkapa.ai
08/07/2025, 8:35 PMforce_metrics_backfill variable forces Elementary to recalculate and refresh all cached metrics for the relevant period. However, it does not override the configured training_period. Instead, it ensures that metrics are recalculated for the period defined by your test configuration (such as training_period and detection_period). It does not automatically go all the way back to the earliest data unless your training_period is set to cover that entire range. In summary: - force_metrics_backfill applies to the period defined by your configuration (e.g., training_period), not to the entire history of your data unless you explicitly set the training_period to cover all history [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1708533386888479) [troubleshooting guide](https://docs.elementary-data.com/oss/general/troubleshooting). If you want to backfill metrics for all historical data, you must set your training_period accordingly and then use force_metrics_backfill to ensure all those metrics are recalculated.Colin
08/07/2025, 8:37 PMconfig:
error_if: "= -1"
warn_if: "= -1"kapa.ai
08/07/2025, 8:37 PMkapa.ai
08/07/2025, 8:37 PMseverity: warn or custom `error_if`/`warn_if` conditions) individually for each test if you want to override the default behavior [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1690908170163389) [another example](https://slack.com/archives/C02CTC89LAX/p1706126443387969). There have been suggestions and feature requests to allow for global configuration of severity or silencing for Elementary tests, but as of now, this functionality does not exist. If this is important for your workflow, you may consider opening a feature request with the Elementary team. Summary: You currently need to duplicate the config for each Elementary test to silence warnings and errors, as there is no global configuration available for this purpose.Colin
08/07/2025, 9:07 PMmodels:
+copy_grants: true
dbt_artifacts:
+schema: dbt_artifacts
elementary:
+schema: dbt_elementary
+tags: ["elementary"]
volume_anomalies:
error_if: "= -1" # Never error
warn_if: "= -1" # Never warnkapa.ai
08/07/2025, 9:07 PMkapa.ai
08/07/2025, 9:07 PMdbt_project.yml in the way you described. While you can set some configurations for models and tests at the project level, the specific approach of nesting test-specific configs (like error_if and warn_if for volume_anomalies) under the elementary section in dbt_project.yml is not supported by dbt or Elementary. Elementary tests do not support global silencing (e.g., setting error_if: "= -1" and warn_if: "= -1") for all tests via a single configuration block in dbt_project.yml. You must set these configurations individually for each test. This limitation has been confirmed in multiple community discussions, where users have asked for a way to apply such settings globally and have been told it is not currently possible [see Slack discussion](https://slack.com/archives/C02CTC89LAX/p1690908170163389) [another example](https://slack.com/archives/C02CTC89LAX/p1706126443387969). If you want to silence warnings and errors for Elementary tests, you will need to duplicate the config for each test. Summary: No, you cannot configure Elementary test silencing globally in dbt_project.yml as shown in your example. Each test must be configured individually.