The

data_monitoring_metrics

table is a core component of Elementary's anomaly detection system. It's an incremental table that stores metrics over time, such as volume, freshness, and data quality metrics collected from your monitored tables. When anomaly detection tests run, they query this historical data to compare against the latest values, allowing Elementary to identify unusual patterns or deviations [Elementary dbt package models](https://docs.elementary-data.com/dbt/package-models#anomaly-detection). As for why Elementary processes so much data during anomaly detection: 1. Training data requirements: By default, Elementary needs at least 14 days of historical data to establish a baseline for what's "normal" before it can accurately detect anomalies. This is why it queries a significant amount of historical data from

data_monitoring_metrics

even when the monitored table itself might be small [Anomaly Troubleshooting Guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide). 2. Time-based analysis: For timestamp-based tests (recommended), Elementary groups data into time buckets (default: daily) and needs sufficient historical buckets to establish patterns. Without enough historical data points, you'll see the "Not enough data to calculate anomaly score" message [Anomaly Troubleshooting Guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide). 3. Seasonality considerations: When using seasonality features (like day_of_week), Elementary needs even more historical data to establish seasonal patterns, which is why you might see it processing months of data [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1689000364837139). There have been performance improvements to reduce the data scan size. For example, one GitHub issue mentioned that using the timestamp_column instead of scanning the entire table can significantly reduce query time from 20 minutes to 1 minute [GitHub Issue](https://github.com/elementary-data/elementary/issues/1286). The data in

data_monitoring_metrics

accumulates over time and is crucial for the anomaly detection functionality. If you're not using Elementary's anomaly detection tests, this table won't contain any data, as confirmed by Elementary's support team in a Slack conversation [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1688373015213789).

Deleting the

data_monitoring_metrics

table would significantly impact Elementary's anomaly detection capabilities. Here's why this approach isn't recommended: 1. Loss of training data: The

data_monitoring_metrics

table stores historical metrics that are essential for anomaly detection. Elementary uses this historical data to establish what "normal" patterns look like for your data [Elementary dbt package models](https://docs.elementary-data.com/dbt/package-models#anomaly-detection). 2. Test functionality: While Elementary would recreate the table structure if deleted, your anomaly detection tests would effectively restart their training period. This means they wouldn't be able to detect anomalies until enough new data is collected again (typically 14 days by default). 3. False positives: Without sufficient historical data, you're likely to experience more false positives as the system won't have enough context to determine what constitutes an anomaly. If you're concerned about performance, there are better alternatives than deleting the table: - Targeted cleanup: If you need to reset metrics for a specific table due to data behavior changes, you can use targeted deletion instead of dropping the entire table:

DELETE FROM data_monitoring_metrics WHERE [condition]

as suggested by Elementary's support team [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1681289966274379). - Configure time periods: You can adjust the

training_period

and other configuration parameters to control how much historical data is used. - Use timestamp columns: Configuring timestamp columns can significantly improve performance by allowing Elementary to more efficiently query only relevant data. The

data_monitoring_metrics

table is designed to be incremental, meaning it's optimized to handle growing data volumes over time. Deleting it daily would defeat the purpose of anomaly detection, which relies on establishing patterns over time. If you have specific performance concerns, I'd recommend reaching out to Elementary's support team for optimization strategies rather than implementing a daily deletion schedule. Would you like more information about optimizing Elementary's performance without losing the benefits of its anomaly detection capabilities? [Elementary dbt package models](https://docs.elementary-data.com/dbt/package-models#anomaly-detection)

Based on the knowledge sources, there are several reasons why Elementary tests might be skipped when running

dbt build

, even when the model builds successfully: 1. Incorrect test configuration placement: One common issue is placing the Elementary test inside the

config:

block of your model YAML instead of at the same level as the

config:

block. As mentioned in a Slack conversation, Elementary tests should not be inside the config block but one level higher:

yaml    # Incorrect    config:      elementary:        timestamp_column: 'fact_date'      tests:        - elementary.table_anomalies:            tags: ['elementary']     # Correct    config:      elementary:        timestamp_column: 'fact_date'    tests:      - elementary.table_anomalies:          tags: ['elementary']

[Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1730300160839989) shows a user who fixed this exact issue. 2. Elementary package installation issues: Make sure you've properly installed the Elementary package and run

dbt deps

. Several users reported issues where Elementary tests weren't running because the package wasn't properly installed or initialized [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1703263929892129). 3. Missing Elementary models: Elementary tests require the Elementary models to be built first. If you're running Elementary for the first time or after an upgrade, you need to run

dbt run -s elementary

before running your tests [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1741856512416669). 4. Tag selection issues: If you're using tags to select which tests to run, make sure the tags are correctly configured. Some users reported issues with tag selection not matching any nodes [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1703263929892129). 5. On-run-end hook failures: If you have custom on-run-end hooks that fail before Elementary's on-run-end hook, it might prevent Elementary from properly processing test results [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1722936228529189). To troubleshoot, I recommend: 1. Check your YAML configuration to ensure tests are at the correct level 2. Run

dbt run -s elementary

to ensure Elementary models are built 3. Check your dbt logs for any specific error messages related to Elementary 4. Verify that your Elementary package is properly installed with

dbt deps

If the issue persists, examining the detailed logs with

dbt build --debug

might provide more insight into why the tests are being skipped. Would you like me to help troubleshoot any specific error messages you're seeing?

Based on the knowledge sources, when you add an Elementary data test to a dbt model, the test behavior follows these patterns: When the test gets run: 1. Elementary tests run during

dbt test

commands, just like standard dbt tests. As mentioned in a Slack conversation: "Once elementary dbt package is installed, you can run all elementary tests just like dbt tests" [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1686675189142809). 2. Tests will also run during

dbt build

commands, which combines both

dbt run

and

dbt test

. For models that don't change (including views): Elementary tests will still run even if the model code doesn't change and the model is materialized as a view. This is because: 1. Elementary's anomaly detection tests are time-based and look for changes in data patterns over time, not just changes in model code. They rely on the data in the

data_monitoring_metrics

table which accumulates metrics over time. 2. The materialization type (view, table, etc.) doesn't affect whether tests run - tests are executed against the data in the relation, regardless of how that relation is materialized. It's worth noting that Elementary models themselves (which are different from your models with Elementary tests) only need to be run when you first install Elementary or when upgrading to a new version. As one Elementary support staff mentioned: "you only need to run the Elementary models upon upgrading Elementary models. The reason being that schema changes to Elementary tables are applied upon running its models" [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1715108071844039). If you're finding that your Elementary tests aren't running despite not changing your model code, you might want to check your test configuration placement in the YAML file, as incorrect placement is a common issue [Slack forum discussion](https://slack.com/archives/C02CTC89LAX/p1730300160839989).