shobia
11/06/2025, 9:35 AMkapa.ai
11/06/2025, 9:36 AMkapa.ai
11/06/2025, 9:36 AMyaml data_tests: - elementary.volume_anomalies: anomaly_direction: both ignore_small_changes: spike_failure_percent_threshold: 1 drop_failure_percent_threshold: 1 tags: ["elementary", "daily_elementary", "refined4", "common_refined4"] ### What does this test do? - elementary.volume_anomalies monitors the row count of your table over time, split into time buckets (daily by default). - anomaly_direction: both means the test will flag anomalies for both unexpected spikes (increases) and drops (decreases) in row count. - ignore_small_changes with both thresholds set to 1 means the test will only fail if the row count in the detection period is more than 1% higher (spike) or 1% lower (drop) than the mean value in the training period, in addition to being statistically anomalous. This helps filter out minor fluctuations that are not meaningful. - tags are just metadata for organizing and filtering tests. So, this test will alert you if there is a statistically significant and meaningful (greater than 1% change) spike or drop in the number of rows in your table, based on recent historical data. This is useful for catching unexpected data volume changes, such as missing data or duplicate loads, while ignoring very small, insignificant changes [ignore_small_changes](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/ignore_small_changes) [volume_anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/volume-anomalies). --- ### How does the "average" column anomaly work? The average column anomaly monitor checks for anomalies in the average value of a numeric column over time. It works by: - Calculating the average value of the column for each time bucket (e.g., daily). - Comparing the average in the detection period (e.g., last 2 days) to the historical averages in the training period (e.g., previous 14 days). - If the average in the detection period is an outlier compared to the expected range (based on historical data), it is flagged as an anomaly. #### Example configuration: yaml models: - name: your_model columns: - name: your_numeric_column tests: - elementary.column_anomalies: column_anomalies: - average This will monitor the average value of your_numeric_column for anomalies. If, for example, the average value suddenly increases or decreases beyond what is expected based on historical trends, the test will fail and alert you [column_anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies) [column_anomalies config](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/column-anomalies). --- If you want to add the average column anomaly test, you can use the example above and adjust it for your model and column. If you need more details or a specific example for your data, let me know!shobia
11/06/2025, 9:38 AM