Niels van Beem
05/19/2025, 12:05 PMkapa.ai
05/19/2025, 12:05 PMkapa.ai
05/19/2025, 12:05 PMNiels van Beem
05/19/2025, 12:09 PMkapa.ai
05/19/2025, 12:09 PMkapa.ai
05/19/2025, 12:10 PMcolumn_name in the metrics_anomaly_score table is empty and why you might not see the expected results in your report: 1. Volume and Freshness Tests Are Table-Level The elementary.volume_anomalies and elementary.freshness_anomalies tests are table-level tests, not column-level. For these tests, the metrics are associated with the table as a whole, so the column_name field in the metrics tables (including metrics_anomaly_score) will be empty or null. This is expected behavior for these types of tests. Only column-level anomaly tests (like elementary.column_anomalies) will populate the column_name field with a value [see this discussion](https://slack.com/archives/C02CTC89LAX/p1708533386888479). 2. Not Enough Training Data If you see messages like "Not enough data to calculate anomaly score" or if the report is missing expected results, it could be due to insufficient training data. By default, Elementary requires a minimum number of historical data points (typically 14) to calculate anomaly scores. If your table does not have enough historical data in the CREATED_TIME column, or if the test hasn't run enough times to collect sufficient data, the anomaly score will not be calculated [see troubleshooting guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide). 3. Timestamp Column Issues If the CREATED_TIME column has missing, null, or incorrectly formatted values, metrics may not be collected as expected. Ensure that CREATED_TIME is populated and in the correct format for all relevant rows [see troubleshooting guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide). 4. Test Configuration and Execution Make sure that: - The tests are configured correctly in your YAML. - You are running dbt test and the Elementary on-run-end hooks are completing successfully. - The Elementary tables in your warehouse are up to date and compatible with your package version [see this answer](https://slack.com/archives/C02CTC89LAX/p1688995519653729). Summary: For table-level anomaly tests, it is normal for column_name to be empty in metrics_anomaly_score. If you are not seeing results, check for sufficient training data, correct timestamp column values, and successful test execution. If you want to monitor anomalies at the column level, use elementary.column_anomalies or elementary.all_columns_anomalies tests. If you have confirmed all the above and still do not see results, please provide more details about your data and test run history for further troubleshooting.Noya Offer
05/19/2025, 12:27 PM