shobia
11/24/2025, 6:45 AMtest.pem_dbt.elementary_column_anomalies_item_branch__parquet_common_refined4_both__2__null_count__itembranch_creation_date__3__3__None.77313c61c8kapa.ai
11/24/2025, 6:45 AMkapa.ai
11/24/2025, 6:45 AMcolumn_anomalies test, so Elementary can’t compute an anomaly score and prints: > Not enough data to calculate anomaly scores … Key points from the docs and Slack threads: 1. Timestamp vs. no timestamp - If you configure a timestamp_column, Elementary buckets data by time and can often start detecting anomalies immediately, as long as there is enough historical data in those buckets.[[Anomaly params](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/anomaly-params); [Troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] - If you do not have a timestamp (or set it to null as in your config), Elementary treats each test run as a time point; you must accumulate enough runs over different days for a training set.[[Troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] In your YAML you have: yaml timestamp_column: null which effectively configures this as a non‑timestamp test. 2. How much data is “enough”? There are two slightly different thresholds in the material you shared: - Some docs/issues mention a default min_training_set_size of 14 (14 days / runs).[[Slack 1682967620](https://slack.com/archives/C02CTC89LAX/p1682967620411319); [Slack 1692691883](https://slack.com/archives/C02CTC89LAX/p1692691883697119)] - The Troubleshooting guide describes “Not enough data to calculate anomaly” as fewer than 7 training data points.[[Troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] Either way, the message means: Elementary has fewer than the required number of historical points for null_count on this column. 3. What you can do Option A – Use a real timestamp column (recommended) If your table has a creation or partition date column, configure it as the table‑level timestamp instead of putting timestamp_column: null in the test: yaml models: - name: item_branch__parquet config: elementary: timestamp_column: creation_date # or the correct time column columns: - name: "itembranch.creation date" tests: - elementary.column_anomalies: column_anomalies: - null_count anomaly_sensitivity: 2 anomaly_direction: both ignore_small_changes: spike_failure_percent_threshold: 3 drop_failure_percent_threshold: 3 tags: ["elementary", "daily_elementary", "refined4", "common_refined4"] With a proper timestamp and enough historical data in that column, Elementary can often calculate anomaly scores immediately.[[Column anomalies config](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies); [Configure tests](https://docs.elementary-data.com/data-tests/add-elementary-tests#configure-your-elementary-anomaly-detection-tests)] Option B – Keep it non‑timestamp and build training data If you really want to run without a timestamp: - Keep timestamp_column out of the test (don’t set it to null at the test level; just omit it, as in the non‑timestamp example in the docs).[[Column anomalies config](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies)] - Run the test on multiple different days so that data_monitoring_metrics accumulates enough null_count points for this column.[[Troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide); [Slack 1687786556](https://slack.com/archives/C02CTC89LAX/p1687786556886149)] - Optionally lower min_training_set_size globally if you only need a demo or are okay with less stable thresholds (the exact YAML for that isn’t in the snippets you provided, only that it exists).[[Slack 1700704884](https://slack.com/archives/C02CTC89LAX/p1700704884937269); [Slack 1692691883](https://slack.com/archives/C02CTC89LAX/p1692691883697119)] 4. How to verify Use the troubleshooting queries to confirm metrics are being collected: sql SELECT metric_name, metric_timestamp, metric_value FROM your_schema.data_monitoring_metrics WHERE table_name = 'item_branch__parquet' AND column_name = 'itembranch.creation date' ORDER BY metric_timestamp DESC; You should see multiple null_count rows over different timestamps; if there are fewer than the training minimum, you’ll keep getting the “Not enough data” message.[[Troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] --- In short: the message is expected until you either (a) provide a real timestamp column with enough history, or (b) run the non‑timestamp test enough times (or with a lower min_training_set_size) to build a training set.