@kapa.ai what's the default number of days of comparison when we do volume_anomalies test in a table?

For

elementary.volume_anomalies

, there are two relevant default periods to be aware of: • Training period (the historical window used to establish "normal" behavior): 14 days by default. [training_period docs] • Detection period (the window within which anomalies are actually checked): 2 days by default. [detection_period docs] So in practice, the test compares the row counts of the last 2 days (detection period) against the historical baseline built from the previous 14 days (training period). [volume_anomalies docs] > Note: Both

training_period

and

detection_period

only apply when a

timestamp_column

is configured. Without a timestamp column, the training period is built up over 14 test runs (not calendar days), and the detection period still defaults to 2 days. [Anomaly Troubleshooting Guide] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai so if i added a lot of rows to a table today, and my settings is like this: data_tests: - elementary.volume_anomalies: arguments: ignore_small_changes: spike_failure_percent_threshold: 10 drop_failure_percent_threshold: 10 what does the test do exactly?

Since your configuration has no

timestamp_column

, here's exactly what the test does: What happens without a timestamp_column 1. Each test run, Elementary performs a

COUNT(*)

on the entire table and saves that number as a new data point in

data_monitoring_metrics

. [Anomaly Troubleshooting Guide] 2. After enough runs (default: 14 runs to build the training set), it compares the current total row count to the average and standard deviation of previous runs. [volume_anomalies docs] 3. If the current row count is statistically anomalous (default: 3 standard deviations from the mean), it's flagged. How ignore_small_changes affects this With your settings (

spike_failure_percent_threshold: 10

,

drop_failure_percent_threshold: 10

), the test will only fail if both conditions are true: [ignore_small_changes docs] • The z-score is anomalous (exceeds sensitivity threshold), AND • The current metric is more than 10% above the training mean (for a spike), or more than 10% below (for a drop) Practical implication for your scenario If you added a lot of rows today: • The test will check if the new total row count is statistically anomalous compared to previous runs. • Even if it is statistically anomalous, it will only fail if the row count also exceeds the training mean by more than 10%. • If your training set is small or has high variance, the expected range can become very wide, potentially allowing large spikes to pass as "normal." [Slack discussion] > Note: The

time_bucket

,

training_period

,

detection_period

, and

backfill_days

parameters are effectively not used when

timestamp_column

is null — the test simply snapshots the full table row count on each run. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai but the Detection period is 2 days right? it will calculate the average of the last 2 days and compare it with the last 14 days ?

Good question — but there's an important clarification needed here, especially given your configuration (no

timestamp_column

). With a timestamp_column (timestamp-based tests) Yes, the detection period and training period work as you described: • Detection period (default: 2 days): The time buckets being tested for anomalies. • Training period (default: 14 days): The historical buckets used to calculate the expected range (average + standard deviation). So it compares the last 2 days' bucket metrics against the baseline built from the previous 14 days. [volume_anomalies docs] Without a timestamp_column (your case) Your configuration has no

timestamp_column

, so the behavior is different: • There are no time buckets. Each test run produces a single data point (the full table

COUNT(*)

). • The training set is built from the last 14 test runs (not calendar days). • The detection period concept (2 days) does not apply in the same way — instead,

backfill_days

controls how many recent runs are checked for anomalies. [Slack discussion] So in your case, the test is not averaging the last 2 days and comparing to 14 days. It's comparing the latest run's total row count against the statistical baseline (mean + stddev) built from the previous runs. [Anomaly Troubleshooting Guide] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.