@kapa.ai I have this test for a model in dbt:

- elementary.column_anomalies:
              config:
                severity: warn
              arguments:
                ignore_small_changes:
                  spike_failure_percent_threshold: 10
                  drop_failure_percent_threshold: 10

sometimes the I get warns from this test with messages like this:

in column X, the last missing_count value is 17.000. the average for this metric is 20.829.

We have 40000 rows in this table, and changing from 20.8 to 17 rows is just a too small change so I don't want to get alert from this test. How to fix it?

Looking at your case: the metric dropped from ~20.8 to 17, which is about an 18% drop — this exceeds your

drop_failure_percent_threshold: 10

, so the test correctly fires. The issue is that a change of ~3–4 rows out of 40,000 is simply not meaningful to you, even if it's statistically significant. Here are your options: 1. Raise drop_failure_percent_threshold Since the drop from 20.8 → 17 is ~18%, you need to raise the threshold above that. For example, setting it to

25

or higher would suppress this kind of noise:

- elementary.column_anomalies:
    config:
      severity: warn
    arguments:
      ignore_small_changes:
        spike_failure_percent_threshold: 10
        drop_failure_percent_threshold: 25

[ignore_small_changes docs] 2. Lower anomaly_sensitivity

ignore_small_changes

works on top of the z-score check — both conditions must be true for the test to fail. Raising

anomaly_sensitivity

(e.g., from the default

3

to

4

or

5

) widens the expected range, making the z-score less likely to flag small deviations as anomalous. [ignore_small_changes docs]

- elementary.column_anomalies:
    config:
      severity: warn
    arguments:
      anomaly_sensitivity: 5
      ignore_small_changes:
        spike_failure_percent_threshold: 10
        drop_failure_percent_threshold: 25

3. What's currently not possible There is no built-in absolute threshold (e.g., "only alert if

missing_count

changes by more than 5 rows"). A

spike_failure_absolute_threshold

/

drop_failure_absolute_threshold

has been raised as a feature request by the community but was closed as stale. [GitHub issue] [community discussion] Recommendation: For your case (a change of ~3–4 rows out of 40,000), raising

drop_failure_percent_threshold

to something like

25–30

combined with a higher

anomaly_sensitivity

is the most practical fix with the current configuration options. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai I don't understand the difference between spike_failure_percent_threshold and

anomaly_sensitivity

, explain in detail with examples

anomaly_sensitivity vs spike_failure_percent_threshold These two parameters control different stages of the anomaly detection process, and both conditions must be true for a test to fail. Stage 1: anomaly_sensitivity — Is this statistically unusual? This controls how wide the "expected range" is, based on standard deviations from the historical mean. The formula for the upper bound (spike) is:

upper_bound = training_avg + (anomaly_sensitivity × training_stddev)

If the detected value exceeds this bound, it is flagged as a statistical anomaly. • A lower value = narrower range = more sensitive (flags smaller changes) • A higher value = wider range = less sensitive (only flags extreme changes) • Default is

3

(i.e., 3 standard deviations) Example: Suppose

missing_count

has a training average of

20

and a standard deviation of `2`: Stage 2: spike_failure_percent_threshold — Is this change large enough to care about? Even if Stage 1 flags a value as statistically anomalous,

ignore_small_changes

adds a second filter: the change must also exceed a certain percentage of the training mean.

threshold = training_avg × (1 + spike_failure_percent_threshold / 100)

The test only fails if the value exceeds both the sensitivity bound and this percentage threshold. [ignore_small_changes docs] Example: Training average =

20

, `spike_failure_percent_threshold: 10`:

percent_threshold = 20 × 1.10 = 22

So even if

anomaly_sensitivity

flags a value of

21

as statistically anomalous, the test will not fail because 21 is not more than 10% above the mean of 20. Putting it together — your actual case Your

missing_count

went from avg

20.829

→

17.0

(a drop of ~18%). Since both conditions are met, the test warns. To suppress it, you need to raise

drop_failure_percent_threshold

above 18.4%, or raise

anomaly_sensitivity

so Stage 1 no longer flags it as anomalous. [ignore_small_changes docs] Summary Both must be satisfied simultaneously for the test to fail. They are complementary —

anomaly_sensitivity

catches statistical outliers, while

ignore_small_changes

ensures those outliers are also practically significant. [ignore_small_changes docs] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help., with interactive elements