New message text here

@kapa.ai Also, for this test, the table should be append only, but I'd like to test that. Is there a way to make the test fail if the count ever decreases or if it increases by more than one standard deviation?

Elementary’s

dimension_anomalies

test is designed to detect unexpected changes in row counts for each dimension (or combination of dimensions) over time. However, based on the available documentation and community discussions, there is no explicit configuration to make the test fail only if the count ever decreases (i.e., to enforce strict append-only behavior), nor to set a custom threshold such as "increase by more than one standard deviation" for the anomaly detection. The anomaly detection algorithm in Elementary is built to flag significant deviations (spikes or drops) from historical patterns, and you can control the sensitivity and direction of anomalies using parameters like

anomaly_sensitivity

and

anomaly_direction

(set to

spike

,

drop

, or

both

). But you cannot directly specify a rule such as "fail if count decreases" or "fail if increase > 1 stddev"—the thresholds are determined by the anomaly detection algorithm, not by user-defined static rules [dimension_anomalies test configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies#test-configuration), [how anomaly detection works](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works). If you need to enforce strict append-only behavior (i.e., row count never decreases), you may want to use a custom dbt test or a test from another package (such as dbt-expectations) that allows you to set explicit row count constraints. Elementary does support showing the results of such custom or dbt-expectations tests in its UI [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1677563837770559). Summary: - You can use

anomaly_direction: drop

to focus on drops, but not to enforce "never decrease." - There is no built-in way to set a custom threshold like "increase by more than one standard deviation." - For strict append-only checks, consider a custom dbt test or dbt-expectations. If you need help writing such a custom test, let me know!