@kapa.ai Given a table with columns:

activity_id

,

activity_timestamp

,

activity_source

,

activity_type

,

activity_metric_1

,

activity_metric_2

I want to have a test which checks this table daily for row counts, and identifies if there is a drop off or complete absence for a

activity_source

, given that different sources will have different usual volumes.

Using dimension_anomalies for Per-Source Volume Monitoring The best fit for your use case is

elementary.dimension_anomalies

. This test monitors the row count per dimension value (in your case, per

activity_source

) over time, and alerts on unexpected changes in distribution — including drops or complete absence for a specific source. It handles different usual volumes per source naturally, since each source's history is tracked independently. Here's the configuration:

models:   - name: your_table_name     config:       elementary:         timestamp_column: activity_timestamp     data_tests:       - elementary.dimension_anomalies:           dimensions:             - activity_source           timestamp_column: activity_timestamp           anomaly_direction: drop        # only alert on drops/absences           time_bucket:             period: day             count: 1           training_period:             period: day             count: 14                   # 14 days of history (default)           detection_period:             period: day             count: 1                    # check yesterday's completed bucket

How this works • The test groups rows by

activity_source

and counts rows per daily time bucket. [dimension anomalies] • Each

activity_source

value is tracked independently, so a source with naturally high volume won't mask a drop in a low-volume source. [dimension anomalies] • Setting

anomaly_direction: drop

means it will only alert on unexpected decreases or complete absence, not on spikes. • The test only evaluates completed time buckets, so running it during the day will evaluate yesterday's data. [volume anomalies] • It is recommended to configure

activity_source

as a low-cardinality field for best results — if you have many distinct sources, consider filtering with

where_expression

. [dimension anomalies] Optional: Ignore very small fluctuations If you want to avoid noise from minor drops, you can add:

ignore_small_changes:   drop_failure_percent_threshold: 10   # only fail if drop > 10%

This ensures only meaningful drops (beyond 10% relative to historical average) trigger a failure. [dimension anomalies] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.