Amy G
02/20/2025, 4:34 PMAmy G
02/20/2025, 4:34 PMcurrent_date() - 1 however, the average for the training period is 7 weeks prior inclusive of current_date() - 1 . I was looking at anomaly_exclude_metrics but when I configure this it filters out the date for both the detection and training period.
- elementary.volume_anomalies: # Monitors the row count of table over time per time bucket
# Known issue that detection date is included in training period: <https://github.com/elementary-data/elementary/issues/1491>
# The following suggestion excluded the detection date from both the detection and training period <https://github.com/elementary-data/elementary/issues/1491#issuecomment-2384317102>
# anomaly_exclude_metrics: "MAX(bucket_end) OVER (partition by metric_name, full_table_name, column_name, dimension, dimension_value, bucket_seasonality order by bucket_end asc rows between unbounded preceding and unbounded following) = bucket_end"
description: |
An anomaly detection test to compare volume of tickets created during the last complete day, against the
average of the last 7 weeks for that same day (within 3 standard deviations). This is because volumes
vastly differ depending on the day of the week.
severity: warn
timestamp_column: created_at # We could use received_at, but we want to capture volume of new tickets
anomaly_sensitivity: 3 # Default
seasonality: day_of_week # Volumes differ day to day so we want to compare against the same day of the week
detection_period: # We want to detect anomalies in volume from the last complete day
period: day
count: 1
training_period: # We want a larger training period to detect anomalies against the previous day
period: week
count: 1 # When seasonality: day_of_week is configured, training_period is by default multiplied by 7.
time_bucket: # We want to count new rows per day
period: day
count: 1
tags: [ 'elementary' ]Amy G
02/20/2025, 4:35 PMAmy G
02/20/2025, 5:04 PMThe last row_count value is 46409.000. The average for this metric is 44981.000.
Count of yesterday's rows: 46409
select count(1) from my_table
where created_at::date = '2025-02-19';
-- 46409
Average over last 7 weeks including yesterday: 44981
with cte as (
select
created_at::date as created_dt,
count(1) as c
from my_table
where 1=1
and created_dt > '2025-02-19'::date - 7*7
and dayname(created_dt) = 'Wed'
group by created_dt
order by created_dt
)
select avg(c)::int from cte
;
Average over last 7 weeks excluding yesterday: 44743
(6 days of data)
with cte as (
select
created_at::date as created_dt,
count(1) as c
from my_table
where 1=1
and created_dt < '2025-02-19'
and created_dt > '2025-02-19'::date - 7*7
and dayname(created_dt) = 'Wed'
group by created_dt
order by created_dt
)
select avg(c)::int from cte
;
I would expect the training set to not include the detection dateAmy G
02/24/2025, 1:28 PMNoya Offer
02/25/2025, 8:33 AMAmy G
02/25/2025, 8:43 AMNoya Offer
02/26/2025, 8:45 AMAmy G
02/26/2025, 8:53 AMAmy G
03/31/2025, 3:42 PMNoya Offer
04/08/2025, 6:55 AMAmy G
04/08/2025, 7:37 AMJoost Boonzajer Flaes
10/22/2025, 10:35 AMAmy G
10/22/2025, 12:31 PM