@kapa.ai Is there a way to adjust the detection period to only fail if the most recent run was an anomaly? We run multiple times per day, so it won't work to just set the detection period to one day

This is a known limitation in Elementary, and the answer depends on whether your table has a

timestamp_column

or not. Without a timestamp_column The

backfill_days

parameter controls the detection period, with a minimum value of 1 day. Since you run multiple times per day, setting

backfill_days: 1

will still include all runs within the last 24 hours — so if an earlier run in the same day was anomalous, the test will continue to fail even after a fix. [backfill_days limitation] There is no supported way to scope detection to only the single most recent run without a timestamp column. With a timestamp_column If your table has a timestamp column, you have more control. You can use smaller

time_bucket

periods (e.g., hourly) so that each run maps to a finer-grained bucket:

models:
  - name: your_table_name
    data_tests:
      - elementary.volume_anomalies:
          arguments:
            timestamp_column: created_at
            time_bucket:
              period: hour
              count: 1

This way, anomalies are scoped to hourly buckets rather than daily ones, reducing the window in which a fixed anomaly continues to cause failures. [detection_period issue] Workaround: suppression_interval Regardless of your setup, the recommended workaround for repeated alerts on the same issue is the

suppression_interval

parameter, which prevents duplicate alerts for the same problem over a configurable number of hours. [suppression_interval workaround] Important caveat There is also an open bug (GitHub issue #1523) where

detection_period: 1 day

uses a non-strict

>=

condition, meaning it can flag anomalies from 2 days ago rather than just 1. This has been acknowledged as a bug by the Elementary team but may not yet be fixed. [detection_period bug] In summary, Elementary does not currently support scoping detection to strictly the most recent run when running multiple times per day without a timestamp column. Using hourly time buckets with a

timestamp_column

is the closest available approach. 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.

@kapa.ai If we have tests running hourly, detection delay of 1 day considers 24 previous bucket. A test failure can keep throwing error for 24 job runs. Can we set detection period on hourly level? How would you configure the test for it with suppression_interval

Thinking completed Can you set the detection period at the hourly level? No, you cannot currently set the detection period to an hourly level. While the documentation for

detection_period

mentions periods like

day

,

week

, and

month

[detection_period#page-title], attempts to set

period: hour

result in a compilation error (

Missing mandatory configuration: ['backfill_days']

). Under the hood, Elementary translates detection periods into full days (

backfill_days

), requiring the detection period to be longer than 24 hours (minimum of 1 day) [Not able to set up detection period - 'hour' for column_anomalies tests; Add --hours-back CLI option for edr monitor]. As a result, any anomaly detected within that 1-day detection window will cause the test to continue failing across your hourly runs throughout that day. How to configure the test and handle repeated alerts with suppression_interval While the underlying test execution will still fail on subsequent runs until the anomalous bucket ages out of the detection window, you can prevent alert spam by using

alert_suppression_interval

[Slack forum user question]. 1. Configure the test in your .yml file To limit the re-alerting for the 24-hour period while monitoring on an hourly bucket, configure the test properties with hourly buckets and add

alert_suppression_interval: 24

to the test's

meta

configuration:

models:
  - name: your_model_name
    data_tests:
      - elementary.volume_anomalies:
          arguments:
            timestamp_column: your_timestamp_column
            time_bucket:
              period: hour
              count: 1
            detection_period:
              period: day
              count: 1
          config:
            meta:
              alert_suppression_interval: 24

Notes on this configuration: •

time_bucket

: Groups data points into 1-hour increments [detection_period propery works wrong]. •

detection_period

: Left at the minimum possible of 1 day [Not able to set up detection period - 'hour' for column_anomalies tests]. •

alert_suppression_interval: 24

: Snoozes repeated alert notifications for the same issue for 24 hours [Alerts Configuration and Customization]. 2. Alternatively, set suppression globally via the CLI If you don't want to set it on every individual test, you can supply

--suppression-interval 24

directly to your

edr monitor

command:

edr monitor --suppression-interval 24

[edr monitor --filters statuses:warn --owner @xxxx --sla] 👍 0 button 👎 0 button ℹ️ Tip: Mention @elementary.ai in the thread for follow-ups or DM the app for 1-to-1 help.