Igor Dal Bo
08/21/2025, 2:12 PMdata_tests:
- elementary.freshness_anomalies:
timestamp_column: load_timestamp
training_period:
period: hour
count: 336
time_bucket:
period: hour
count: 2
detection_period:
period: hour
count: 2
I have seen this was previously a problem and was solved in one recent version, however I still get the error
Missing mandatory configuration: ['backfill_days']
> in macro validate_mandatory_configuration (macros/edr/tests/test_configuration/get_anomalies_test_configuration.sql)
> called by macro get_anomalies_test_configuration (macros/edr/tests/test_configuration/get_anomalies_test_configuration.sql)
> called by macro test_table_anomalies (macros/edr/tests/test_table_anomalies.sql)
> called by macro test_freshness_anomalies (macros/edr/tests/test_freshness_anomalies.sql)
> called by test elementary_freshness_anomalies_stg__cloudflare_logs_hour__2__hour__2__load_timestamp__hour__336 (models/cloudflare/_model.yml)
even if in the documentation it looks possible to use hour as value for everythingkapa.ai
08/21/2025, 2:12 PMkapa.ai
08/21/2025, 2:12 PMMissing mandatory configuration: ['backfill_days'] This is a known issue and has not yet been fully resolved, even in recent versions. The root cause is that, under the hood, the detection_period and training_period parameters are still being translated to "days_back" and "backfill_days", and the code currently requires these to be at least one day (i.e., 24 hours or more). As a result, using period: hour with a count less than 24 will trigger this error, even though the documentation suggests it should work. This limitation is confirmed in several sources: - A GitHub issue explicitly states that detection_period must be longer than one day, and that using a shorter period (e.g., 2 hours) will not work and will result in the error you are seeing. There is no current workaround to allow for true hourly detection periods, though there is interest from the community and the Elementary team in supporting this in the future. [Not able to set up detection period - 'hour' for column_anomalies tests](https://github.com/elementary-data/elementary/issues/1646) - Slack discussions and other GitHub issues confirm that this is a bug/limitation, and that the only current workaround is to set detection_period to at least one day. [Slack thread on hourly detection_period](https://slack.com/archives/C02CTC89LAX/p1704990385240799) - The documentation does not reflect this limitation, but the implementation does not yet support sub-day detection periods for these tests. Summary: Even though the documentation suggests you can use hour-level granularity for detection_period and training_period, in practice, you must set detection_period to at least one day (period: day, count: 1) to avoid the "Missing mandatory configuration: ['backfill_days']" error. Hourly detection periods are not yet supported, and there is no current workaround to enable this behavior in the latest version of the package. If you need more granular (hourly) anomaly detection, you will need to wait for a future release where this is properly supported, or consider alternative approaches as suggested in the GitHub issues and Slack threads referenced above.