Sofia Alevizopoulou
06/20/2024, 9:06 AMdata_tests:
- elementary.volume_anomalies:
training_period:
period: day
count: 3
detection_period:
period: day
count: 1
this is the metric_properties from elementary.data_monitoring_metrics
{
"dimensions": null,
"event_timestamp_column": null,
"freshness_column": null,
"time_bucket": {
"count": 1,
"period": "day"
},
"timestamp_column": null,
"where_expression": null
}
Upon examining the elementary.metrics_anomaly_score table, I noticed that the training_avg and training_stddev values are higher than expected. It appears that the calculation is not considering the specified 3-day training period from elementary.data_monitoring_metrics.
As you can see the count of the table this month is 35 rows (used to have ~1000 rows in the previous months). Today I reduced it to 10 rows and later to 1 row to see if the test will fail but as expected it doesn't fail as the training_avg is too big (I was expected to be 35).
I was wondering if I miss anything else from the configuration to correctly calculate the training_avg and training_stddev so the anomlay score can be calculated correctly.
Thank you!