Hi Everyone I am trying to understand the anomaly detection the following is the yml I have. version: 2 sources: - name: dq_source # Name of the database containing the source description: "Source database containing transactional data." schema: dq_source # Schema where the table is located tables: - name: test_quality columns: - name: a tests: - elementary.column_anomalies: timestamp_column: datetime column_anomalies: - average detection_period: period: day count: 1 training_period: period: day count: 5 time_bucket: period: day count: 1 the following are the data datapoints I have. a datetime 30 2025-01-31T080000.000+00:00 70 2025-01-31T200000.000+00:00 40 2025-02-01T080000.000+00:00 60 2025-02-01T200000.000+00:00 20 2025-02-02T080000.000+00:00 80 2025-02-02T200000.000+00:00 500 2025-02-04T080000.000+00:00 500 2025-02-04T200000.000+00:00 50 2025-02-03T080000.000+00:00 50 2025-02-03T200000.000+00:00 200 2025-02-05T152509.958+00:00 20000 2025-02-05T161317.333+00:00 I was expecting that for 4th I should be getting an anomaly because the value of a is quite large but it didn’t happen. Instead the range is updated to 463 to 743. More over the data points 2000 is not getting considered. I think I am missing something completely. Please can someone help me in understanding.

Hey Dharmin! The values shown for February 5th at 00:00 are actually the average for February 4th. What happens when if you run this again today?

Now it considers but it also updates the range. Not sure how it is arriving at the range. My understanding was it is using z-score. If that is the case i should be getting z score value 0-3.

The expected range for each data point is updated every time a new value comes in. It uses the latest value to adjust the expected range, and then that value is checked against it. This is why the anomaly graph looks the way it does 🙂

But this updation of range should happen based on training data right? If it's adjusted to any latest value how will the algorithm know if the latest value is a anomaly or not?