Hello,
I have a quick question and I was wondering if somebody could help. I am working in the data governance team at veepee and we have recently started using elementary. I am currently in the process of setting up volume anomaly testing on all of the raw tables that we have to ingest data.
I want to incorporate the seasonality parameter into my volume anomaly tests at the
seasonality: day_of_week level. However, when I do this I am told that I need to have my time bucket at day level.
My problem here is that since elementary is only running tests on data where the time bucket needs to be completed - in the case that we have a volume anomaly we will only capture it the day after it occurs - but, we want to capture the anomaly the day it occurs.
Any suggestions on how to get around this issue? Below is an example test we are using.
Thanks in advance
tests:
- elementary.volume_anomalies:
timestamp_column: "meta_processing_time" # timestamp field to group the volum
tags: ["volume_freshness"]
days_back: 14 # nb of days of data to retrieve
anomaly_sensitivity: 3 # 1 lowest - 3 highest
anomaly_direction: drop #spike: Increase of volume | drop: decrease of volume | both: spike & drop
backfill_days: 2
time_bucket:
period: day
count: 1
seasonality: day_of_week
config:
severity: warn # warn | error