I may have found another bug thinking face in the volume ano Elementary Community #community-support

I may have found another bug :thinking_face: in th...

Ange Melhuish

10/13/2023, 7:01 PM

I may have found another bug 🤔 in the volume_anomalies test. When setting

period: week

and

count: 3

we had a error that looks to be due to a date miscalculation

Ange Melhuish

10/13/2023, 7:04 PM

We're using databricks on the latest version of elementary and dbt - Looking into the logs we see that 3 weeks causes an ending timestamp that is less than the start like here

Copy code

...        from (select explode(sequence(cast('2023-09-04T00:00:00+00:00' as timestamp), cast('2023-08-14T00:00:00+00:00' as timestamp), interval 3 week)) as edr_bucket_start) ...

Copy code

...      from (select explode(sequence(cast('2023-09-25T00:00:00+00:00' as timestamp), cast('2023-10-23T00:00:00+00:00' as timestamp), interval 2 week)) as edr_bucket_start) ...

Both 2 and 4 week intervals work but not 3 surprisingly. Is this because of our data or a bug?

Ella Katz

10/17/2023, 9:44 AM

Hi @Ange Melhuish, Can you send me the full configuration of the test?

Ange Melhuish

10/17/2023, 1:22 PM

for sure! It's pretty minimal

Copy code

- elementary.volume_anomalies:
          time_bucket:
            period: week
            count: 2
          config:
            severity: warn

Ella Katz

10/17/2023, 1:46 PM

interesting 🤔 what version of dbt_data_reliability package are you using?

Ella Katz

10/17/2023, 1:47 PM

also, can you try to run the test with 3 weeks period and add to your command line:

--vars '{"force_metrics_backfill": true}'

Ange Melhuish

10/17/2023, 2:01 PM

We recently upgraded to 0.11.1 from 0.9.4 😅 the backfill worked 🎉 Thanks so much!

Ella Katz

10/17/2023, 2:29 PM

Great! Glad to hear 😄

35 Views

Open in Slack

Previous Next