Hi, I have a question about the

column_anomalies

test (elementary 0.18.3). Is there a way to suppress warnings or errors if the tested value is below a certain threshold? For example, in the configuration below, I'd like to avoid any alerts if

total_cost < 20

, even if there's a spike. Is this currently possible either through configuration or a workaround?

- elementary.column_anomalies:
          timestamp_column: "dt"
          column_name: "total_cost"
          column_anomalies:
            - max
          anomaly_direction: spike
          anomaly_sensitivity: 1
          ignore_small_changes:
            spike_failure_percent_threshold: 30
          fail_on_zero: false
          dimensions: ["cluster_name"]
          tags: ['elementary-test']
          config:
            severity: warn
          meta:
            description: "Daily cost anomaly test for all-purpose clusters. To check problematic cluster, check elementary UI"
            alert_fields: ["table","column","tags","description","owners","result_message","test_parameters"]

Hey @Mikhail Kuklin! I see that you did adjust sensitivity and spike failure threshold, which is what I would recommend. Are you getting many failures on what you'd consider "normal" behavior? We are considering a warn if/error if option. We should be looking into it soon!

Hey @Noya Offer. Thank you for your reply. Yes, the problem is that majority of cases are not significant for us but they are significant from statistics point of view 😃 For example (attached printscreen), max column is the cost and it is 2.5$ and we do not care if it is less than 20$.

I understand! Then why don't you raise the sensitivity?

How sensitivity can help with the limit? 🤔