<@U088WCP3B4P> can I add a SQL code to a column_an...
# ask-elementary-ai
g
@kapa.ai can I add a SQL code to a column_anomaly test and use it to group a metric?
k
A message from kapa.ai
Yes, you can group a metric in a column_anomalies test using the dimensions parameter, which accepts a list of SQL expressions. This allows you to segment your anomaly detection by one or more columns or even by SQL expressions, effectively grouping your metric as needed. For example, you can configure your test like this:
Copy code
yaml models:   - name: your_model     columns:       - name: your_metric_column         tests:           - elementary.column_anomalies:               column_anomalies:                 - sum               dimensions:                 - category_column                 - another_dimension
The documentation specifies that the dimensions parameter can be a list of SQL expressions, so you can use SQL code (such as CASE statements or window functions) to define your grouping logic. Each unique combination of dimension values will be monitored separately for anomalies. This feature is available for column_anomalies, all_columns_anomalies, and dimension_anomalies tests. It is recommended to use low-cardinality fields for dimensions to avoid excessive segmentation and performance issues. For more details and examples, see the official documentation on dimensions and recent changelog updates introducing this feature [Dimensions configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions), [Column anomalies by dimensions](https://docs.elementary-data.com/changelog#column-anomalies-by-dimensions). If you want to use more complex SQL logic, you can include SQL expressions in the dimensions list, such as:
Copy code
yaml dimensions:   - CASE WHEN region = 'US' THEN 'North America' ELSE 'Other' END   - product_category
This will group your anomaly detection by the result of the SQL expression and the product_category column. In summary: Yes, you can use SQL code in the dimensions parameter of a column_anomalies test to group your metric for anomaly detection.
g
@kapa.ai how to insert that in the following example
Copy code
column_name: affiliate_commissions
              config: 
                
                group_by_columns: [site, utm_source]        # was shop_name, affiliate_session_traffic_channel
                time_bucket: { period: day, count: 1 }
                training_period: 28
                detection_period: 3
                anomaly_sensitivity: 3.2
                where_expression: "event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 35 DAY)"
                severity: warn
                column_anomalies: [sum]
                min_training_set_size: 21
                anomaly_direction: "drop"
                seasonality: "day_of_week"
k
A message from kapa.ai
To group your column_anomalies test by dimensions (such as site and utm_source), you should use the dimensions parameter instead of group_by_columns. The correct configuration, based on the latest Elementary documentation, looks like this:
Copy code
yaml column_name: affiliate_commissions config:    dimensions:     - site     - utm_source   time_bucket:     period: day     count: 1   training_period: 28   detection_period: 3   anomaly_sensitivity: 3.2   where_expression: "event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 35 DAY)"   severity: warn   column_anomalies:     - sum   min_training_set_size: 21   anomaly_direction: "drop"   seasonality: "day_of_week"
- Replace group_by_columns with dimensions. - The dimensions parameter accepts a list of columns or SQL expressions to group by. - This will monitor the sum of affiliate_commissions for each (site, utm_source) combination separately for anomalies. For more details, see the official documentation on [dimensions configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions) and [column_anomalies test configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies#test-configuration).
g
@kapa.ai put an sql code as example insde the dimensios
k
A message from kapa.ai
Certainly! You can use SQL expressions inside the dimensions parameter of a column_anomalies test. For example, if you want to group by a transformed version of utm_source (e.g., uppercased) and a conditional site label, your configuration could look like this:
Copy code
yaml column_name: affiliate_commissions config:    dimensions:     - UPPER(utm_source)     - CASE WHEN site = 'main' THEN 'primary' ELSE 'secondary' END   time_bucket:     period: day     count: 1   training_period: 28   detection_period: 3   anomaly_sensitivity: 3.2   where_expression: "event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 35 DAY)"   severity: warn   column_anomalies:     - sum   min_training_set_size: 21   anomaly_direction: "drop"   seasonality: "day_of_week"
This will group the anomaly detection by the uppercased utm_source and by whether site is 'main' or not. The documentation confirms that dimensions can be a list of SQL expressions, not just column names. Each unique combination of these expressions will be monitored separately for anomalies [dimensions configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions). If you need more complex logic, you can use any valid SQL expression as an entry in the dimensions list.