Can I adjust zscore sensitivity for specific field...
# ask-elementary-ai
n
Can I adjust zscore sensitivity for specific fields in tables and modify thresholds for different fields? I.e. pageviews 2.2, checkouts 1.8?
k
A message from kapa.ai
Yes, you can adjust the z-score sensitivity (anomaly_sensitivity) for specific fields (columns) in your tables using Elementary. This allows you to set different sensitivity levels for different fields, such as pageviews at 2.2 and checkouts at 1.8. To do this, you define the anomaly detection test at the column level and specify the desired anomaly_sensitivity for each field. Here’s an example configuration:
Copy code
yaml models:   - name: your_model     columns:       - name: pageviews         tests:           - elementary.column_anomalies:               column_anomalies:                 - null_count               anomaly_sensitivity: 2.2       - name: checkouts         tests:           - elementary.column_anomalies:               column_anomalies:                 - null_count               anomaly_sensitivity: 1.8
This configuration will apply a sensitivity of 2.2 to the pageviews column and 1.8 to the checkouts column. The anomaly_sensitivity parameter controls how wide the expected range is (in terms of standard deviations from the mean), so a lower value will make the test more sensitive and flag more anomalies, while a higher value will make it less sensitive and flag fewer anomalies. This approach is supported for all anomaly detection tests in Elementary, including column-level tests like column_anomalies and all_columns_anomalies, as well as table-level tests like volume_anomalies and freshness_anomalies [anomaly_sensitivity documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/anomaly-sensitivity). If you want to set different thresholds for different fields, you should use the anomaly_sensitivity parameter as shown above. Note that setting absolute thresholds (e.g., "alert if value > 1000") is not currently supported; thresholds are based on z-score sensitivity, not fixed values [Slack discussion on absolute thresholds](https://slack.com/archives/C02CTC89LAX/p1706243206971089). In summary, you can customize the z-score sensitivity per field/column, but not set absolute value thresholds.
n
My current config applies to all columns, could i only specify 1 or 2 fields with different settings, or would i need to define all fields with their own thresholds? @kapa.ai
k
A message from kapa.ai
You do not need to define all fields with their own thresholds. You can specify custom settings (such as anomaly_sensitivity) for only the 1 or 2 fields you want to treat differently, and the rest will use the default or global configuration. For example, you can use elementary.column_anomalies on just the specific columns you want to override, while the rest of the columns can be monitored using elementary.all_columns_anomalies or by inheriting the default settings. This approach is supported and commonly used: - You can define multiple column_anomalies tests on specific columns with custom settings. - The rest of the columns do not need to be explicitly listed unless you want to override their settings as well. This is confirmed in community discussions, where users define column_anomalies on specific columns for custom behavior, and all_columns_anomalies for general coverage. You do not need to repeat the configuration for every column unless you want to customize each one individually [Slack discussion on column-level config](https://slack.com/archives/C02CTC89LAX/p1692020813598059). So, you can specify custom settings for just the fields you care about, and leave the rest to use the default or global settings.