https://linen.dev logo
Join Slack
Powered by
This message was deleted.
# ask-elementary-ai
s
This message was deleted.
k
A message from kapa.ai
To check for anomalies in each event count column (e.g., 
event_1_count
, 
event_2_count
, etc.) in your daily aggregated table, you should use the 
elementary.column_anomalies
test. This test can monitor metrics like 
standard_deviation
for numeric columns and alert when values deviate from the expected range. Here’s an example of how you can configure this in your 
schema.yml
(or equivalent) for each event count column. This setup will monitor the standard deviation and alert if the daily value is outside the expected range (by default, 3 standard deviations, but you can adjust the sensitivity):
Copy code
yaml models:   - name: your_table_name     config:       elementary:         timestamp_column: date_pt     columns:       - name: event_1_count         tests:           - elementary.column_anomalies:               column_anomalies:                 - standard_deviation               time_bucket:                 period: day                 count: 1               anomaly_sensitivity: 1  # Lower values = more sensitive (default is 3)       - name: event_2_count         tests:           - elementary.column_anomalies:               column_anomalies:                 - standard_deviation               time_bucket:                 period: day                 count: 1               anomaly_sensitivity: 1       # Repeat for other event count columns as needed
- The 
anomaly_sensitivity
parameter controls how many standard deviations from the mean are considered anomalous. The default is 3 (following the empirical rule), but you can set it lower (e.g., 1 or 2) for more sensitive alerts. - The 
time_bucket
ensures the test runs daily. This configuration will fire an alert whenever the daily event count is not within the specified number of standard deviations from the historical mean, as calculated by Elementary’s anomaly detection method using Z-score [[column_anomalies doc](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies)] [[anomaly detection method](https://docs.elementary-data.com/data-tests/data-anomaly-detection)]. If you want to apply this to unique user counts as well, simply add similar blocks for those columns. Let me know if you need an example for a specific column or want to adjust the sensitivity further!
A message from kapa.ai
Yes, currently in dbt and with Elementary's open-source package, you need to define the anomaly detection test for each individual column you want to monitor. There is no built-in, DRY (Don't Repeat Yourself) way to apply a column anomaly test to all columns or a subset of columns in a table with a single configuration block. This is a limitation of how dbt tests are defined: dbt does not support global or schema-level test definitions that automatically apply to all columns or models [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1670863565035799)][[Slack answer](https://slack.com/archives/C02CTC89LAX/p1676902538214649)][[Slack answer](https://slack.com/archives/C02CTC89LAX/p1686671035677819)]. However, Elementary does provide the 
all_columns_anomalies
test, which can be applied at the model level to run anomaly detection on all columns in a table. This is the closest option to a DRY approach, but it applies the same configuration to all columns and may not be as granular as configuring each column individually. You can also use parameters like 
exclude_regexp
to skip certain columns if needed [[all_columns_anomalies doc](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/all-columns-anomalies)]. Example:
Copy code
yaml models:   - name: your_table_name     config:       elementary:         timestamp_column: date_pt     tests:       - elementary.all_columns_anomalies:           column_anomalies:             - standard_deviation           time_bucket:             period: day             count: 1           anomaly_sensitivity: 1           exclude_regexp: "^(date_pt)$"
This will apply the anomaly detection to all columns except those matching the 
exclude_regexp
pattern. Summary: - You must define tests per column for fine-grained control. - Use 
all_columns_anomalies
for a DRY approach at the model level, but with less flexibility per column. There is currently no way in open-source dbt/Elementary to define a test once and have it automatically apply to all relevant columns or models without some repetition [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1670863565035799)][[Slack answer](https://slack.com/archives/C02CTC89LAX/p1676902538214649)].
A message from kapa.ai
You can achieve this by using the 
dimensions
parameter in the 
elementary.all_columns_anomalies
test. This parameter allows you to segment your anomaly detection by one or more columns (such as 
device
), so that each device's counts are only compared against historical values for that same device. Here’s how you can configure it in your `schema.yml`:
Copy code
yaml models:   - name: your_table_name     config:       elementary:         timestamp_column: day_pt     tests:       - elementary.all_columns_anomalies:           column_anomalies:             - standard_deviation           dimensions:             - device           time_bucket:             period: day             count: 1           anomaly_sensitivity: 1
With this configuration, the anomaly detection for each count column will be performed separately for each device. For example, the test for 
device='android'
on a given day will only compare to previous records where 
device='android'
[[dimensions configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions)] [[all_columns_anomalies test config](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/all-columns-anomalies#test-configuration)]. This is the recommended way to ensure anomalies are detected within each segment of your data, not across different device types.
A message from kapa.ai
Thank you for pointing this out. According to the most recent documentation and changelogs, the 
dimensions
parameter is now supported in the 
all_columns_anomalies
test. This was a feature request in the past, but it has since been implemented, as confirmed by both the [closed GitHub issue](https://github.com/elementary-data/elementary/issues/1127) and the [official documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions). However, if you are seeing the error: > macro 'dbt_macro__test_all_columns_anomalies' takes no keyword argument 'dimensions' this likely means that your installed version of the Elementary dbt package does not yet support the 
dimensions
parameter for 
all_columns_anomalies
. This feature was added in a recent release, so you will need to upgrade your Elementary package to the latest version. What to do: 1. Check your Elementary package version Make sure you are using a version that supports 
dimensions
in 
all_columns_anomalies
. The feature was confirmed as implemented in May 2024 [[GitHub issue 1127](https://github.com/elementary-data/elementary/issues/1127)]. 2. Upgrade the package Update your 
packages.yml
to the latest version and run `dbt deps`:
Copy code
yaml    packages:      - package: elementary-data/elementary        version: [">=0.12.0"]
Then run:
Copy code
dbt deps
3. Try again After upgrading, the following configuration should work:
Copy code
yaml    models:      - name: your_table_name        config:          elementary:            timestamp_column: day_pt        tests:          - elementary.all_columns_anomalies:              column_anomalies:                - standard_deviation              dimensions:                - device              time_bucket:                period: day                count: 1              anomaly_sensitivity: 1
Summary: The error you are seeing is due to an outdated Elementary package. Upgrade to the latest version to use the 
dimensions
parameter with 
all_columns_anomalies
[[dimensions documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions)]. If you continue to have issues after upgrading, please double-check the package version and consult the [changelog](https://github.com/elementary-data/elementary/issues/1127) for confirmation of feature availability.
A message from kapa.ai
I'm sorry, but the knowledge sources provided do not contain the full source code for 
test_all_columns_anomalies.py
. While there are references to the implementation and location of this file in the Elementary dbt package repository, the actual code content is not included in the documents above. If you need the source code, you may want to check the official [elementary-data/dbt-data-reliability GitHub repository](https://github.com/elementary-data/dbt-data-reliability) directly, as referenced in the knowledge sources, or consult your forked version for the specific file [[Github repo reference](https://github.com/elementary-data/dbt-data-reliability)][[Slack answer](https://slack.com/archives/C02CTC89LAX/p1704969119592479)].
A message from kapa.ai
Yes, 
dimension_anomalies
is another option, but it is important to understand what it does and how it differs from 
all_columns_anomalies
or 
column_anomalies
. The 
elementary.dimension_anomalies
test monitors the frequency (row count) of values in the configured dimension(s) over time and alerts on unexpected changes in the distribution. It is best suited for detecting drops or spikes in the number of rows for specific dimension values (e.g., number of events per device per day), not for monitoring metrics like sum, mean, or standard deviation of a metric column within each dimension [[dimension_anomalies documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies)]. For example, if you have a table with columns 
day_pt
, 
device
, and 
event_count
, 
dimension_anomalies
will alert if the number of rows for a given device on a given day changes unexpectedly, but it will not alert if the value of 
event_count
is anomalous for a device on a given day. If your goal is to monitor the actual values in the 
event_count
columns (e.g., detect if the count for 
device='android'
is unusually high or low compared to historical values for 
android
), you should use 
column_anomalies
or 
all_columns_anomalies
with the 
dimensions
parameter (if supported in your version) [[Slack clarification](https://slack.com/archives/C02CTC89LAX/p1682676629448539)]. Summary: - Use 
dimension_anomalies
to detect anomalies in row counts per dimension. - Use 
column_anomalies
or 
all_columns_anomalies
(with 
dimensions
) to detect anomalies in metric values (like event counts) per dimension. If you need to monitor the metric values themselves (not just row counts), 
dimension_anomalies
is not the right tool.
A message from kapa.ai
You need to add the following flag to your `dbt_project.yml`:
Copy code
yaml flags:   require_explicit_package_overrides_for_builtin_materializations: false
when you are using dbt version 1.8 or above and want Elementary to function fully—specifically, to allow Elementary to override dbt’s built-in materializations. This override is required for Elementary to collect failed row samples and failed row counts for dbt tests, including native dbt tests and expectations. Without this flag, these features will not work as intended in dbt 1.8+ [[Elementary troubleshooting](https://docs.elementary-data.com/oss/general/troubleshooting)][[Quickstart: Install Elementary dbt package](https://docs.elementary-data.com/oss/quickstart/quickstart-cli-package#step-by-step-install-elementary-dbt-package)]. - If you are using dbt 1.7 or below, you do not need to add this flag, but you may see a warning about materialization overrides, which can be ignored for now. - If you are using dbt 1.8 or above and want Elementary to collect test result samples and failed row counts, you must add this flag. This is a temporary measure while the dbt and Elementary teams work on a more permanent solution [[Elementary troubleshooting](https://docs.elementary-data.com/oss/general/troubleshooting)].
A message from kapa.ai
When you add the flag
Copy code
yaml flags:   require_explicit_package_overrides_for_builtin_materializations: false
to your 
dbt_project.yml
(and you are using dbt 1.8 or above), you are explicitly allowing Elementary to override dbt's built-in materializations. This enables Elementary to collect failed row samples and failed row counts for dbt tests, which are essential for its anomaly detection and reporting features. After adding the flag, you may still see a deprecation warning from dbt, but Elementary will work as expected. This is a temporary measure until a more permanent solution is implemented in dbt. The warning can be ignored for now [[Elementary troubleshooting](https://docs.elementary-data.com/oss/general/troubleshooting)]. If you do not add this flag, Elementary will not be able to collect these test results, and some features may not work as intended in dbt 1.8+ [[Quickstart: Install Elementary dbt package](https://docs.elementary-data.com/oss/quickstart/quickstart-cli-package#step-by-step-install-elementary-dbt-package)].
A message from kapa.ai
You can find the actual anomaly scores for each test performed in the following locations: - elementary_test_results: This table contains the results for all tests, including anomaly tests. It will have information about which tests passed or failed, and some metadata about the test runs [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1663047264555179)]. - test_result_rows: This table contains detailed data for the anomaly tests, including the metrics and anomaly scores used to generate the graphs and results in the Elementary report. This is where you can inspect the time series and anomaly scores for each test [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1692873553144229)][[Slack answer](https://slack.com/archives/C02CTC89LAX/p1704824907817899)]. - metrics_anomaly_score: This view (or table, depending on your setup) contains the anomaly scores and related statistics for each metric and test. If you want to see the actual anomaly score, threshold, metric value, and other statistics, this is the most direct place to look. If it is empty, check that your tests are configured with a timestamp column and that there is enough historical data for anomaly detection [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1666376716952649)]. Example query:
Copy code
sql SELECT * FROM your_elementary_schema.metrics_anomaly_score WHERE full_table_name = 'DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND' ORDER BY bucket_start DESC;
If you do not see data in 
metrics_anomaly_score
, check 
test_result_rows
for the detailed results, as this table is also referenced as containing the data used for anomaly graphs and scores [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1692873553144229)]. If you still do not see the expected data, ensure your test configuration includes a 
timestamp_column
and that there is sufficient historical data for the anomaly detection to work [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1666376716952649)].
A message from kapa.ai
Based on the knowledge sources, there is evidence that 
metrics_anomaly_score
is used to store anomaly scores for volume anomalies, and users are able to see their results there. However, for 
all_columns_anomalies
, there have been reports that the expected metrics (such as 
metric_name
, 
metric_value
, and anomaly scores) do not always appear in 
data_monitoring_metrics
or 
metrics_anomaly_score
as they do for other tests. In some cases, users have noted that results for column anomaly tests are present in 
elementary_test_results
but not in 
data_monitoring_metrics
or 
metrics_anomaly_score
[[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. Trusted support staff have confirmed that 
elementary_test_results
and 
data_monitoring_metrics
are the main tables for storing test results and metrics, but there is no explicit documentation in the knowledge sources that guarantees 
all_columns_anomalies
results will always be written to 
metrics_anomaly_score
in the same way as 
volume_anomalies
. In fact, there are user reports indicating that this is not always the case, and that the linkage between these tables is not direct for all test types [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. In summary: Yes, there appears to be different behavior in where results are written for 
volume_anomalies
versus 
all_columns_anomalies
. While 
metrics_anomaly_score
reliably contains results for volume anomalies, it may not always contain results for all columns anomalies, which might instead be found in 
elementary_test_results
. The knowledge sources do not provide a definitive technical explanation for this difference, but user experience and support responses confirm the behavior you are seeing.
A message from kapa.ai
To see which specific events failed in your 
all_columns_anomalies
test, along with their event counts and anomaly scores, you should query the tables where Elementary stores test results and anomaly metrics. However, based on the knowledge sources, there is some inconsistency in where these results are stored for 
all_columns_anomalies
compared to other tests like 
volume_anomalies
. Recommended approach: 1. Check 
elementary_test_results
 This table contains the results for all tests, including anomaly tests. You can filter for failed tests and inspect the 
test_results_query
column, which contains the SQL used to identify failed rows. For example:
Copy code
sql    SELECT *    FROM your_elementary_schema.elementary_test_results    WHERE status = 'fail'      AND test_name LIKE '%all_columns_anomalies%'
You can use the 
test_results_query
from the result to see the specific rows (events) that failed the anomaly test, and it may include the event counts and anomaly scores [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1658915396893639)]. 2. Check 
data_monitoring_metrics
 This table may contain the raw metrics for the tests, including metric values and possibly anomaly scores. You can try:
Copy code
sql    SELECT *    FROM your_elementary_schema.data_monitoring_metrics    WHERE full_table_name = 'YOUR_TABLE_NAME'      AND metric_name = 'standard_deviation' -- or the metric you are monitoring    ORDER BY bucket_end DESC
This may show you the metric values and anomaly scores for each event and time bucket [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1685717983788969)]. 3. Note on 
metrics_anomaly_score
 While 
metrics_anomaly_score
reliably contains results for 
volume_anomalies
, it may not always contain results for 
all_columns_anomalies
due to differences in how results are written for these tests [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. Summary: - Query 
elementary_test_results
for failed 
all_columns_anomalies
tests and use the 
test_results_query
to inspect failed events and their counts. - Optionally, check 
data_monitoring_metrics
for metric values and anomaly scores. - There is no guarantee that 
metrics_anomaly_score
will have the results for 
all_columns_anomalies
. If you do not find the expected details in these tables, it may be due to the current limitations or differences in how Elementary stores results for different test types.
A message from kapa.ai
Elementary’s anomaly detection tests, such as 
volume_anomalies
and 
column_anomalies
, operate by aggregating data over the specified training and detection periods. For large raw tables, this can result in heavy queries, especially if the aggregation is not incremental or if the table is not partitioned/clustered on the relevant columns. This is a known performance concern, and there are user reports and feature requests about this issue [[GitHub Issue 1329](https://github.com/elementary-data/elementary/issues/1329)]. Currently, the knowledge sources indicate that the recommended approach for handling large datasets is to create pre-aggregated tables (e.g., daily event counts) and then run Elementary’s anomaly detection tests on these smaller, aggregated tables. This approach significantly improves performance because the anomaly detection queries operate on much less data [[Slack discussion](https://slack.com/archives/C02CTC89LAX/p1703054856878699)]. There is no mention in the knowledge sources of a native, built-in way in Elementary to perform these aggregations incrementally on the raw data within the anomaly detection tests themselves. Users who need better performance are advised to pre-aggregate their data before running the tests. Summary: - For large raw tables, it is recommended to create aggregated tables (e.g., daily counts) and run anomaly detection on those. - There is no native Elementary feature to perform incremental aggregation directly on raw data within the anomaly tests at this time [[GitHub Issue 1329](https://github.com/elementary-data/elementary/issues/1329)][[Slack discussion](https://slack.com/archives/C02CTC89LAX/p1703054856878699)]. If you have further performance needs, you may consider contributing to or following the related feature requests in the Elementary GitHub repository.
A message from kapa.ai
Your configuration error:
Copy code
Compilation Error in test elementary_column_anomalies_xau_event_counts_standard_deviation__voice_channel_effect_sent_count__day__30__device_platform__day__day__30 (models/core/user_experience/users/xau_event_counts.yml) 'dict object' has no attribute 'count'
is caused by the way you are specifying the 
time_bucket
parameter in your 
elementary.column_anomalies
test. In your config, you have:
Copy code
yaml time_bucket:   period: day
But according to the knowledge sources, the 
time_bucket
configuration requires both a 
period
and a 
count
key, for example:
Copy code
yaml time_bucket:   period: day   count: 1
The error message is specifically telling you that the macro is looking for a 
count
attribute in the 
time_bucket
dictionary, but it is missing [[column_anomalies test configuration](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies#test-configuration)]. How to fix: Update your test configuration for each 
elementary.column_anomalies
test to include both 
period
and 
count
under 
time_bucket
, like this:
Copy code
yaml time_bucket:   period: day   count: 1
Apply this change to all your 
elementary.column_anomalies
test blocks. Example:
Copy code
yaml - elementary.column_anomalies:     training_period:       period: day       count: 30     detection_period:       period: day       count: 30     column_anomalies:       - standard_deviation     dimensions:       - device_platform     time_bucket:       period: day       count: 1     config:       severity: warn     tags: ["skip_tests_during_backfills"]
This should resolve the compilation error you are seeing.
A message from kapa.ai
Based on the knowledge sources, there is evidence that results for 
column_anomalies
(and by extension, 
all_columns_anomalies
) do not always appear in the 
metrics_anomaly_score
table or view. Multiple users have reported that while 
metrics_anomaly_score
reliably contains results for tests like 
volume_anomalies
, it may not contain results for 
column_anomalies
. Instead, results for 
column_anomalies
are often found in the 
elementary_test_results
table, and not in 
metrics_anomaly_score
or 
data_monitoring_metrics
as one might expect for other test types. This has been confirmed by both user experience and support responses in the knowledge sources [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. So, to answer your question: No, 
column_anomalies
does not consistently write results to 
metrics_anomaly_score
. You should check 
elementary_test_results
for the outcomes of these tests.
A message from kapa.ai
Your 
event_timestamp_column
is showing as 
null
because, in Elementary's configuration and anomaly detection tests, the 
event_timestamp_column
is a specific parameter used for freshness anomaly tests (such as 
event_freshness_anomalies
), not for general anomaly detection tests like 
column_anomalies
. In your provided dbt config, you are using 
column_anomalies
and specifying 
timestamp_column: day_pt
, which is the correct parameter for time-bucketed anomaly detection on event counts. The 
event_timestamp_column
will remain 
null
unless you are running a freshness anomaly test and explicitly set it. For your use case (anomaly detection on event counts by day, device, and event name), you only need to set 
timestamp_column
, which you have done correctly. The output you are seeing is expected and does not indicate a misconfiguration for your current test type [[timestamp_column documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/timestamp-column)]. If you were running a freshness anomaly test, you would use the 
event_timestamp_column
parameter. For 
column_anomalies
, it is not required or used, so it defaults to 
null
in the test metadata. Summary: - 
event_timestamp_column
is not relevant for 
column_anomalies
tests, so it is 
null
. - Your use of 
timestamp_column: day_pt
is correct for your anomaly detection configuration.
A message from kapa.ai
To see all failed column anomalies, you should query the 
elementary_test_results
table and filter for tests related to column anomalies with a failed status. Here’s an example SQL query:
Copy code
sql SELECT * FROM your_elementary_schema.elementary_test_results WHERE status = 'fail'   AND test_name LIKE '%column_anomalies%'
This will return all failed column anomaly tests, including metadata about the test, the affected column, and other relevant details. The 
elementary_test_results
table is the primary place where Elementary stores the results of all tests, including column anomalies [[dbt package models](https://docs.elementary-data.com/dbt/package-models#run-results)]. If you want to see more details about the failed rows themselves, you can join with the 
test_result_rows
table using the test result ID:
Copy code
sql SELECT r.*, t.result_row FROM your_elementary_schema.elementary_test_results r LEFT JOIN your_elementary_schema.test_result_rows t   ON r.id = t.elementary_test_results_id WHERE r.status = 'fail'   AND r.test_name LIKE '%column_anomalies%'
This will give you both the test metadata and a sample of the failed rows, if available [[dbt package models](https://docs.elementary-data.com/dbt/package-models#run-results)].
k
@kapa.ai what is the name of the table that has table name, column name, metric name, latest value, metric average, and thresholds for different standard deviations
k
A message from kapa.ai
The table that contains the table name, column name, metric name, latest value, metric average, and thresholds for different standard deviations is typically called 
metrics_anomaly_score
in the Elementary schema. This table (or view) includes fields such as 
full_table_name
, 
column_name
, 
metric_name
, 
metric_value
(latest value), 
training_avg
(metric average), 
min_metric_value
, and 
max_metric_value
(thresholds for standard deviations) [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1666376716952649)]. However, please note that not all test types consistently write their results to 
metrics_anomaly_score
. For some tests, such as 
column_anomalies
, results may instead be found in 
elementary_test_results
or other tables, depending on your configuration and Elementary version [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)].
k
@kapa.ai models: - name: xau_event_counts description: Fill in description for xau_event_counts meta: version: '0.1.0' data_purposes: [] columns: - name: day_pt data_type: DATETIME description: day_pt meta: timestamp_column: true - name: event_name data_type: STRING description: event_name - name: device_platform data_type: STRING description: device_platform - name: event_count data_type: INT64 description: event_count tests: - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - standard_deviation dimensions: - device_platform - event_name time_bucket: period: day count: 1 config: severity: warn - name: event_count_unique_users data_type: INT64 description: event_count_unique_users tests: - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - standard_deviation dimensions: - device_platform - event_name time_bucket: period: day count: 30 config: severity: warn tests: - empty_partition - duplicate_check config: unique_key: - day_pt - event_name - device_platform What is wrong with my config? I want to compare the most recent day_pt value to the previous 30 days, but from the metric_anamoly_score table, it looks like it is bucketing data into days of 30. I want to test each 1 day and see if any day has a bad value
k
A message from kapa.ai
The issue in your config is with the 
time_bucket
parameter for the 
event_count_unique_users
column. You have:
Copy code
yaml time_bucket:   period: day   count: 30
This configuration tells Elementary to aggregate data into buckets of 30 days each, so each metric value represents a 30-day period, not a single day. That’s why in your 
metrics_anomaly_score
table, you see data bucketed into 30-day intervals. To compare each single day (
day_pt
) to the previous 30 days, you should set:
Copy code
yaml time_bucket:   period: day   count: 1
This will create daily buckets, so each metric value is for one day, and the anomaly detection will compare each day to the previous 30 days (as set in your 
training_period
). Corrected config for `event_count_unique_users`:
Copy code
yaml - name: event_count_unique_users   data_type: INT64   description: event_count_unique_users   tests:     - elementary.column_anomalies:         timestamp_column: day_pt         training_period:           period: day           count: 30         detection_period:           period: day           count: 30         column_anomalies:           - standard_deviation         dimensions:           - device_platform           - event_name         time_bucket:           period: day           count: 1         config:           severity: warn
This will ensure that each day is tested individually against the previous 30 days, which matches your intent [[time_bucket documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/time-bucket)].
k
@kapa.ai i have records that are TRUE in metrics_anomaly_score but the run that inserted those records did not give a warning, even though I have severity set to warn (from config in my last message). Why didn't dbt raise an alert? 214109 1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN] 214109 1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.27s] 214109 214109 1 of 5 START sql incremental model dbt_testing.xau_event_counts__kyle_mcleland . [RUN] 214332 1 of 5 OK created sql incremental model dbt_testing.xau_event_counts__kyle_mcleland [CREATE TABLE (2.7k rows, 7.7 TiB processed) in 143.93s] 214332 2 of 5 START test duplicate_check_xau_event_counts_ ............................ [RUN] 214332 3 of 5 START test elementary_column_anomalies_xau_event_counts_standard_deviation__event_count__day__30__device_platform__event_name__day__1__day_pt__day__30 [RUN] 214332 4 of 5 START test elementary_column_anomalies_xau_event_counts_standard_deviation__event_count_unique_users__day__30__device_platform__event_name__day__30__day_pt__day__30 [RUN] 214332 5 of 5 START test empty_partition_xau_event_counts_ ............................ [RUN] 214333 214333 + xau_event_counts 214333 [WARNING]: Installed package 'elementary' is overriding the built-in materialization 'test'. Overrides of built-in materializations from installed packages will be deprecated in future versions of dbt. For more information: https://docs.getdbt.com/reference/global-configs/legacy-behaviors 214334 2 of 5 PASS duplicate_check_xau_event_counts_ .................................. [PASS in 1.49s] 214334 Elementary: Started running data monitors on: DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count_unique_users 214334 Elementary: Started running data monitors on: DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count 214336 5 of 5 PASS empty_partition_xau_event_counts_ .................................. [PASS in 3.04s] 214340 Elementary: Finished running data monitors on: DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count_unique_users 214340 Elementary: Finished running data monitors on: DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count 214341 Elementary: Not enough data to calculate anomaly scores on 
test.transformation_v2.elementary_column_anomalies_xau_event_counts_standard_deviation__event_count_unique_users__day__30__device_platform__event_name__day__30__day_pt__day__30.6125b25d54
214342 4 of 5 PASS elementary_column_anomalies_xau_event_counts_standard_deviation__event_count_unique_users__day__30__device_platform__event_name__day__30__day_pt__day__30 [PASS in 9.64s] 214342 3 of 5 PASS elementary_column_anomalies_xau_event_counts_standard_deviation__event_count__day__30__device_platform__event_name__day__1__day_pt__day__30 [PASS in 9.77s] 214342 214353 1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN] 214353 1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 11.05s] 214353 214353 Finished running 1 incremental model, 2 project hooks, 4 data tests in 0 hours 2 minutes and 49.33 seconds (169.33s). 214403 214403 Completed successfully 214403 214403 Done. PASS=7 WARN=0 ERROR=0 SKIP=0 TOTAL=7
k
A message from kapa.ai
Based on the knowledge sources, there are a few reasons why you might see records in 
metrics_anomaly_score
that indicate anomalies (i.e., TRUE for anomaly detection), but your dbt test run does not raise a warning—even though you have set 
severity: warn
in your config. Possible reasons: 1. Not Enough Data for Detection: The log line:
Copy code
Elementary: Not enough data to calculate anomaly scores on ...
indicates that, for at least some partitions, there was insufficient historical data to calculate an anomaly score. Elementary requires a minimum amount of historical data (as defined by your 
training_period
) to determine if a value is anomalous. If this threshold is not met, the test will not fail or warn, even if the current value is unusual [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1666376716952649)]. 2. Test Status vs. Anomaly Score: The presence of a record in 
metrics_anomaly_score
with an anomaly does not always mean the dbt test will fail or warn. The dbt test status is determined by the test logic and configuration at runtime. If the anomaly score is calculated but does not cross the configured threshold (e.g., not enough standard deviations away from the mean), the test will pass, but the metric will still be recorded in 
metrics_anomaly_score
for observability and reporting purposes [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. 3. Severity Configuration: You have set 
severity: warn
, which should cause dbt to raise a warning if the test fails. However, if the test does not actually "fail" (i.e., the anomaly is not considered significant enough by the test logic), no warning will be raised, even if the metric is flagged as anomalous in the metrics table. 4. Test Output Table Differences: There are known differences in how Elementary writes results for different test types. Sometimes, the anomaly is recorded in 
metrics_anomaly_score
, but the dbt test itself does not reflect a warning or error due to the above reasons [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. Summary: - The most likely reason is that there was not enough historical data for the test to trigger a warning, even though the metric was recorded. - Alternatively, the anomaly detected did not meet the threshold to trigger a warning based on your configuration. - This behavior is consistent with how Elementary and dbt tests interact, as described in the knowledge sources. If you want to ensure warnings are raised, double-check your 
training_period
, 
detection_period
, and the amount of historical data available, and confirm that your thresholds are set as intended.
k
@kapa.ai When do a dbt test, how can I tell the dbt test to run for a specific date? it looks like it's doing a bucket of 4/25 - 5/25 even though im dong a dbt build for historical dates
k
A message from kapa.ai
Elementary’s anomaly detection tests use the 
time_bucket
and 
detection_period
configurations to determine which data is included in each test run. By default, the test will aggregate data into buckets (e.g., daily) and then compare the most recent bucket(s) as defined by 
detection_period
against the historical training period. There is no direct way in the knowledge sources to tell a dbt test to run for a specific date (e.g., only for 
2024-05-01
) via a dbt command-line argument or test config. The test will always operate on the most recent completed bucket(s) according to your 
timestamp_column
and 
time_bucket
settings. For example, if your 
detection_period
is set to 1 day, it will test the most recent day in your data, not a user-specified date [[time_bucket documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/time-bucket)]. If you are running a dbt build for historical dates, but the test is still aggregating over a range (e.g., 4/25–5/25), it is because the test is designed to operate on the latest available data in the table, not on a specific date you are building. Summary: - Elementary anomaly tests do not support running for a user-specified date; they always operate on the most recent completed time bucket(s) as defined by your configuration. - If you need to test a specific date, you would need to filter your data/model to only include that date before running the test, but this is not natively supported as a test parameter in Elementary or dbt. If you need more granular control, you may need to adjust your model or data to only include the date of interest before running the test.
k
@kapa.ai max date in my test table is 5/8, but the bucket stat and bucket end are 4/25-5/25
k
A message from kapa.ai
Based on the knowledge sources, the reason you are seeing a bucket start and end of 4/25–5/25, even though the max date in your test table is 5/8, is due to how Elementary calculates time buckets for anomaly detection tests. Elementary splits your data into time buckets based on the 
time_bucket
configuration (e.g., daily, weekly, monthly) and determines the range of buckets to analyze using the latest available data and the 
detection_period
and 
training_period
settings. The buckets are not always aligned to the max date in your table; instead, they are calculated to ensure full buckets and to cover the configured periods. For example, if you have a monthly bucket (
period: month, count: 1
), the bucket might span from 4/25 to 5/25, even if your latest data is only up to 5/8. This is because Elementary ensures that buckets are complete and may extend the end date to the next full period boundary [[time_bucket documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/time-bucket)]. Additionally, if your data does not fill the entire bucket (e.g., no data after 5/8), the bucket will still be created for the full period, but the metrics for dates after your max date will be zero or null. Summary: - Buckets are determined by your 
time_bucket
configuration and may not align exactly with your max data date. - Elementary creates full buckets based on period boundaries, not just the data's max date. - If your data ends before the bucket end, the bucket still spans the full configured period. If you want buckets to align more closely with your data, consider adjusting your 
time_bucket
settings or ensuring your data covers the full period of interest.
k
@kapa.ai here is my config: models: - name: xau_event_counts description: Fill in description for xau_event_counts meta: version: '0.1.0' data_purposes: [] columns: - name: day_pt data_type: DATETIME description: day_pt meta: timestamp_column: true - name: event_name data_type: STRING description: event_name - name: device_platform data_type: STRING description: device_platform - name: event_count data_type: INT64 description: event_count tests: - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - standard_deviation dimensions: - device_platform - event_name time_bucket: period: day count: 1 config: severity: warn - name: event_count_unique_users data_type: INT64 description: event_count_unique_users tests: - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - standard_deviation dimensions: - device_platform - event_name time_bucket: period: day count: 1 config: severity: warn tests: - empty_partition - duplicate_check config: unique_key: - day_pt - event_name - device_platform here is a sample export: Row id full_table_name column_name metric_name metric_value source_value bucket_start bucket_end bucket_duration_hours updated_at dimension dimension_value metric_properties created_at metric_type 1 2ad4bbd39b47b661464b020a6ad6ea31 DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count_unique_users standard_deviation 0.0 null 2025-04-25 070000 UTC 2025-05-25 070000 UTC 720 2025-05-25 214335.176782 UTC device_platform; event_name console; voice_channel_effect_sent {"dimensions": ["device_platform", "event_name"], "event_timestamp_column": null, "freshness_column": null, "time_bucket": {"count": 30, "period": "day"}, "timestamp_column": "day_pt", "where_expression": null} 2025-05-25 214346.937717 UTC standard_deviation 2 9ade2f7a43e42cd27e61d09618397533 DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count_unique_users standard_deviation 0.0 null 2025-04-25 070000 UTC 2025-05-25 070000 UTC 720 2025-05-25 214335.176782 UTC device_platform; event_name console; ack_messages {"dimensions": ["device_platform", "event_name"], "event_timestamp_column": null, "freshness_column": null, "time_bucket": {"count": 30, "period": "day"}, "timestamp_column": "day_pt", "where_expression": null} 2025-05-25 214346.937717 UTC standard_deviation 3 c71f1e44178625259242dc399d73249e DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count_unique_users standard_deviation 0.0 null 2025-04-25 070000 UTC 2025-05-25 070000 UTC 720 2025-05-25 214335.176782 UTC device_platform; event_name console; app_opened {"dimensions": ["device_platform", "event_name"], "event_timestamp_column": null, "freshness_column": null, "time_bucket": {"count": 30, "period": "day"}, "timestamp_column": "day_pt", "where_expression": null} 2025-05-25 214346.937717 UTC standard_deviation 4 220a75b3581b81ec5dfd22ab81ca9e22 DISCORD-DATA-VAULT-PRD.DBT_TESTING.XAU_EVENT_COUNTS__KYLE_MCLELAND event_count_unique_users standard_deviation 0.0 null 2025-04-25 070000 UTC 2025-05-25 070000 UTC 720 2025-05-25 214335.176782 UTC device_platform; event_name console; friends_list_viewed {"dimensions": ["device_platform", "event_name"], "event_timestamp_column": null, "freshness_column": null, "time_bucket": {"count": 30, "period": "day"}, "timestamp_column": "day_pt", "where_expression": null} 2025-05-25 214346.937717 UTC standard_deviation As you can see, it's bucketing from 4/25-5/25. I want to test the standard deviation of records where day_pt = '2025-05-08'. how should i updaet my config and hwat dbt commands should i run
k
A message from kapa.ai
Based on your config and the sample export, the reason you see buckets spanning 4/25–5/25 is because the 
time_bucket
in your metrics is set to 
{"count": 30, "period": "day"}
. This means Elementary is aggregating 30 days into a single bucket, so the standard deviation is calculated over the entire 30-day period, not for each individual day. How to test for a specific day (e.g., 
day_pt = '2025-05-08'
): 1. Update your config: Set your 
time_bucket
to daily by using `count: 1`:
Copy code
yaml    time_bucket:      period: day      count: 1
This will ensure that each bucket represents a single day, so the anomaly detection will be performed for each day individually, including 
2025-05-08
. 2. No dbt command to target a specific date: There is no built-in dbt or Elementary feature to run the anomaly test for only a specific date. The test will always operate on the most recent (or all available) daily buckets, as defined by your configuration. If you want to test only a specific date, you would need to filter your model to only include that date before running the test, but this is not natively supported as a test parameter in Elementary or dbt [[time_bucket documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/time-bucket)]. 3. Run your dbt tests as usual: After updating your config, run:
Copy code
dbt build --select xau_event_counts
or
Copy code
dbt test --select xau_event_counts
This will run the anomaly detection for each day in your data, including 
2025-05-08
, and you will see daily buckets in your results. Summary: - Set 
time_bucket: {period: day, count: 1}
to get daily anomaly detection. - There is no way to run the test for only a specific date via dbt/Elementary config; tests always run for all available buckets. - Run your dbt tests as usual; the results will now include a bucket for each day, including the date you care about. If you need to focus on a single date, you can filter your model to only include that date before running the test, but this is a manual workaround, not a built-in feature.
k
@kapa.ai from my config you can see my time bucket is configured as: time_bucket: period: day count: 1 so why am i getting buckets of 4/25-5/25?
k
A message from kapa.ai
Given your configuration:
Copy code
yaml time_bucket:   period: day   count: 1
you should be getting daily buckets (i.e., each bucket should represent a single day). However, your sample export from 
metrics_anomaly_score
shows buckets spanning from 4/25 to 5/25 (a 30-day window), which does not match your configuration. Possible reasons based on the knowledge sources: 1. Mismatch Between Config and Output: If your config is set to 
count: 1
but the output shows a 30-day bucket, it suggests that either: - The config being used by Elementary at runtime is not the one you expect (e.g., a cached or older version is being used). - There may be a bug or unexpected behavior in how Elementary is reading or applying the 
time_bucket
parameter for this test type. 2. Test Type Differences: There are known inconsistencies in how different test types write results and interpret configuration parameters, especially for 
column_anomalies
and 
all_columns_anomalies
. Some users have reported that the metrics tables do not always reflect the test configuration as expected [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)]. 3. Metric Properties in Output: Your sample export’s 
metric_properties
field shows 
"time_bucket": {"count": 30, "period": "day"}
even though your config is set to 
count: 1
. This suggests that the test is actually running with a 30-day bucket, not a 1-day bucket. This could be due to a misconfiguration, a default value being applied elsewhere, or a bug. What you should do: - Double-check that your dbt project is using the correct and most up-to-date config file. - Make sure there are no overrides or conflicting configurations elsewhere in your project. - If you are certain your config is correct and up-to-date, this may be a bug or an issue with how Elementary is parsing the config for this test type. Summary: Even though your config specifies a daily bucket, the output and the 
metric_properties
field indicate that a 30-day bucket is being used. This points to either a config mismatch, a bug, or an unexpected default being applied. The knowledge sources do not provide a definitive technical explanation for this discrepancy, so if the issue persists after verifying your config, it is recommended to reach out to Elementary support or check for open issues related to this behavior [[Slack thread](https://slack.com/archives/C02CTC89LAX/p1737377522035139)].
k
@kapa.ai if configuted correctly, does it always output results to data_monitoring_metrics? I just re-ran the test and nothing is being written to that table. Is something wrong with my config I pasted above
k
A message from kapa.ai
If your Elementary anomaly detection test is configured correctly and is running, it should write results to the 
data_monitoring_metrics
table. This table is specifically intended for metrics collected via Elementary's anomaly detection tests, such as 
column_anomalies
and 
volume_anomalies
[[Slack answer](https://slack.com/archives/C02CTC89LAX/p1688373015213789)]. However, there are a few scenarios where you might not see data in `data_monitoring_metrics`: - Test Not Actually Running: If the test is not being picked up by dbt (e.g., due to a config or selection issue), no results will be written. - Not Enough Data: If there is not enough historical data to calculate anomaly scores (as required by your 
training_period
), Elementary may not write results for those periods [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1681298359181089)]. - Materialization or Hook Issues: If the Elementary models are not materialized as tables, or if the on-run-end hook is not firing correctly, data may not be loaded into the metrics tables [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1718816885919229)]. - Custom Macros or Project Configs: If your project has custom macros that alter table names or materializations, this can prevent Elementary from writing to the expected tables [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1681849464180719)]. If you have verified that your config matches the one you pasted above and you are still not seeing results in 
data_monitoring_metrics
, it is possible that one of these issues is affecting your setup. Double-check that: - The test is actually running (check dbt logs for test execution). - There is enough data for the training and detection periods. - The Elementary models are materialized as tables, not views. - No custom macros are interfering with Elementary's table naming or materialization. If all of these are correct and you still see no data, it may be a version-specific bug or an issue with your environment, as there have been user reports of similar issues in the knowledge sources [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1688373015213789)].
k
@kapa.ai does column_anomalies write to data_monitoring_metrics
k
A message from kapa.ai
Yes, according to the knowledge sources, the 
column_anomalies
test is intended to write its results to the 
data_monitoring_metrics
table. This table is specifically mentioned as the destination for metrics collected via Elementary's anomaly detection tests, including 
column_anomalies
[[Slack answer](https://slack.com/archives/C02CTC89LAX/p1688373015213789)]. However, there are user reports indicating that, in some cases, results for 
column_anomalies
may not always appear in 
data_monitoring_metrics
as expected. This could be due to configuration issues, insufficient data, or version-specific behavior. But by design, 
column_anomalies
should write to 
data_monitoring_metrics
when configured and executed correctly [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1688373015213789)].
k
@kapa.ai should column_anomalies test be added to a column specifically or to entire model tests? is this config correct? models: - name: xau_event_counts description: Fill in description for xau_event_counts meta: version: '0.1.0' data_purposes: [] columns: - name: day_pt data_type: DATETIME description: day_pt meta: timestamp_column: true - name: event_name data_type: STRING description: event_name - name: device_platform data_type: STRING description: device_platform - name: event_count data_type: INT64 description: event_count - name: event_count_unique_users data_type: INT64 description: event_count_unique_users tests: - empty_partition - duplicate_check - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - standard_deviation dimensions: - device_platform - event_name time_bucket: period: day count: 1 config: severity: warn config: unique_key: - day_pt - event_name - device_platform
k
A message from kapa.ai
The 
elementary.column_anomalies
test should be added to a specific column, not to the model-level 
tests:
block. According to the knowledge sources, the correct way to configure 
column_anomalies
is under the 
tests:
section of the column you want to monitor. This allows the test to run anomaly detection on that particular column's values [[column_anomalies documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies#test-configuration)]. Your current config is incorrect because you have placed 
elementary.column_anomalies
under the model-level 
tests:
block, which is not supported for this test type. Correct configuration example:
Copy code
yaml models:   - name: xau_event_counts     description: Fill in description for xau_event_counts     meta:       version: '0.1.0'       data_purposes: []     columns:       - name: day_pt         data_type: DATETIME         description: day_pt         meta:           timestamp_column: true       - name: event_name         data_type: STRING         description: event_name       - name: device_platform         data_type: STRING         description: device_platform       - name: event_count         data_type: INT64         description: event_count         tests:           - elementary.column_anomalies:               timestamp_column: day_pt               training_period:                 period: day                 count: 30               detection_period:                 period: day                 count: 30               column_anomalies:                 - standard_deviation               dimensions:                 - device_platform                 - event_name               time_bucket:                 period: day                 count: 1               config:                 severity: warn       - name: event_count_unique_users         data_type: INT64         description: event_count_unique_users         tests:           - elementary.column_anomalies:               timestamp_column: day_pt               training_period:                 period: day                 count: 30               detection_period:                 period: day                 count: 30               column_anomalies:                 - standard_deviation               dimensions:                 - device_platform                 - event_name               time_bucket:                 period: day                 count: 1               config:                 severity: warn     tests:       - empty_partition       - duplicate_check     config:       unique_key:         - day_pt         - event_name         - device_platform
Summary: - 
column_anomalies
must be defined under the specific column’s 
tests:
block, not at the model level. - Your current config is not correct for `column_anomalies`—move the test to the relevant columns as shown above [[column_anomalies documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies#test-configuration)].
k
@kapa.ai im confused about my test results because im testing a table thats 1 record per dimension (event/device) per day, so 91 total records since we have 13 events and 6 dimensions. when i run a test for 5/9 the "training set size" is 1095. how can the training set size be so large when i dont have that much data. if i have a dimensions defined as followed, shouldnt there only be 30 events in the training
Copy code
dimensions:
                - device_platform
                - event_name
k
A message from kapa.ai
Your expectation is correct: if you have a table with 1 record per (event, device) per day, and you define 
dimensions
as 
device_platform
and 
event_name
, then for a single (event, device) combination, the training set for a daily anomaly test with a 30-day training period should only include up to 30 records (one per day for that combination). If you are seeing a "training set size" of 1095, this suggests that the training set is being calculated across all combinations of your dimensions, not just for a single (event, device) pair. For example, if you have 13 events and 6 device platforms, that's 78 combinations. Over 30 days, that would be 78 × 14 = 1092 (if the training set is 14 days) or 78 × 30 = 2340 (if 30 days), so 1095 is in the expected range for the total number of records across all combinations. This behavior is consistent with how Elementary splits the table by the configured dimensions and runs anomaly detection for each combination. The "training set size" you see in the test result is likely the total number of training records across all dimension combinations, not just for the specific (event, device) being tested [[dimension_anomalies logic](https://slack.com/archives/C02CTC89LAX/p1687870015985909)]. If you want to see the training set size for a specific (event, device) pair, you would need to look at the detailed metrics for that combination, not the overall training set size reported for the test as a whole. Summary: - The large training set size reflects the total across all dimension combinations, not per combination. - For a single (event, device) pair, the training set should be up to 30 records (one per day). - This is expected behavior given how Elementary aggregates and reports training set sizes when dimensions are used [[dimension_anomalies logic](https://slack.com/archives/C02CTC89LAX/p1687870015985909)].
k
@kapa.ai i can see that the average is also using the average of the entire table. i want the average to be compared to only the same dimension. for example, i have an "platform=overall" dimension, and i want that dimension to only be compared to other dimension where platform = overall, but the "average" shown on the metrics is definitely the average of all dimensions, not just the overall dimension.
k
A message from kapa.ai
You are correct that, for dimension-based anomaly detection, the average should be calculated within each dimension value (e.g., only for 
platform = overall
, not across all platforms). According to the knowledge sources, when you configure the 
dimensions
parameter in 
column_anomalies
or 
dimension_anomalies
, Elementary is supposed to segment the data by those dimensions and monitor each segment separately. This means the average, standard deviation, and anomaly detection should be performed per dimension value [[dimensions documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions)]. However, there have been user reports and even open issues indicating that, in some cases, the partitioning in the anomaly score calculation was not correctly including the dimension values, resulting in averages being calculated across all data instead of per dimension. For example, a GitHub issue describes that the calculation should be:
Copy code
sql avg(metric_value) over (partition by metric_name, full_table_name, column_name, dimension, dimension_value order by bucket_start asc ...)
but was missing the 
dimension
and 
dimension_value
in the partition, leading to the problem you described [[Github Issue #1729](https://github.com/elementary-data/elementary/issues/1729)]. Summary: - Expected behavior: When using 
dimensions
, averages and anomaly scores should be calculated per dimension value (e.g., only for 
platform = overall
). - Observed issue: There have been cases where the average is incorrectly calculated across all dimensions, not just the one being tested. This is a known issue and has been reported by users [[Github Issue #1729](https://github.com/elementary-data/elementary/issues/1729)]. - What to do: If you are experiencing this, it may be due to a bug or an outdated version of Elementary. Check for updates or follow the linked GitHub issue for progress on a fix. If you are on a recent version and still see this issue, it is likely a bug, and you should consider commenting on or following the open issue for updates.
k
@kapa.ai i think actually im looking for dimension anomaly, can you help modify my config to make it work with dimension anomalies models: - name: xau_event_counts description: Fill in description for xau_event_counts meta: version: '0.1.0' data_purposes: [] columns: - name: day_pt data_type: DATETIME description: day_pt meta: timestamp_column: true - name: event_name data_type: STRING description: event_name - name: device_platform data_type: STRING description: device_platform - name: event_count data_type: INT64 description: event_count tests: - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - sum dimensions: - device_platform - event_name time_bucket: period: day count: 1 config: severity: warn - name: event_count_unique_users data_type: INT64 description: event_count_unique_users tests: - elementary.column_anomalies: timestamp_column: day_pt training_period: period: day count: 30 detection_period: period: day count: 30 column_anomalies: - sum dimensions: - device_platform - event_name time_bucket: period: day count: 1 config: severity: warn tests: - empty_partition - duplicate_check config: unique_key: - day_pt - event_name - device_platform
k
A message from kapa.ai
Certainly! To use the dimension anomaly test instead of 
column_anomalies
, you should use the 
elementary.dimension_anomalies
test at the model level, specifying your desired dimensions. This test will monitor the row count per combination of your specified dimensions (e.g., per 
device_platform
and 
event_name
) over time and alert on unexpected changes in their distribution [[dimension_anomalies documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies)]. Here’s how you can modify your config:
Copy code
yaml models:   - name: xau_event_counts     description: Fill in description for xau_event_counts     meta:       version: '0.1.0'       data_purposes: []     columns:       - name: day_pt         data_type: DATETIME         description: day_pt         meta:           timestamp_column: true       - name: event_name         data_type: STRING         description: event_name       - name: device_platform         data_type: STRING         description: device_platform       - name: event_count         data_type: INT64         description: event_count       - name: event_count_unique_users         data_type: INT64         description: event_count_unique_users     tests:       - empty_partition       - duplicate_check       - elementary.dimension_anomalies:           dimensions:             - device_platform             - event_name           timestamp_column: day_pt           time_bucket:             period: day             count: 1           training_period:             period: day             count: 30           detection_period:             period: day             count: 30           config:             severity: warn     config:       unique_key:         - day_pt         - event_name         - device_platform
Key points: - Place 
elementary.dimension_anomalies
under the model-level 
tests:
block, not under a specific column. - Specify your dimensions under 
dimensions:
. - Use 
timestamp_column
and 
time_bucket
to control the time granularity. - You can also set 
training_period
and 
detection_period
as needed. This configuration will monitor the row count for each 
(device_platform, event_name)
combination per day and alert if there are unexpected changes in their distribution [[dimension_anomalies test config](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies#test-configuration)].
A message from kapa.ai
Yes, the 
dimension_anomalies
test in Elementary only monitors row count per dimension over time. It does not support checking the sum of a column or other aggregate metrics by dimension. This is confirmed in both the official documentation and multiple user discussions: the test is designed to alert on unexpected changes in the distribution of row counts for the configured dimensions, and is best used on low-cardinality fields [[dimension_anomalies documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies)]. If you want to check the sum of a column (e.g., event counts) by dimension, you should use the 
column_anomalies
test with the 
dimensions
parameter. This allows you to monitor aggregate metrics such as the sum, mean, or standard deviation of a column, grouped by your specified dimensions. This feature was added to address exactly this use case, as confirmed by Elementary support and recent updates [[GitHub Issue #1386](https://github.com/elementary-data/elementary/issues/1386)][[GitHub Issue #1253](https://github.com/elementary-data/elementary/issues/1253)]. Summary: - 
dimension_anomalies
= row count only - To check the sum by dimension, use 
column_anomalies
with the 
dimensions
parameter.
A message from kapa.ai
To see the details of your anomalies for the warnings raised by the 
elementary.column_anomalies
tests, you should query the 
elementary_test_results
table in your Elementary schema. This table contains the results of all tests, including which dimension values and columns triggered anomalies. Here’s an example query you can use:
Copy code
sql SELECT * FROM your_elementary_schema.elementary_test_results WHERE status = 'warn'   AND test_name LIKE '%column_anomalies%' ORDER BY detected_at DESC;
This will show you the test runs that resulted in warnings, including metadata about the test, the affected column, and the dimension values. If you want to see more details about the specific rows or metrics that triggered the anomaly, you can also check the 
test_result_rows
table (if available in your setup), which may contain samples of the failed or anomalous rows:
Copy code
sql SELECT r.*, t.result_row FROM your_elementary_schema.elementary_test_results r LEFT JOIN your_elementary_schema.test_result_rows t   ON r.id = t.elementary_test_results_id WHERE r.status = 'warn'   AND r.test_name LIKE '%column_anomalies%' ORDER BY r.detected_at DESC;
These queries will help you identify which dimension values and columns were flagged as anomalous by your tests [[dbt package models](https://docs.elementary-data.com/dbt/package-models#run-results)].
A message from kapa.ai
For incremental models, Elementary handles the training period differently than for regular tables or views: - On the first test run (or on a full refresh), the values for the full 
training_period
are calculated. - On subsequent runs, only the values for the 
detection_period
(i.e., the most recent buckets) are recalculated. The rest of the training data is not recomputed unless there are gaps (e.g., if the test hasn't run for several days, it will backfill the missing days), or unless you force a backfill. This means that for a daily incremental model with a 30-day training period, **the 30 days of training data are not recomputed in full each time**—only the new or changed data (as defined by the detection period) is recalculated on each run. This is an optimization to improve performance on large incremental tables [[training_period documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/training-period)]. If you want to force a full recomputation of the training data, you can use the 
force_metrics_backfill
variable in your dbt command [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1698154501770259)]. Summary: - For incremental models, only the new/changed data in the detection period is recomputed on each run; the rest of the training period is not recalculated unless needed or forced.
A message from kapa.ai
The error 
Missing mandatory configuration: ['backfill_days']
occurs when running an Elementary anomaly detection test and the required 
backfill_days
parameter is not provided. This is most commonly seen in older versions of the Elementary dbt package, or when using a configuration that expects the old parameter names. What to do: - In recent versions of Elementary, the parameters 
backfill_days
and 
days_back
have been replaced by 
detection_period
and 
training_period
, which are more flexible and support units like day, week, and month. - If you are seeing this error, you are likely using an older version of Elementary or a test type that still expects 
backfill_days
. You should either: - Add the 
backfill_days
parameter to your test configuration, or - Upgrade your Elementary dbt package to the latest version and use the new 
detection_period
and 
training_period
parameters instead. For example, the new configuration would look like:
Copy code
yaml detection_period:   period: day   count: 2 training_period:   period: day   count: 14
If you are using a recent version and still see this error, double-check that all your test configs use the new parameter names and that your dbt package is up to date [[Slack answer](https://slack.com/archives/C02CTC89LAX/p1705898668461709)][[Slack answer](https://slack.com/archives/C02CTC89LAX/p1705927938525479)]. Summary: - The error means the test is missing the 
backfill_days
parameter. - Use 
detection_period
and 
training_period
in newer versions. - Upgrade your Elementary package if you want to use the new parameters.
41 Views
Open in Slack
PreviousNext
Powered by