To configure a volume anomaly test for a table that is fully truncated and loaded (with no timestamp or date column), you should use the

elementary.volume_anomalies

test without specifying a

timestamp_column

. In this mode, Elementary will simply take a snapshot of the total row count each time the test runs and compare it to previous snapshots. This is ideal for your use case, as the table is overwritten multiple times a week and you want to monitor for unexpected changes in total row count between loads. Here’s how you can configure it in your

schema.yml

or

models

file:

yaml models:   - name: your_table_name     tests:       - elementary.volume_anomalies

Key points: - No `timestamp_column`: By omitting this, the test will monitor the total row count at each test execution, not split by time buckets. - Training period: By default, Elementary waits for 14 test runs to build a baseline before it starts flagging anomalies. You can adjust this with the

min_training_set_size

parameter if you want to start detecting anomalies sooner, but be aware that a shorter training period may increase false positives. - Test frequency: Run the test as often as your table is loaded (e.g., after each load or daily) to ensure you capture each new state. - Behavior: After enough runs, the test will compare the current row count to the historical row counts and alert if there is a significant deviation. This approach is specifically recommended for truncate-and-load tables without timestamps, as confirmed by Elementary support and documentation: > "If you don’t define a timestamp column for this table, we will simply save the entire row count of the table when the test runs... Once the test runs enough times (for 14 days in a row by default, can be configurable) - we have enough training set and if in the next time the test runs there will be a drop we will alert on it" [Slack discussion on truncate and load tables](https://slack.com/archives/C02CTC89LAX/p1650989426047689) > "You should use the test without a timestamp column. This way each test execution will count the total volume, and after there are enough data points (14 days if daily execution) the test will start comparing the last row count to the previous data points." [Slack: Overwrite tables and volume anomalies](https://slack.com/archives/C02CTC89LAX/p1710327533594939) No additional configuration is required unless you want to adjust sensitivity or training period. This setup will allow you to monitor for unexpected changes in row count after each load, even without a timestamp column.

Great question! For volume anomaly tests without a

timestamp_column

(like your truncate-and-load table), the "training period" refers to the number of test runs, not calendar days. - Each time you run the test, Elementary records the row count as a new data point. - By default, it requires 14 runs (not 14 days) to build the baseline before it starts detecting anomalies. - If you run the test multiple times in a single day, each run counts toward the training set. This is confirmed in the documentation and community responses: > "When using anomaly test without a timestamp column the default days to train on is 14, meaning for accuracy reasons the test will hold for 14 days since it was first configured, and build the metric. ... you can change the

min_training_set_size

to rely on less than 14 days, but this may cause some inaccuracies in the test threshold" [Slack: No timestamp column training period](https://slack.com/archives/C02CTC89LAX/p1692691883697119) > "If you don't have a timestamp configured ... Training period data builds up over multiple test runs, using the test run time as its timestamp column. This requires time to collect enough points; for a 14 day training period, the test would need 14 different runs on different days to have a full training set." [Anomaly Troubleshooting Guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide) So, for non-timestamp tests, it's 14 runs (not 14 days). If you do 14 runs in one day, you'll have a full training set after those 14 runs. For timestamp-based tests, the training period is based on calendar days (e.g., 14 days of historical data). If you need to start detecting anomalies sooner, you can lower the

min_training_set_size

, but this may increase the risk of false positives.

Based on the available knowledge sources, the recommended and supported way to build up the training set for a non-timestamp-based volume anomaly test is to run the test multiple times so that Elementary collects the required number of data points (e.g., 14 runs for the default training period). There is no official documentation or support guidance suggesting that you can or should manually populate the

data_monitoring_metrics

table with made-up data to simulate the training period. The guidance is clear that for non-timestamp tests, "training period data builds up over multiple test runs, using the test run time as its timestamp column. This requires time to collect enough points; for a 14 day training period, the test would need 14 different runs on different days to have a full training set" [Anomaly Troubleshooting Guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide). If you want to test the anomaly detection logic in your local/dev environment, the supported approach is to: 1. Run your dbt test command (e.g.,

dbt test --select elementary.volume_anomalies

) repeatedly—14 times or as many as your configured

min_training_set_size

. 2. After the required number of runs, Elementary will start evaluating for anomalies. There is no mention in the knowledge sources of a supported or recommended way to manually insert rows into the metrics table to simulate test runs. If you attempt to do so, it may not trigger the full anomaly detection logic as expected, and could lead to inconsistent or unsupported behavior. Summary: You should simulate the 14 runs by actually running the test 14 times in your dev environment. Manually populating the metrics table is not documented or supported as a way to simulate the training period for Elementary anomaly detection tests.

For your use case—a truncate-and-load table **without a timestamp column**—you generally do not need to specify

detection_period

,

training_period

,

backfill_days

, or

days_back

(sometimes called

back_days

). Here’s what each of these configurations does, and how they apply (or don’t) to your scenario: --- ### 1. detection_period - Purpose: Defines the recent period (e.g., last 2 days) in which Elementary looks for anomalies. - Usage: Only relevant when you have a

timestamp_column

configured, as it determines which time buckets are considered for anomaly detection. - For your use case: Not needed. Without a timestamp column, the test just compares the latest row count to previous runs, not to specific time periods. [Detection period documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period) --- ### 2. training_period - Purpose: Sets how much historical data is used to calculate the expected range for anomaly detection. - Usage: With a

timestamp_column

, it refers to a number of days/weeks/months. Without a timestamp, it refers to the number of test runs. - For your use case: Optional. By default, Elementary uses 14 runs as the training set. You can adjust this if you want a shorter or longer baseline, but it’s not required. [Slack: Training period for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1692691883697119) --- ### 3. backfill_days - Purpose: Used to recalculate metrics for previous time buckets (when using a timestamp column) or to define the detection period for non-timestamp tests. - Usage: For non-timestamp tests, it only affects the detection period (i.e., how many recent runs are checked for anomalies). - For your use case: Optional. You can use it to control how many recent runs are considered for anomaly detection, but the default is usually sufficient. [Slack: backfill_days for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1688112798629339) --- ### 4. days_back (sometimes called

back_days

) - Purpose: For timestamp-based tests, it controls how far back in time to look for training data. - Usage: For non-timestamp tests, it refers to the number of previous test runs to use as the training set. - For your use case: Optional. You can set this if you want to use fewer or more runs for your baseline, but it’s not required. [Slack: days_back for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1690901726388079) --- Summary for your use case: You do not need to specify any of these configs unless you want to customize the default behavior (e.g., use fewer than 14 runs for training). The test will work out of the box by comparing the row count from each run to the previous runs. If you want to adjust how quickly the test starts detecting anomalies or how sensitive it is, you can set

training_period

or

min_training_set_size

to a lower value. If you need more control over which recent runs are checked for anomalies, you can use

backfill_days

. Otherwise, the defaults are sufficient for a truncate-and-load table without a timestamp column.

Based on the knowledge sources,

training_period

and

min_training_set_size

are related but not exactly the same thing. -

training_period

(for timestamp-based tests) defines the time window (e.g., 14 days) from which Elementary collects data to build the training set for anomaly detection. For non-timestamp tests, it effectively translates to the number of test runs, since each run is treated as a new data point [[Anomaly Troubleshooting Guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)]. -

min_training_set_size

is the minimum number of data points required in the training set before Elementary will start evaluating for anomalies. For non-timestamp tests, this means the test will not start detecting anomalies until at least this many runs have occurred. You can set this to a lower value if you want the test to start detecting anomalies sooner, but this may increase the risk of false positives [[Slack: min_training_set_size for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1692691883697119)]. In summary: - For non-timestamp tests,

training_period

(in days) and

min_training_set_size

(in runs) both control how much history is used, but

min_training_set_size

is the hard minimum number of runs required before anomaly detection starts. - They are closely related, but not always identical—`training_period` is about the window of data considered, while

min_training_set_size

is about the minimum number of data points needed to begin anomaly detection. If you want the test to start detecting anomalies with fewer runs, set

min_training_set_size

to a lower value. If you want to control how much history is used for the baseline, adjust

training_period

(for timestamp-based tests) or the number of runs (for non-timestamp tests).

Based on the knowledge sources, the expected behavior is that Elementary should only calculate an anomaly score once the number of data points meets or exceeds the

min_training_set_size

. In your case, with

min_training_set_size: 3

, the anomaly score should not be calculated until the third run. From the documentation: > "The anomaly score is only calculated if there are enough metric results. Let’s say you have metrics just for 2-3 days, this is not enough to determine what is anomalous and what is normal. If you configure a

timestamp_column

, the first test execution would pick up 14 days back by default so it could detect an anomaly from the first run." [Slack: Anomaly score calculation](https://slack.com/archives/C02CTC89LAX/p1666376716952649) And for non-timestamp tests: > "This occurs when there are fewer than 7 training data points. To resolve: ... For non-timestamp tests: Run your tests multiple times to build up training data. ... The test will need data for at least 7 time buckets (e.g 7 days) to calculate the anomaly." [Anomaly Troubleshooting Guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide) However, there is no explicit mention in the knowledge sources about a scenario where the anomaly score is calculated before reaching the

min_training_set_size

. If you observed that the score was calculated on the 2nd run (with

min_training_set_size: 3

), this could indicate a bug or unexpected behavior in the version of Elementary you are using. Summary: According to the documented logic, the anomaly score should not be calculated until the number of data points meets the

min_training_set_size

. If you see otherwise, it may be a bug, and you may want to check your package version or reach out to Elementary support for clarification. The knowledge sources do not provide an explanation for this specific behavior.

The latest stable version of the Elementary dbt package, according to the official documentation, is 0.16.4:

yaml packages:   - package: elementary-data/elementary     version: 0.16.4

[See official quickstart documentation](https://docs.elementary-data.com/oss/quickstart/quickstart-cli-package#step-by-step:install-elementary-dbt-package) Your current version, 0.19.1, is not mentioned in the provided knowledge sources, and there is no information about versions newer than 0.16.4. Therefore, I cannot confirm the stability or compatibility of 0.19.1 based on the available documentation. Regarding dbt compatibility: Elementary 0.16.4 is compatible with dbt-core 1.8.x, as the documentation provides configuration guidance for dbt 1.8 and above, including required flags for materialization overrides starting from dbt 1.8 [[Quickstart: dbt 1.8+ flags](https://docs.elementary-data.com/oss/quickstart/quickstart-cli-package#step-by-step:install-elementary-dbt-package)]. Summary: - The latest stable Elementary dbt package version in the knowledge sources is 0.16.4. - This version is compatible with dbt 1.8.7. - There is no information in the knowledge sources about version 0.19.1 or its compatibility.

To see the expected range of rows for a volume anomaly (the same range shown in the EDR report UI graph), you should look at the

metrics_anomaly_score

view in your Elementary schema. This view contains the anomaly score calculations, including the expected minimum and maximum values for the metric being monitored. As confirmed in the knowledge sources: > "The graph that's shown is based on

metrics_anomaly_score

, can you check your results there?" [Slack: Column anomalies and expected range](https://slack.com/archives/C02CTC89LAX/p1738760155338339) Additionally, the

metrics_anomaly_score

view is described as: > "This is a view on

data_monitoring_metrics

that runs the same query the anomaly detection tests run to calculate anomaly scores. The purpose of this view is to provide visibility to the results of anomaly detection tests." [Elementary dbt package models](https://docs.elementary-data.com/dbt/package-models#anomaly-detection) So, query the

metrics_anomaly_score

view to see the expected range (min and max) for row counts used in volume anomaly detection.

The

training_set_size

column in the

metrics_anomaly_score

table represents the number of data points used to calculate the expected range (average, standard deviation, etc.) for anomaly detection. In other words, it shows how many historical values were included in the training set for the calculation of the anomaly score for that particular metric and time bucket. This is confirmed in the query logic and documentation: > "count(metric_value) over (partition by metric_name, full_table_name, column_name order by bucket_start asc rows between unbounded preceding and current row) as training_set_size" [Slack: metrics_anomaly_score query](https://slack.com/archives/C02CTC89LAX/p1720623688508099) The training set is the historical data Elementary uses to determine what is "normal" for your metric, and

training_set_size

tells you how many such data points were used for that calculation.

To control the spike and drop threshold range in percentage for your use case (e.g., fail the test if the row count drops by more than 10% compared to previous runs), you should use the

ignore_small_changes

configuration in your

elementary.volume_anomalies

test. Here’s how you can set it up:

yaml models:   - name: your_table_name     tests:       - elementary.volume_anomalies:           ignore_small_changes:             drop_failure_percent_threshold: 10  # Fail if row count drops by more than 10%             spike_failure_percent_threshold: 10 # (Optional) Fail if row count increases by more than 10%

- `drop_failure_percent_threshold`: The test will only fail if the row count drops by more than this percentage compared to the mean value in the training period. - `spike_failure_percent_threshold`: Similarly, for increases. This configuration helps you avoid false positives from small fluctuations and ensures the test only fails for significant changes, such as a drop greater than 10% in your example. For more details, see the official documentation on [ignore_small_changes](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/ignore_small_changes).

You can use the

ignore_small_changes

configuration to control the spike and drop thresholds in percentage, but **it does not replace `anomaly_sensitivity`**—they work together. -

anomaly_sensitivity

controls how wide the expected range is, based on standard deviations from the mean. -

ignore_small_changes

adds an additional rule: the anomaly will only be flagged if the change exceeds your specified percentage threshold. If you do not specify

anomaly_sensitivity

, the default value (usually 3) will be used. You are not required to set both, but you can if you want more control over the anomaly detection behavior. The documentation and community discussions confirm that both parameters can be used together, and

ignore_small_changes

acts as an extra filter on top of the sensitivity calculation [[ignore_small_changes docs](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/ignore_small_changes)]. In summary: You do not have to specify

anomaly_sensitivity

if you are happy with the default, but you can use both together for finer control. If you only set

ignore_small_changes

, the test will still use the default sensitivity for the underlying calculation.

To receive a Slack alert as soon as a test passes, fails, or warns, you need to: 1. Run your dbt tests (e.g.,

dbt test

). 2. Immediately run the

edr monitor

command after your tests complete. This command checks for new test results and sends alerts to Slack based on the latest outcomes. Important notes: - Elementary does not send alerts in real time as each individual test finishes. Instead, it sends alerts after the dbt run is complete and

edr monitor

is executed. This is because Elementary relies on dbt's artifacts, which are only finalized at the end of the run. So, the alert will be sent as soon as you run

edr monitor

after your tests finish [[Slack: dbt task state and alert timing](https://slack.com/archives/C02CTC89LAX/p1674071962667739)]. - You can configure your CI/CD or orchestration tool (like Airflow, dbt Cloud, etc.) to run

edr monitor

right after your dbt tests to minimize delay. Slack alert configuration: - Make sure you have set up Slack integration as described in the documentation. You can use either a Slack token or webhook in your CLI command or config file [[Setup Slack Alerts](https://docs.elementary-data.com/oss/guides/alerts/send-slack-alerts)]. - Example CLI command:

bash   edr monitor --slack-token <your_slack_token> --slack-channel-name <your_channel>

Or, if using a webhook:

bash   edr monitor --slack-webhook <your_slack_webhook_url>

Alert types: - By default, Elementary sends alerts for failures and warnings. If you want to customize which statuses trigger alerts, you can use alert filters in your configuration [[Alerts Configuration](https://docs.elementary-data.com/oss/guides/alerts/alerts-configuration)]. Summary: You will get Slack alerts for test results (fail/warn) as soon as you run

edr monitor

after your dbt tests. Real-time (per-test) alerts are not supported; alerts are sent after the dbt run completes and the monitor command is executed.

The detection period in Elementary defines the recent time window (e.g., last 2 days by default) in which the test looks for anomalies. Only data points within this detection period are checked against the expected range calculated from the training set. If any value in the detection period is an outlier, the test will fail—even if the most recent value is now acceptable. From the documentation: > "If the detection_period are set to 2 days, only data points in the last 2 days will be included in the detection period and could be flagged anomalous. ... If there were any anomalies in the detection period, the test will fail." [Detection period documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period) This means that if your previous run had an anomaly and that data point is still within the detection period, the test will continue to fail until the detection period no longer includes that anomalous value. Even if your latest row count is now in the green zone, the presence of a recent anomaly within the detection period will cause the test to fail. You can adjust the detection period in your test configuration if you want to change how many recent time buckets are considered for anomaly detection. For more details, see the [detection period documentation](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period).

For a truncate-and-load table without a timestamp column, the detection period is controlled by the

backfill_days

parameter. In this context,

backfill_days

determines how many of the most recent test runs are included in the detection period for anomaly detection. To set the detection period to only consider the most recent run, configure your test like this:

yaml models:   - name: your_table_name     tests:       - elementary.volume_anomalies:           backfill_days: 1

This will ensure that only the latest run is checked for anomalies, and previous anomalous runs will not cause the test to fail once a new, acceptable row count is present. This behavior is confirmed in the knowledge sources: > "For tests with

timestamp_column

this

backfill_days

config is used both for recalculating metrics and detection period. For no

timestamp_column

it’s used only as detection period." [Slack: backfill_days for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1688112798629339) So, for your use case, set

backfill_days: 1

to have the detection period include only the most recent run.

If you set

backfill_days: 1

and the test still failed, it means that the most recent run is still being flagged as anomalous by Elementary. The

backfill_days

parameter, when used without a

timestamp_column

, controls the detection period by specifying how many of the most recent runs are checked for anomalies. If the latest run is within the expected range, the test should pass; if it is still outside the expected range, the test will fail—even if previous runs are no longer considered in the detection period. From the knowledge sources: > "For tests with

timestamp_column

this

backfill_days

config is used both for recalculating metrics and detection period. For no

timestamp_column

it’s used only as detection period." [Slack: backfill_days for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1688112798629339) If your latest run is in the "green zone" on the graph but the test still fails, possible reasons include: - The latest row count is still outside the calculated expected range (based on your training set and sensitivity). - The expected range may be skewed by previous anomalies or by a small training set. - The anomaly detection logic (e.g., sensitivity or ignore_small_changes) may still flag the latest value as anomalous. If you believe the latest value should not be flagged, review your test configuration for parameters like

anomaly_sensitivity

or

ignore_small_changes

, and check the actual values in the

metrics_anomaly_score

table to see the calculated expected range and z-score for the latest run. If the issue persists and the latest value is clearly within the expected range, there may be a bug or a need to further adjust your configuration. The knowledge sources do not provide more specific troubleshooting steps for this scenario.

Yes, your test is likely still failing because the last failed (anomalous) run is within the detection period defined by

backfill_days: 1

. For non-timestamp tests,

backfill_days

controls how many of the most recent runs are included in the detection period. If the most recent run is anomalous, the test will fail—even if previous runs are no longer considered. This behavior is confirmed in the knowledge sources: > "For tests with

timestamp_column

this

backfill_days

config is used both for recalculating metrics and detection period. For no

timestamp_column

it’s used only as detection period." [Slack: backfill_days for non-timestamp tests](https://slack.com/archives/C02CTC89LAX/p1688112798629339) So, if your last failed test is still the most recent run, and it is anomalous, the test will continue to fail until a new, non-anomalous run occurs.

Your understanding is correct: if you fix the row count and rerun your dbt build on the same day (or within a short time frame), the volume anomaly test will still fail if the previous anomalous run is within the detection period (controlled by

backfill_days

for non-timestamp tests). This is because Elementary flags the test as failed if any run within the detection period is anomalous, not just the most recent one. As for whether this is a "good" behavior, the knowledge sources acknowledge this as the current design, but do not explicitly state whether it is ideal or not. There are user discussions expressing that this can be inconvenient, especially for truncate-and-load tables or when rapid fixes are made and you want the test to reflect only the latest state: > "the anomaly will still show failure in the report, even after things are normalized, if there has been at least one anomaly in the last 2 days (if using the default) or whatever value of

backfill_days

is in the anomaly test configuration." [Slack: Cleaning up false positives and detection period](https://slack.com/archives/C02CTC89LAX/p1686034284021189) This behavior is by design, as the detection period is meant to capture recent anomalies for visibility and alerting. However, some users have found it less than ideal for certain workflows and have discussed workarounds or requested changes. Summary: - This is the intended behavior according to the current design. - Some users find it inconvenient, especially when quick remediations are made. - The knowledge sources do not state whether this is "good" or "bad," but do acknowledge the limitation and user feedback around it.