https://linen.dev logo
Join Slack
Powered by
<@U088WCP3B4P> I have a model that failed then tha...
# ask-elementary-ai
f
@kapa.ai I have a model that failed then that succeeded, and I am getting the Slack alert for failure - i don't want it as it has succeeded afterwards > how to fix it ? + in the summary it is shown as success
k
A message from kapa.ai
Currently, Elementary does not have built-in support to suppress Slack alerts for failures if the model or test has subsequently succeeded before the alert is sent. This is a known limitation and has been raised by multiple users as a feature request. The current behavior is that if a failure occurs and is recorded, Elementary will send an alert for that failure, even if the model or test later succeeds before the alert is sent. This can result in "false alarms" where you receive alerts for issues that have already been resolved by the time you see the notification. The recommended workaround is to run the edr monitor command immediately after each dbt execution. This minimizes the window in which a failure can be recorded and then resolved before the alert is sent, but it does not fully eliminate the issue. Users have noted that this approach can increase operational overhead, but as of now, it is the only way to ensure alerts are as up-to-date as possible with the latest run status. There is no current configuration or flag to only send alerts for the latest state or to deduplicate/suppress alerts for failures that have since been resolved by a successful run. The Elementary team has acknowledged this limitation and indicated that improvements to the alert lifecycle are planned for the future, which should address this use case [see discussion here](https://slack.com/archives/C02CTC89LAX/p1712245885735629) and [here](https://slack.com/archives/C02CTC89LAX/p1690801293721309). In summary: - There is no current way to automatically suppress alerts for failures that have since succeeded. - The best practice is to run edr monitor after every dbt run to reduce the chance of stale alerts. - Improvements to this behavior are planned, but not yet available. If this feature is important to your workflow, you are encouraged to open or upvote an issue on the Elementary GitHub to help prioritize its development.
f
@kapa.ai what does edr monitor do ?
k
A message from kapa.ai
The edr monitor command in Elementary reads from the test results table and sends new alerts (such as Slack alerts) for failed tests, models, or source freshness issues according to your configuration. It does not run dbt tests itself; instead, it processes the results of previous dbt runs and sends notifications for any new issues detected. Once an alert is sent, it is marked as sent in the database to prevent duplicate notifications in future runs. You can also control the time window for which alerts are considered using the --days-back parameter, and filter which models or tests to alert on using selectors. In summary, edr monitor is used to automate and manage the alerting process for data quality issues detected by dbt, ensuring your team is notified of new problems as they arise, based on your configuration and scheduling preferences [CLI commands and usage](https://docs.elementary-data.com/oss/cli-commands) [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1664962599452549).
f
@kapa.ai can i exclude failed PR runs from alerts ?
k
A message from kapa.ai
Yes, you can exclude failed PR (Pull Request) runs from alerts in Elementary by configuring your dbt project to only enable Elementary models (and thus alerting) for specific environments, such as production. This is typically done by using the +enabled configuration in your dbt_project.yml file, referencing the target environment name. For example, to only enable Elementary in production:
Copy code
yaml models:   elementary:     +schema: elementary     +enabled: "{{ target.name == 'prod' }}"
With this setup, Elementary models (including data monitors and alerting) will not run in PR or CI environments where the target name is not 'prod', effectively suppressing alerts from those runs. This approach has been confirmed as effective by community members and support staff in the Elementary Slack community [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1738778909958349). Make sure your PR jobs are using a different target name than production, and verify your dbt_project.yml is set up accordingly.
f
@kapa.ai what's the difference between warn and error freshness in slack alerts ? are they both send using edr command
k
A message from kapa.ai
The difference between warn and error freshness in Slack alerts is based on the thresholds you set for your dbt source freshness checks. A "warn" alert is triggered when the data freshness exceeds the warn_after threshold, while an "error" alert is triggered when it exceeds the error_after threshold. These thresholds are typically defined in your dbt source configuration. Both warn and error freshness alerts can be sent using the edr monitor command, but by default, Elementary only sends alerts for errors. If you want to receive alerts for warnings as well, you need to set the following variable in your dbt_project.yml:
Copy code
yaml vars:   disable_warn_alerts: false
This will enable Slack alerts for both warn and error freshness statuses when you run edr monitor. If this variable is not set (or set to true), only error alerts will be sent for freshness issues [Slack discussion](https://slack.com/archives/C02CTC89LAX/p1689088826025229).
f
@kapa.ai Can I use -tz, --timezone TEXT parameter with edr report and edr monitor ?
k
A message from kapa.ai
Yes, but with an important nuance. The 
-tz, --timezone
option is a general CLI option that can be used with Elementary commands (it’s documented in the shared CLI options list), and it controls “the timezone that will be used upon calculating the alerts’ time field.” [[CLI options](https://docs.elementary-data.com/oss/cli-commands#cli-options)] However, from the community discussions: - It is supported for alerts (i.e., with 
edr monitor
). [[Slack Q&A timezone](https://elementary-community.slack.com/archives/C02CTC89LAX/p1694165297650309); [CLI options](https://docs.elementary-data.com/oss/cli-commands#cli-options)] - It is not currently supported for the report summary sent by 
edr send-report
, and the maintainer explicitly confirmed that limitation. [[GH issue 1879](https://github.com/elementary-data/elementary/issues/1879)] The docs and threads you provided do not explicitly confirm or show an example of 
--timezone
being used with 
edr report
itself, only with 
edr monitor
and as a global CLI option. Because of that, I can confidently say: - You can use 
-tz/--timezone
with 
edr monitor
. - The knowledge sources do not clearly state that 
-tz/--timezone
affects 
edr report
, so I can’t guarantee it applies there based on the available documentation.
f
@kapa.ai detection_period: 1 day meaning ?
k
A message from kapa.ai
detection_period
controls which recent time buckets are checked for anomalies. When you set:
Copy code
yaml detection_period:   period: day   count: 1
it means: - Only data points in the last 1 day (1 detection period) are in the detection set and can be flagged anomalous. [[detection_period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period); [how it works](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works)] - All earlier buckets (still within the 
training_period
) are used only to learn the “normal” behavior (the training set), not to trigger alerts. [[how it works](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works)] Important implementation detail: - For daily buckets, “1 day” means the last full daily bucket, not “last 24 rolling hours”. So with daily 
time_bucket
, a 
detection_period
of 1 day will typically evaluate yesterday’s bucket (midnight–midnight), not partial data from today. [[detection_period issue discussion](https://github.com/elementary-data/elementary/issues/1523)]
f
@kapa.ai i also have detection_delay: period: day count: 1 Should i remove it if i want to test previous day ?
k
A message from kapa.ai
No, you should keep that 
detection_delay
if you want to test “yesterday” (the last complete day). Here’s how the two work together:
Copy code
yaml detection_period:   period: day   count: 1  detection_delay:   period: day   count: 1
- 
detection_period
= length of the window you test for anomalies (1 day). [[detection period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period)] - 
detection_delay
= how far back you shift that window, to ignore the most recent data (often “today”, which may be partial). [[detection delay](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-delay)] With this combo, Elementary will: - Skip the latest 1 day (today), - Evaluate anomalies for the day before that (yesterday’s full bucket). So if your goal is to detect anomalies for the previous complete day, your current settings are correct and you should not remove 
detection_delay
.
f
@kapa.ai here it says that it's already checking yesterday full day even without delay _- For daily buckets, “1 day” means the last full daily bucket, not “last 24 rolling hours”. So with daily 
time_bucket
, a 
detection_period
of 1 day will typically evaluate yesterday’s bucket (midnight–midnight), not partial data from today_
k
A message from kapa.ai
You’re right to question that; there are a few overlapping concepts in the docs and Slack threads, so let’s separate them clearly. ### 1. What 
detection_period: 1 day
does 
detection_period
defines how long the detection window is, not how far back it is shifted. With:
Copy code
yaml detection_period:   period: day   count: 1
Elementary will look for anomalies in a 1‑day detection window. [[detection period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period)] With daily 
time_bucket
, that window is applied over completed daily buckets only – “The test will only run on completed time buckets”. [[volume anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/volume-anomalies)] However, by itself, 
detection_period: 1 day
does not guarantee “yesterday only” in all situations; the exact cut is implemented in SQL and there has been confusion/bugs around it (non‑strict 
>=
in the detection filter, etc.). [[GH issue 1523](https://github.com/elementary-data/elementary/issues/1523)] ### 2. What 
detection_delay: 1 day
adds 
detection_delay
is the only explicit knob to skip the most recent data:
Copy code
yaml detection_delay:   period: day   count: 1
> “The duration for retracting the detection period… useful when the latest data should be excluded from the test (e.g. test runs before table is fully populated). It does not affect 
detection_period
or 
training_period
.” [[detection delay](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-delay)] So, with:
Copy code
yaml time_bucket:   period: day   count: 1 detection_period:   period: day   count: 1 detection_delay:   period: day   count: 1
the logic is: - Use daily completed buckets. - Ignore the last 1 day (today’s bucket, or the most recent full bucket). - Detect anomalies on the 1‑day window immediately before that → effectively “yesterday” in a daily pipeline that loads with one‑day delay. [[detection delay](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-delay)] ### 3. So should you remove 
detection_delay
? If your data for the previous day is only complete with one day lag (common case), and you want to test “yesterday” as of today’s run: - You should keep 
detection_delay: 1 day
. - It ensures the detection window is shifted back so that you are not looking at an in‑flight or misaligned latest bucket. [[detection delay](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-delay)] If, instead, you want to test the most recent full bucket with no delay (e.g. data is complete as soon as the day closes and your run is late enough), then you could remove 
detection_delay
and rely on 
detection_period
alone plus the “completed buckets only” rule. But note that the open GitHub issue around 
detection_period
semantics shows some edge‑case confusion, so the behavior there has not been fully clarified/fixed yet. [[GH issue 1523](https://github.com/elementary-data/elementary/issues/1523)] Given the current docs, using 
detection_delay: 1 day
is the safest and most explicit way to guarantee that you’re checking the prior day, not the most recent bucket.
f
@kapa.ai i am getting alerts in slack on the next day even though they were fixed and not present in the latest dbt test, how to debug ?
k
A message from kapa.ai
You’re describing a pattern we’ve seen a few times: alerts for a failure are sent later, even though the latest dbt run is green. Here’s how to debug it, step by step. --- ### 1. Confirm it’s expected “pending alert” behavior Elementary stores alerts and only sends them once; unsent ones are considered “pending”. On each 
edr monitor
run, it sends any pending alerts from the last N days. [[Slack pending alerts](https://elementary-community.slack.com/archives/C02CTC89LAX/p1682315767505949); [GitHub 1972 comments](https://github.com/elementary-data/elementary/issues/1972)] This means: - A failure yesterday can still generate an alert today, even if today’s run passed, if: - 
edr monitor
didn’t run (or couldn’t write) yesterday, or - the alert stayed in a pending state. So first question: are you running 
edr monitor
after every dbt test/run, or on a different schedule? If you want alerts to reflect only the latest state, the current workaround many users take is to run 
edr monitor
right after every dbt execution. [[Slack latest state discussion](https://elementary-community.slack.com/archives/C02CTC89LAX/p1712245885735629)] --- ### 2. Check if alerts are staying “pending” Look at your Elementary schema (warehouse): 1. For OSS, check the alerts tables: - 
alerts
/ 
alerts_dbt_tests
/ 
alerts_models
/ 
alerts_source_freshness
(exact tables depend on your version). 2. Inspect whether old alerts have 
sent_at
populated or 
NULL
. Example from a similar case: many alerts had 
sent_at
still 
NULL
, so Elementary kept trying to send them again. [[Slack Redshift sent_at issue](https://elementary-community.slack.com/archives/C02CTC89LAX/p1684173645049519)] If you see: - Many rows with 
sent_at IS NULL
for old failures → alerts are not being marked as sent. Then: - Verify that the user Elementary uses has INSERT/UPDATE privileges on the Elementary schema and alert tables. [[Permissions check thread](https://elementary-community.slack.com/archives/C02CTC89LAX/p1684173645049519)] Also note there was a known Redshift / 
dbt-redshift
issue where 
run-operation
didn’t commit updates, so 
sent_at
never got written, causing repeated alerts. [[Slack Redshift root cause](https://elementary-community.slack.com/archives/C02CTC89LAX/p1684173645049519)] --- ### 3. Check for duplicate senders If you are on Elementary Cloud and also running the CLI 
edr monitor
 in your DAG, both can send alerts: - This leads to duplicates or alerts appearing “again” when the DAG re‑runs. [[Slack double alerts cloud+CLI](https://elementary-community.slack.com/archives/C02CTC89LAX/p1690801293721309)] To debug: - Confirm whether Elementary Cloud is enabled for the same environment. - If yes, and you still run 
edr monitor
in Airflow / CI, remove the CLI part and let Cloud handle alerts. --- ### 4. Confirm test failures really exist only in older runs Sometimes what looks like a “ghost” alert is actually: - A failure from a local run against production, or - Another job writing artifacts into the same Elementary schema. Those failures can surface later as alerts, even if your main production job is green. [[Slack local runs causing repeats](https://elementary-community.slack.com/archives/C02CTC89LAX/p1747400841069959)] To debug: - Check 
dbt_invocations
/ test results tables for runs from: - Local machines - Other CI jobs / DAGs - Consider: - Running local dbt against a separate target. - Or disabling artifact uploads for non‑prod targets using the on‑run-end hooks config. [[Disable artifact upload](https://elementary-community.slack.com/archives/C02CTC89LAX/p1747400841069959)] --- ### 5. Look at the monitor logs and macros being run When 
edr monitor
runs, it: - Queries “pending” alerts via macros like: - 
get_pending_model_alerts
- 
get_pending_source_freshness_alerts
- Then updates them to “sent”. [[Slack logs with get_pending_*](https://elementary-community.slack.com/archives/C02CTC89LAX/p1694422097058109)] To debug: - Check 
edr.log
and the internal dbt log for the monitor project: - It’s under the 
elementary/monitor/dbt_project/logs/dbt.log
path in your Python env. [[Internal project log location](https://elementary-community.slack.com/archives/C02CTC89LAX/p1684173645049519)] - Verify those macros complete without errors and that “update sent times” queries are not failing silently. --- ### 6. Specific patterns to watch for From similar reported cases: - Source freshness alerts resent after 
dbt run
 Past‑day freshness alerts were marked sent, then later changed back to pending after another run, and resent. [[Slack source freshness repeating](https://elementary-community.slack.com/archives/C02CTC89LAX/p1698758233878939)] - Delayed warn alerts 
warn
-severity singular tests appeared a day late because of a suppression or hidden config; overriding with 
--suppression-interval 0
helped debug. [[Slack delayed warn alerts](https://elementary-community.slack.com/archives/C02CTC89LAX/p1737531383494819)] If you see a consistent 24h delay, try:
Copy code
bash edr monitor --override-dbt-project-config --suppression-interval 0
and see if the behavior changes. [[Slack delayed warn alerts](https://elementary-community.slack.com/archives/C02CTC89LAX/p1737531383494819)] --- ### 7. Current limitation: “only latest state” alerts Multiple users have asked for: “only send alerts for the latest run / don’t alert if the issue is already fixed.” Today: - Elementary will send an alert for a failure once it’s recorded, even if a later run already passed before the alert fires. [[Slack lifecycle discussion](https://elementary-community.slack.com/archives/C02CTC89LAX/p1690801293721309); [Slack same concern](https://elementary-community.slack.com/archives/C02CTC89LAX/p1712245885735629); [GitHub issue 1972](https://github.com/elementary-data/elementary/issues/1972)] The main mitigations are: - Run 
edr monitor
as close as possible to each dbt run. - Avoid extra runs (local, CI) writing into the same Elementary schema. - Ensure 
sent_at
is updated correctly so alerts are not resent. --- ### If you share more details If you can share: - How/when you run 
dbt test
and 
edr monitor
, - Whether you use Cloud + CLI together, - Which warehouse & adapter you use, I can point you to the most likely cause among the patterns above and suggest a narrower next step (e.g., specific queries to run on your Elementary schema).
f
@kapa.ai I am using this command to update elementary after a dbt test dbt run --select elementary.edr.dbt_artifacts Is it correct ?
k
A message from kapa.ai
Yes, that command is correct for updating the dbt artifact metadata after your dbt tests:
Copy code
bash dbt run --select elementary.edr.dbt_artifacts
This runs the 
dbt_artifacts
models (e.g. 
dbt_models
, 
dbt_tests
, 
dbt_sources
, etc.) which keep your project metadata up to date. This is exactly what we recommend when you disable the automatic artifact upload (
disable_dbt_artifacts_autoupload: true
) and want to refresh artifacts manually after code/config changes. [[On‑run‑end hooks](https://docs.elementary-data.com/dbt/on-run-end_hooks); [dbt artifacts](https://docs.elementary-data.com/dbt/dbt-artifacts); [Slack manual artifacts run](https://elementary-community.slack.com/archives/C02CTC89LAX/p1688041433774369)] A typical pattern when hooks are disabled is: 1. Run your regular 
dbt run
/ 
dbt test
jobs (results still go to 
dbt_run_results
, 
elementary_test_results
via on‑run‑end). [[On‑run‑end hooks](https://docs.elementary-data.com/dbt/on-run-end_hooks)] 2. Periodically (e.g. on deploy / after schema changes) run:
Copy code
bash    dbt run --select elementary.edr.dbt_artifacts
to refresh metadata. [[On‑run‑end hooks](https://docs.elementary-data.com/dbt/on-run-end_hooks)] So yes: using that command after your dbt test or as part of your deployment process is the correct way to keep Elementary’s artifact tables current when you manage artifact uploads manually.
f
@kapa.ai What does this do ? - elementary.dimension_anomalies: # looks for anomalies in weekly row count by ID_app config: severity: error arguments: timestamp_column: DT_installation detection_period: # Optional, The period for anomaly detection. period: week # Optional count: 1 # Optional detection_delay: period: day count: 1 # excludes current/last day dimensions: - ID_app meta: description: "Adjust - Anomaly detection on weekly installs"
k
A message from kapa.ai
This configuration adds an Elementary dimension anomalies test that monitors weekly installs per 
ID_app
and alerts on abnormal changes. Breaking it down:
Copy code
yaml - elementary.dimension_anomalies:     config:       severity: error     arguments:       timestamp_column: DT_installation       detection_period:         period: week         count: 1       detection_delay:         period: day         count: 1       dimensions:         - ID_app     meta:       description: "Adjust - Anomaly detection on weekly installs"
What it does: - Test type: 
elementary.dimension_anomalies
Counts rows grouped by the configured 
dimensions
and looks for unexpected changes in that distribution over time. [[Dimension anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies)] - Dimension: 
ID_app
The test groups by 
ID_app
and monitors the row count per app as the metric. [[dimensions config](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/dimensions)] - Time axis: 
timestamp_column: DT_installation
Elementary uses 
DT_installation
to build time buckets (here, weeks). If 
timestamp_column
is configured, the distribution is collected per 
time_bucket
instead of over all history. [[timestamp_column](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/timestamp-column); [dimension anomalies behavior](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/dimension-anomalies)] - Detection window:
Copy code
yaml   detection_period:     period: week     count: 1
Elementary will search for anomalies only in the last 1-week detection period (the detection set). [[detection_period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-period); [how it works](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works)] - Detection delay:
Copy code
yaml   detection_delay:     period: day     count: 1
Shifts the detection window back by 1 day, excluding the most recent day (to avoid testing partially loaded data). It doesn’t change the length of 
detection_period
, only moves it back in time. [[detection_delay](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-delay)] - Training vs detection: Historical weekly per‑`ID_app` counts (within the training period; default if not overridden) form the training set used to compute the expected range, and the shifted 1‑week detection window is the detection set compared against it. [[How anomaly tests work](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works)] - Severity: 
severity: error
If an anomaly is detected (e.g., a spike or drop in weekly installs for a specific 
ID_app
), the dbt test will fail with error severity, which can then trigger error-level alerts/behavior in your pipelines. (Severity behavior is standard dbt test config; Elementary respects it for its tests.) [[Add tests + severity hint](https://docs.elementary-data.com/data-tests/add-elementary-tests#configure-your-elementary-anomaly-detection-tests)] In effect: this test flags apps whose weekly install count (based on 
DT_installation
) looks anomalously high or low, evaluated over a 1‑week window that excludes the most recent day, and treats such anomalies as errors.
f
@kapa.ai data_tests: - elementary.column_anomalies: config: severity: warn tags: anomaly arguments: column_anomalies: - null_percent timestamp_column: DT_creation Should this end up in metrics_anomaly_score table ? or is it only for sum anomalies ?
k
A message from kapa.ai
Yes, 
elementary.column_anomalies
with 
null_percent
should write metrics into 
data_monitoring_metrics
, and those metrics are what 
metrics_anomaly_score
reads. It is not limited to 
sum
anomalies. From the docs: - 
elementary.column_anomalies
“executes column level monitors and anomaly detection on the column”; all supported column monitors (including 
null_percent
) are part of the same framework. [[Column anomalies test](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies); [column monitors list](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/column-anomalies)] - Anomaly detection tests store their metrics in `data_monitoring_metrics`; 
metrics_anomaly_score
is a view on that table “that runs the same query the anomaly detection tests run to calculate anomaly scores.” [[Anomaly detection models](https://docs.elementary-data.com/dbt/package-models#anomaly-detection)] So, for your config:
Copy code
yaml data_tests:   - elementary.column_anomalies:       config:         severity: warn         tags: anomaly       arguments:         column_anomalies:           - null_percent         timestamp_column: DT_creation
you should expect: 1. Metrics rows (with 
metric_name = 'null_percent'
) in 
elementary.data_monitoring_metrics
. 2. Corresponding anomaly scores in 
elementary.metrics_anomaly_score
once there is enough history and the test has run multiple times. The only caveats the sources mention: - If there’s not enough historical data, the test may return “Not enough data to calculate anomaly score” and you won’t see anomaly rows yet, even though metrics start accumulating. [[Insufficient data case](https://elementary-community.slack.com/archives/C02CTC89LAX/p1687331770365499)] - 
sum
is just an opt‑in monitor; it uses the same mechanism as the default monitors like 
null_percent
. [[Column monitors list](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/column-anomalies)] If you don’t see 
null_percent
in 
data_monitoring_metrics
, the next debugging step is to query that table filtered by your model/column and check whether any column metrics are being written.
f
@kapa.ai it's working for one user but not for another one.. what could be the issue ?
k
A message from kapa.ai
For your 
column_anomalies
test with 
null_percent
+ 
timestamp_column
, it should write metrics that end up in 
metrics_anomaly_score
(not only 
sum
). [[Column anomalies](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies); [Anomaly models](https://docs.elementary-data.com/dbt/package-models#anomaly-detection)] Since it works for one user but not another, the most likely causes (based on similar Slack threads) are: 1. Different dbt / Elementary versions There were bugs in some versions where: - Metrics weren’t written to 
data_monitoring_metrics
, or - Column anomaly metrics didn’t show up in the UI / score models until upgrading. [[Freshness bug 0.8.1](https://elementary-community.slack.com/archives/C02CTC89LAX/p1689669432523659); [column_anomalies “Not enough data” bug 0.8.1](https://elementary-community.slack.com/archives/C02CTC89LAX/p1688298056468299)] Check that both users are on the same Elementary package version and dbt version. 2. Different configuration / syntax Small YAML differences can cause the test to compile but not actually collect metrics: - 
column_anomalies
must be a list, not a string; otherwise you get “Not enough data to calculate anomaly score” / no metrics. [[column_anomalies must be list](https://elementary-community.slack.com/archives/C02CTC89LAX/p1688298056468299)] - Indentation / structure issues can cause one user’s config to fall back to defaults (or not run the anomaly macro at all). [[Indentation issue causing extra metrics](https://elementary-community.slack.com/archives/C02CTC89LAX/p1759179499606799)] Compare the exact YAML that each user is running (including 
dbt_project.yml
vars like 
training_period
, 
days_back
, global 
elementary
config). 3. Different execution / environment In several cases anomalies worked in one env but not another because: - Elementary was disabled in a dev target via 
+enabled: "{{ false if target.name == 'dev' else true }}"
. [[all_columns_anomalies missing from report](https://elementary-community.slack.com/archives/C02CTC89LAX/p1673608521417509)] - One user ran tests in dev where Elementary models/test collection were off. [[Freshness anomalies on dev](https://elementary-community.slack.com/archives/C02CTC89LAX/p1689669432523659)] Verify both users: - Use the same target (
profiles.yml
/ dbt Cloud environment), - Hit the same warehouse/schema for 
elementary
models. 4. Not enough data collected yet for one user Anomaly scores appear only after there are **≥ 7 training data points**; before that you may see “Not enough data to calculate anomaly score” and nothing in 
metrics_anomaly_score
, even though 
data_monitoring_metrics
is populating. [[Troubleshooting guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] For the user where it “doesn’t work”: - Check `data_monitoring_metrics`:
Copy code
sql      SELECT *      FROM elementary.data_monitoring_metrics      WHERE table_name = '<your_model>'        AND column_name = 'account_id'          -- or your column        AND metric_name = 'null_percent'      ORDER BY metric_timestamp DESC;
- If there are rows here but none in 
metrics_anomaly_score
, they probably don’t yet have enough history. [[Troubleshooting guide](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] 5. Package upgrade but models not re‑run After upgrading Elementary, users have had metrics missing until they ran the Elementary models again:
Copy code
bash    dbt run -s elementary
This fixed several anomaly‑related issues. [[Upgrade then run elementary](https://elementary-community.slack.com/archives/C02CTC89LAX/p1681298359181089)] --- How to narrow it down For the user where it “doesn’t work”: 1. Confirm their Elementary package + dbt versions match the working user. 2. Run:
Copy code
bash    dbt test --select tag:anomaly    dbt run --select elementary
3. Query:
Copy code
sql    -- metrics collected?    SELECT *    FROM elementary.data_monitoring_metrics    WHERE table_name = '<your_model>'      AND column_name = '<your_column>'      AND metric_name = 'null_percent';     -- anomaly scores?    SELECT *    FROM elementary.metrics_anomaly_score    WHERE table_name = '<your_model>'      AND column_name = '<your_column>'      AND metric_name = 'null_percent';
If you share which of these queries return rows (for each user), I can point to the exact likely cause based on the patterns above.
f
@kapa.ai how can i check my package version ?
k
A message from kapa.ai
You can check your Elementary package versions in three different places, depending on what you mean: 1. Elementary dbt package version (in your project) Open your 
packages.yml
in the dbt project and look for the Elementary entry:
Copy code
yaml    packages:      - package: elementary-data/elementary        version: ...
This is the dbt package version used by dbt runs. [[Upgrade advice](https://elementary-community.slack.com/archives/C02CTC89LAX/p1684952078647519); [Version check in packages.yml](https://elementary-community.slack.com/archives/C02CTC89LAX/p1697209544439999)] 2. Elementary Python/CLI (
edr
) version On the machine/CI where 
edr
runs, execute:
Copy code
bash    pip show elementary-data
This prints the installed CLI version. [[CLI version workaround](https://elementary-community.slack.com/archives/C02CTC89LAX/p1681874868916669)] 3. Version recorded in your warehouse by Elementary Elementary also records its version in the 
dbt_invocations
table. You can query:
Copy code
sql    SELECT elementary_version    FROM <ELEMENTARY_SCHEMA>.dbt_invocations    ORDER BY generated_at DESC    LIMIT 1;
This returns the version associated with the latest run that wrote metadata. [[Version from dbt_invocations](https://elementary-community.slack.com/archives/C02CTC89LAX/p1674726838236829); [How edr gets version](https://elementary-community.slack.com/archives/C02CTC89LAX/p1678787357601019)]
f
@kapa.ai data_monitoring_metrics is empty
k
A message from kapa.ai
If 
data_monitoring_metrics
is empty, it means no anomaly‑detection metrics are being collected yet. All the anomaly tests (
volume_anomalies
, 
column_anomalies
, 
dimension_anomalies
, 
freshness_anomalies
) write their metrics into this table, so it must have data before 
metrics_anomaly_score
will show anything. [[Metrics table purpose](https://elementary-community.slack.com/archives/C02CTC89LAX/p1688373015213789?thread_ts=1688373015.213789&amp;cid=C02CTC89LAX)] Here’s how to debug: --- ### 1. Confirm you’re actually running anomaly tests Only anomaly tests populate 
data_monitoring_metrics
. Regular dbt tests do not. [[Metrics table purpose](https://elementary-community.slack.com/archives/C02CTC89LAX/p1688373015213789?thread_ts=1688373015.213789&amp;cid=C02CTC89LAX)] Make sure you have tests like:
Copy code
yaml data_tests:   - elementary.column_anomalies:       arguments:         column_anomalies:           - null_percent         timestamp_column: DT_creation
and that you’re running:
Copy code
bash dbt test --select <your_model_or_tag>
Double‑check the YAML structure matches the docs for anomaly tests. [[column_anomalies docs](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/column-anomalies)] --- ### 2. Run the Elementary models after configuring tests After adding or changing anomaly tests, Elementary often needs its internal models refreshed:
Copy code
bash dbt run -s elementary
This step has fixed missing anomaly data for others after changes/upgrades. [[Run elementary after upgrade](https://elementary-community.slack.com/archives/C02CTC89LAX/p1681298359181089?thread_ts=1681298359.181089&amp;cid=C02CTC89LAX)] --- ### 3. Check that anomaly tests actually succeed If the anomaly test fails to compile or run, no metrics will be written. Look in your 
dbt.log
and 
edr.log
for the failing test name. In multiple cases, compilation errors (wrong timestamp type, bad config) prevented 
data_monitoring_metrics
from being populated. [[Freshness anomalies compilation error](https://elementary-community.slack.com/archives/C02CTC89LAX/p1689669432523659?thread_ts=1689669432.523659&amp;cid=C02CTC89LAX)] --- ### 4. Verify timestamp configuration (if used) If you use `timestamp_column`: - Ensure the column exists and is of type `timestamp`/`date`. - Check for `NULL`s in that column; bad timestamps can prevent buckets from being created and metrics from being written. [[Anomaly troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] If you run the test without a timestamp: - Metrics are collected per test run using run time as the “timestamp”; you must run the test multiple times (different days/buckets) to see enough points, but you should still see rows appear in 
data_monitoring_metrics
as you run it. [[Anomaly troubleshooting – non‑timestamp tests](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] --- ### 5. Environment / schema mismatch Make sure you’re querying the same Elementary schema and database that your tests write to. A similar issue happened when users looked in a different schema than where 
data_monitoring_metrics
was actually created. [[Wrong table name/schema example](https://elementary-community.slack.com/archives/C02CTC89LAX/p1681849464180719?thread_ts=1681849464.180719&amp;cid=C02CTC89LAX)] --- ### 6. Version issues Some older package versions had bugs where anomaly metrics were not written correctly; upgrading fixed the problem. [[Freshness anomalies bug in 0.8.1](https://elementary-community.slack.com/archives/C02CTC89LAX/p1689669432523659?thread_ts=1689669432.523659&amp;cid=C02CTC89LAX)] Check: - The Elementary dbt package version in 
packages.yml
. - Consider running 
dbt deps
to update to a newer compatible version, then 
dbt run -s elementary
and re‑run your anomaly tests. --- If you share: - The exact YAML for one of your anomaly tests, - Your Elementary package version, - And confirm whether those tests appear in 
elementary_test_results
, I can point you to the most likely missing piece based on the patterns above.
f
@kapa.ai when do we have an anomaly? what score ? in oss ?
k
A message from kapa.ai
In Elementary OSS, anomalies are detected using a **Z‑score (standard score)**–based method: 1. How the score is calculated For each metric value in the detection set: \[ \text{anomaly\_score} = \frac{\text{metric\_value} - \text{training\_avg}}{\text{training\_stddev}} \] This is the number of standard deviations the current value is from the historical mean. [[Anomaly method](https://docs.elementary-data.com/data-tests/data-anomaly-detection); [How tests work](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works)] 2. When is it considered an anomaly? By default, OSS uses a threshold of 3 standard deviations: - Values with |anomaly_score| ≥ 3 are treated as outliers/anomalies. [[Anomaly method](https://docs.elementary-data.com/data-tests/data-anomaly-detection)] - This is controlled by the sensitivity: - 
anomaly_sensitivity
test argument (per‑test) [[anomaly_sensitivity](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/anomaly-sensitivity)] - and, in older docs, a global 
anomaly_score_threshold
var. [[Anomaly method](https://docs.elementary-data.com/data-tests/data-anomaly-detection)] Lower sensitivity (e.g. 1.5) → narrower expected range → **more anomalies**; higher sensitivity (e.g. 4) → wider range → fewer anomalies. [[anomaly_sensitivity](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/anomaly-sensitivity)] 3. Additional conditions A point is only evaluated as a potential anomaly if: - It is in the detection period, not just the training period. [[How tests work](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works)] - There is enough history: at least **7 training data points**; otherwise you get “Not enough data to calculate anomaly”. [[Anomaly troubleshooting](https://docs.elementary-data.com/data-tests/anomaly-detection-tests/Anomaly-troubleshooting-guide)] So in OSS: an anomaly occurs when a value in the detection period has an absolute Z‑score above the configured sensitivity (default 3), with sufficient training data.
36 Views
Open in Slack
PreviousNext
Powered by