<@U088WCP3B4P> I’m using the edr CLI to send a dbt...
# ask-elementary-ai
k
@kapa.ai I’m using the edr CLI to send a dbt source freshness alert to a Slack channel. The alert is being sent, but it’s sent twice with identical content. Upon checking the
ALERTS_V2
metadata table, I found that EDR inserted two identical records, which caused the duplicate Slack alerts. Here’s the command I executed:
Copy code
edr monitor --filters resource_types:source_freshness --slack-webhook MY_WEBHOOK_HOST
I’m using Elementary version 0.16.1, which should already include the fix introduced before version `0.6.10`: https://docs.elementary-data.com/oss/release-notes/releases/0.6.10
k
A message from kapa.ai
Based on the knowledge sources, duplicate Slack alerts for dbt source freshness can occur if Elementary inserts duplicate records into the alerts metadata tables (such as alerts_v2). This issue has been reported by several users, and the root causes identified in the community include: 1. Concurrent Runs of edr monitor: If you run multiple edr monitor commands at the same time (even for different resources or departments), both runs may see the same state in the alerts table and insert duplicate alerts. This is a known behavior, and the recommended workaround is to avoid running edr monitor concurrently—run them sequentially instead. The Elementary team has acknowledged this and is considering a more robust solution, but as of now, running the commands one after another is the best way to prevent duplicates in alerts_v2 and, consequently, duplicate Slack alerts [see discussion](https://slack.com/archives/C02CTC89LAX/p1709287424734769). 2. Artifact Table Duplication: There have been cases where duplicate entries in artifact tables (like dbt_models, dbt_tests, etc.)—often due to parallel dbt runs or concurrency issues—lead to duplicate alerts. Workarounds include running dbt run -s elementary to refresh the tables, or, in some cases, deduplicating the tables manually. However, this is more commonly reported for model/test alerts, but the underlying mechanism could affect source freshness as well [see discussion](https://github.com/elementary-data/elementary/issues/1420). 3. Legacy Issues and Fixes: The deduplication fix you referenced (from version 0.6.10) prevents double alerts from being sent in most cases, but it does not address all scenarios—especially those involving concurrency or race conditions. The fix ensures that, for a single run, only unsent alerts are sent, and once sent, they are marked as such. However, if two processes insert at the same time, both may insert and send the same alert [see discussion](https://slack.com/archives/C02CTC89LAX/p1709287424734769). Summary of Recommendations: - Ensure you are not running multiple edr monitor commands at the same time. - If you must run multiple monitors, try to sequence them to avoid overlap. - If duplicates persist even with sequential runs, check for duplicate entries in the underlying artifact tables and consider refreshing or deduplicating them as a temporary workaround. If you are already following these practices and still see duplicates, it may be a new or unresolved bug. In that case, providing logs and details to the Elementary team (as suggested in the community) would be the next step. Let me know if you need guidance on checking for duplicates in your artifact tables or further troubleshooting steps!
k
@kapa.ai I checked that we don't run the edr monitor commands in the same time
k
A message from kapa.ai
Thank you for clarifying that you are not running edr monitor commands concurrently. Based on the knowledge sources, while concurrent runs are a common cause of duplicate alerts, there are also documented cases where duplicate records appear in the alerts tables (such as alerts_v2) even when edr monitor is not run in parallel. These duplicates can be caused by: - Duplicate entries in artifact tables (like dbt_models, dbt_tests, etc.), which can occur due to previous parallel runs, race conditions, or issues with how dbt artifacts are uploaded and processed. These duplicates can propagate into the alerts tables and result in multiple identical alerts being sent [see discussion](https://github.com/elementary-data/elementary/issues/1420) [see also](https://slack.com/archives/C02CTC89LAX/p1672942935757879) [see also](https://slack.com/archives/C02CTC89LAX/p1672942935757879). - Residual duplicates from earlier issues: Even if you are not currently running commands in parallel, previous duplications in the artifact or alerts tables can persist and cause repeated alerts until the tables are cleaned or refreshed [see discussion](https://slack.com/archives/C02CTC89LAX/p1696435655140539). Recommended next steps based on community guidance: 1. Check for duplicate rows in your artifact tables (such as dbt_models, dbt_tests, dbt_sources) and in alerts_v2, especially for the affected alert type (source freshness). If you find duplicates, you may need to refresh or deduplicate these tables. 2. Run
dbt run -s elementary
to refresh the Elementary tables, which has helped some users resolve duplicate alert issues, at least temporarily [see discussion](https://slack.com/archives/C02CTC89LAX/p1696435655140539). 3. If the problem persists, it may be due to a bug or an edge case not yet fully addressed in the current version. In such cases, providing detailed logs and table snapshots to the Elementary team is recommended for further investigation. If you have already checked for concurrent runs and still see duplicates, the most likely cause is duplicate data in the underlying tables. Cleaning or deduplicating these tables should help, but if not, it may require direct support from the Elementary team. Let me know if you need help with SQL queries to check for duplicates or further troubleshooting steps!