Luiz Felipe Campos da Silva
11/26/2025, 2:50 PMkapa.ai
11/26/2025, 2:50 PMkapa.ai
11/26/2025, 2:50 PMdbt_models artifacts table (same unique_id with different generated_at), which then cause a Databricks Delta MERGE to see “multiple source rows matching target row.” From the shared issues and Slack threads, there are two main causes: 1. Concurrent artifact uploads / concurrent dbt runs - When multiple dbt runs (or edr monitor / edr report) run at the same time, each may try to update the Elementary artifact models, leading to duplicate records per unique_id in dbt_models, dbt_tests, dbt_sources, etc. [[dup bug issue](https://github.com/elementary-data/elementary/issues/1420); [concurrency thread](https://elementary-community.slack.com/archives/C02CTC89LAX/p1672942935757879?thread_ts=1672942935.757879&cid=C02CTC89LAX); [Databricks conflicts](https://elementary-community.slack.com/archives/C02CTC89LAX/p1694431548484389?thread_ts=1694431548.484389&cid=C02CTC89LAX)] - This is especially common on Databricks / Delta where concurrent DELETE/TRUNCATE + INSERT patterns hit DELTA_CONCURRENT_DELETE_READ or “multiple source rows” MERGE errors. [[Databricks error](https://elementary-community.slack.com/archives/C02CTC89LAX/p1747669372225679?thread_ts=1747669372.225679&cid=C02CTC89LAX)] 2. The way artifacts tables are maintained (cache / delete_and_insert race) - There’s an open issue that duplicates appear in dbt_models and other artifacts tables when models are run in parallel, likely due to a race in the delete_and_insert macro / cache behavior. [[dup bug issue](https://github.com/elementary-data/elementary/issues/1420)] ### How to fix your current dbt_models table As a cleanup step, users are successfully de‑duplicating the artifacts by keeping only the latest row per `unique_id`: sql create or replace table elementary.dbt_models as ( select * from elementary.dbt_models qualify row_number() over (partition by unique_id order by generated_at) = 1 ); This pattern is also used for other artifact tables (dbt_tests, dbt_sources, dbt_exposures, dbt_columns) when they suffer from the same issue. [[dup cleanup workaround](https://github.com/elementary-data/elementary/issues/1420)] After running this, rerun your dbt_models model / pipeline and the MERGE error should stop (until duplicates accumulate again). ### How to reduce / prevent the duplicates Based on the threads: 1. Avoid concurrent artifact updates - Don’t run multiple jobs that trigger Elementary artifact updates at the same time (e.g. multiple dbt run -s elementary, or edr monitor and edr monitor send-report running concurrently from the same project). [[concurrency explanation](https://elementary-community.slack.com/archives/C02CTC89LAX/p1672942935757879?thread_ts=1672942935.757879&cid=C02CTC89LAX)] - If you’re on Airflow or another orchestrator, serialize the Elementary‑related steps. 2. Disable automatic artifact uploads and manage them centrally - One suggested mitigation (especially in concurrent environments) is to turn off automatic artifact uploads from on‑run‑end hooks using disable_dbt_artifacts_autoupload = true, and instead run: bash dbt run --select elementary.edr.dbt_artifacts once per deploy to refresh artifacts in a controlled, non‑concurrent job. [[freshness dup discussion](https://elementary-community.slack.com/archives/C02CTC89LAX/p1754991499736459?thread_ts=1754991499.736459&cid=C02CTC89LAX)] 3. Ensure Elementary + dbt versions are aligned / up to date - Some threads mention changes to dbt_models in newer Elementary dbt packages and improvements around concurrency; upgrading both the Elementary dbt package and the edr CLI to recent aligned versions is recommended, although the duplication race is still tracked as an open issue. [[serverless Databricks thread](https://elementary-community.slack.com/archives/C02CTC89LAX/p1747669372225679?thread_ts=1747669372.225679&cid=C02CTC89LAX); [dup bug issue](https://github.com/elementary-data/elementary/issues/1420)] If, after de‑duping and serializing artifact updates, you still see new duplicates and MERGE errors, the remaining behavior is likely tied to the open race condition mentioned in the GitHub issue; there’s no documented final fix yet in the sources provided.