Craig Neasbey
08/12/2024, 2:07 AMselect artifacts_model, metadata_hash, count(*) total
from `uat_elementary`.`dbt_artifacts_hashes`
group by artifacts_model, metadata_hash
order by total desc
Do you know how the metadata_hash generates the hash please? It appears to use the metadata.json so does that mean we need to add more metadata in our ymls to make them unique?
Do you know how the dbt_artifacts_hashes table gets populated please? I can see the table get read but not written to:
https://github.com/elementary-data/dbt-data-reliability/blob/26e24f94cd7dd27cd237d84be268bc51646e1e32/macros/edr/system/hooks/on_run_end.sql#L9
https://github.com/elementary-data/dbt-data-reliability/blob/master/macros/edr/dbt_artifacts/upload_dbt_artifacts.sql#L18
This does not appear to be a thread issue as only 1 thread produces the duplicates.
Our versions:
dbt-core >= 1.7.0, < 1.8.0
dbt-databricks >= 1.7.0, < 1.8.0
packages:
- package: dbt-labs/dbt_utils
version: 1.2.0
- package: calogica/dbt_expectations
version: 0.10.3
- package: elementary-data/elementary
version: 0.15.2
Thanks for any helpCraig Neasbey
08/12/2024, 11:05 PMCraig Neasbey
08/13/2024, 1:27 AM