https://linen.dev logo
Join Slack
Powered by
For basic dbt tests like `not_null` and `unique`, ...
# community-support
j
For basic dbt tests like 
not_null
and 
unique
, a test failure on an upstream model causes a 
SKIP
for downstream models (e.g. if 
model_b
depends on 
model_a
, and a 
unique
test on 
model_a
fails, then 
model_b
will 
SKIP
) What is the expected downstream behavior for an upstream failure with an elementary anomaly test like 
elementary.freshness_anomalies
? I would expect a 
freshness_anomalies
failure in 
model_a
to cause a SKIP in 
model_b
, but that’s not what I’m observing in my project
👀 3
e
Hi @joshuah.kapilivsky, Apologies for the late response and happy new year 🍾 Just making sure here - you are using 
dbt build
, yes? And can you confirm that the tests are indeed failing and not defined as warnings?
j
Hi @Ella Katz, happy new year 🎊 Yes, I am running 
dbt build
and the anomaly test is returning 
FAIL
, not 
WARN
. I wrote up a minimally reproducible example: • The entire DAG is two models: 
upstream_model
and 
downstream_model
. As the names suggest, 
downstream_model
depends on 
upstream_model
(see first screenshot for DAG) • 
upstream_model
is written to trigger a failure in volume anomalies (see second screenshot for volume anomalies graph). Here’s the YML for this model:
Copy code
version: 2

models:
  - name: upstream_model
    description: "This is the upstream model whose test failures should trigger a SKIP in downstream_model"
    tests:
      - elementary.volume_anomalies:
          timestamp_column: "day"
    columns:
      - name: nth_record
        tests:
          - not_null
• Running 
dbt build
returns a 
FAIL
for the volume anomalies test, but 
downstream_model
is still created. You can even see that 
downstream_model
is created before the volume anomalies test is completed:
Copy code
root@467bfdab568d:/dbt# dbt build
16:44:46  Running with dbt=1.6.9
16:44:47  Registered adapter: bigquery=1.6.1
16:44:47  Unable to do partial parsing because a project config has changed
16:44:55  Found 29 models, 2 operations, 2 tests, 0 sources, 0 exposures, 0 metrics, 1032 macros, 0 groups, 0 semantic models
16:44:55
16:44:57
16:44:57  Running 1 on-run-start hook
16:44:57  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
16:44:57  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
16:44:57
16:44:57  Concurrency: 4 threads (target='model_dev')
16:44:57
16:44:57  1 of 31 START sql incremental model elementary.data_monitoring_metrics ......... [RUN]
16:44:57  2 of 31 START sql incremental model elementary.dbt_exposures ................... [RUN]
16:44:57  3 of 31 START sql incremental model elementary.dbt_invocations ................. [RUN]
16:44:57  4 of 31 START sql incremental model elementary.dbt_metrics ..................... [RUN]
16:45:06  3 of 31 OK created sql incremental model elementary.dbt_invocations ............ [MERGE (0.0 rows, 11.3 KiB processed) in 8.72s]
16:45:06  5 of 31 START sql incremental model elementary.dbt_models ...................... [RUN]
16:45:06  1 of 31 OK created sql incremental model elementary.data_monitoring_metrics .... [MERGE (0.0 rows, 240.9 KiB processed) in 8.84s]
16:45:06  6 of 31 START sql incremental model elementary.dbt_run_results ................. [RUN]
16:45:11  2 of 31 OK created sql incremental model elementary.dbt_exposures .............. [MERGE (0.0 rows, 0 processed) in 13.57s]
16:45:11  7 of 31 START sql incremental model elementary.dbt_seeds ....................... [RUN]
16:45:11  4 of 31 OK created sql incremental model elementary.dbt_metrics ................ [MERGE (0.0 rows, 0 processed) in 13.83s]
16:45:11  8 of 31 START sql incremental model elementary.dbt_snapshots ................... [RUN]
16:45:14  6 of 31 OK created sql incremental model elementary.dbt_run_results ............ [MERGE (0.0 rows, 655.8 KiB processed) in 8.06s]
16:45:14  9 of 31 START sql incremental model elementary.dbt_source_freshness_results .... [RUN]
16:45:22  5 of 31 OK created sql incremental model elementary.dbt_models ................. [MERGE (0.0 rows, 19.1 KiB processed) in 15.60s]
16:45:22  10 of 31 START sql incremental model elementary.dbt_sources .................... [RUN]
16:45:22  9 of 31 OK created sql incremental model elementary.dbt_source_freshness_results  [MERGE (0.0 rows, 0 processed) in 7.84s]
16:45:22  11 of 31 START sql incremental model elementary.dbt_tests ...................... [RUN]
16:45:23  7 of 31 OK created sql incremental model elementary.dbt_seeds .................. [MERGE (0.0 rows, 0 processed) in 12.14s]
16:45:23  12 of 31 START sql incremental model elementary.elementary_test_results ........ [RUN]
16:45:24  8 of 31 OK created sql incremental model elementary.dbt_snapshots .............. [MERGE (0.0 rows, 0 processed) in 12.73s]
16:45:24  13 of 31 START sql table model elementary.metadata ............................. [RUN]
16:45:28  13 of 31 OK created sql table model elementary.metadata ........................ [CREATE TABLE (1.0 rows, 0 processed) in 3.99s]
16:45:28  14 of 31 START sql incremental model elementary.schema_columns_snapshot ........ [RUN]
16:45:31  12 of 31 OK created sql incremental model elementary.elementary_test_results ... [MERGE (0.0 rows, 179.7 KiB processed) in 7.83s]
16:45:31  15 of 31 START sql table model demo_rwde_obs.upstream_model .................... [RUN]
16:45:34  10 of 31 OK created sql incremental model elementary.dbt_sources ............... [MERGE (0.0 rows, 0 processed) in 11.98s]
16:45:34  16 of 31 START sql view model elementary.job_run_results ....................... [RUN]
16:45:35  16 of 31 OK created sql view model elementary.job_run_results .................. [CREATE VIEW (0 processed) in 0.96s]
16:45:35  17 of 31 START sql view model elementary.metrics_anomaly_score ................. [RUN]
16:45:35  14 of 31 OK created sql incremental model elementary.schema_columns_snapshot ... [MERGE (0.0 rows, 0 processed) in 6.95s]
16:45:35  18 of 31 START sql view model elementary.monitors_runs ......................... [RUN]
16:45:35  15 of 31 OK created sql table model demo_rwde_obs.upstream_model ............... [CREATE TABLE (14.5k rows, 0 processed) in 4.07s]
16:45:35  19 of 31 START sql view model elementary.model_run_results ..................... [RUN]
16:45:36  17 of 31 OK created sql view model elementary.metrics_anomaly_score ............ [CREATE VIEW (0 processed) in 1.04s]
16:45:36  20 of 31 START sql view model elementary.snapshot_run_results .................. [RUN]
16:45:36  18 of 31 OK created sql view model elementary.monitors_runs .................... [CREATE VIEW (0 processed) in 0.85s]
16:45:36  21 of 31 START sql view model elementary.alerts_anomaly_detection .............. [RUN]
16:45:36  19 of 31 OK created sql view model elementary.model_run_results ................ [CREATE VIEW (0 processed) in 0.92s]
16:45:36  22 of 31 START sql view model elementary.alerts_dbt_tests ...................... [RUN]
16:45:37  11 of 31 OK created sql incremental model elementary.dbt_tests ................. [MERGE (0.0 rows, 1.7 KiB processed) in 14.51s]
16:45:37  23 of 31 START sql view model elementary.alerts_schema_changes ................. [RUN]
16:45:37  20 of 31 OK created sql view model elementary.snapshot_run_results ............. [CREATE VIEW (0 processed) in 0.95s]
16:45:37  24 of 31 START sql incremental model elementary.test_result_rows ............... [RUN]
16:45:37  21 of 31 OK created sql view model elementary.alerts_anomaly_detection ......... [CREATE VIEW (0 processed) in 1.08s]
16:45:37  22 of 31 OK created sql view model elementary.alerts_dbt_tests ................. [CREATE VIEW (0 processed) in 0.94s]
16:45:37  25 of 31 START sql view model elementary.alerts_dbt_source_freshness ........... [RUN]
16:45:37  26 of 31 START test not_null_upstream_model_nth_record ......................... [RUN]
16:45:38  23 of 31 OK created sql view model elementary.alerts_schema_changes ............ [CREATE VIEW (0 processed) in 1.13s]
16:45:38  25 of 31 OK created sql view model elementary.alerts_dbt_source_freshness ...... [CREATE VIEW (0 processed) in 0.99s]
16:45:38  27 of 31 START sql view model elementary.anomaly_threshold_sensitivity ......... [RUN]
16:45:38  28 of 31 START test elementary_volume_anomalies_upstream_model_day ............. [RUN]
16:45:39  27 of 31 OK created sql view model elementary.anomaly_threshold_sensitivity .... [CREATE VIEW (0 processed) in 1.02s]
16:45:39  29 of 31 START sql view model elementary.dbt_artifacts_hashes .................. [RUN]
16:45:39  Elementary: Started running data monitors on: TL-8GNR8WCZD4NX2KHR92X6.DEMO_RWDE_OBS.UPSTREAM_MODEL
16:45:40  29 of 31 OK created sql view model elementary.dbt_artifacts_hashes ............. [CREATE VIEW (0 processed) in 0.98s]
16:45:40  30 of 31 START sql view model elementary.alerts_dbt_models ..................... [RUN]
16:45:41  30 of 31 OK created sql view model elementary.alerts_dbt_models ................ [CREATE VIEW (0 processed) in 1.16s]
16:45:43  24 of 31 OK created sql incremental model elementary.test_result_rows .......... [MERGE (0.0 rows, 720.6 KiB processed) in 6.81s]
16:45:43  26 of 31 PASS not_null_upstream_model_nth_record ............................... [PASS in 6.69s]
16:45:44  31 of 31 START sql table model demo_rwde_obs.downstream_model .................. [RUN]
16:45:46  Elementary: Finished running data monitors on: TL-8GNR8WCZD4NX2KHR92X6.DEMO_RWDE_OBS.UPSTREAM_MODEL
16:45:48  31 of 31 OK created sql table model demo_rwde_obs.downstream_model ............. [CREATE TABLE (14.5k rows, 340.7 KiB processed) in 4.02s]
16:45:52  28 of 31 FAIL 1 elementary_volume_anomalies_upstream_model_day ................. [FAIL 1 in 14.14s]
16:45:52
16:45:52  Running 1 on-run-end hook
16:46:14  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
16:46:14  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
16:46:14
16:46:14
16:46:14  Finished running 14 incremental models, 3 table models, 12 view models, 2 tests, 2 hooks in 0 hours 1 minutes and 19.03 seconds (79.03s).
16:46:14
16:46:14  Completed with 1 error and 0 warnings:
16:46:14
16:46:14  Failure in test elementary_volume_anomalies_upstream_model_day (models/data/upstream_model.yml)
16:46:14    Got 1 result, configured to fail if != 0
16:46:14
16:46:14    compiled Code at target/compiled/rwde_dbt_demos/models/data/upstream_model.yml/elementary_volume_anomalies_upstream_model_day.sql
16:46:14
16:46:14    See test failures:
  -------------------------------------------------------------------------------------------------------------
  select * from `tl-8gnr8wczd4nx2khr92x6`.`rwde_dbt_demo_test`.`elementary_volume_anomalies_upstream_model_day`
  -------------------------------------------------------------------------------------------------------------
16:46:14
16:46:14  Done. PASS=30 WARN=0 ERROR=1 SKIP=0 TOTAL=31
I think the issue is that DBT’s DAG parsing isn’t registering 
downstream_model
as a child of the volume anomalies test. In the 
graph_summary.json
artifact (docs), under 
with_test_edges
, the 
not_null_upstream_model_nth_record
test identifies 
downstream_model
(node 0) as a child, but 
elementary_volume_anomalies_upstream_model_day
does not. (See last two lines below)
Copy code
{
  "_invocation_id": "80db63d1-b29c-4f85-b1e9-ed7b639829f6",
  "linked": {
    "0": {
      "name": "model.rwde_dbt_demos.downstream_model",
      "type": "model"
    },
    "1": {
      "name": "model.rwde_dbt_demos.upstream_model",
      "type": "model",
      "succ": [
        0,
        31,
        32
      ]
    },
    "2": {
      "name": "model.elementary.alerts_anomaly_detection",
      "type": "model"
    },
    "3": {
      "name": "model.elementary.elementary_test_results",
      "type": "model",
      "succ": [
        2,
        10,
        11,
        26
      ]
    },
    "4": {
      "name": "model.elementary.alerts_dbt_models",
      "type": "model"
    },
    "5": {
      "name": "model.elementary.model_run_results",
      "type": "model",
      "succ": [
        4
      ]
    },
    "6": {
      "name": "model.elementary.snapshot_run_results",
      "type": "model",
      "succ": [
        4
      ]
    },
    "7": {
      "name": "model.elementary.alerts_dbt_source_freshness",
      "type": "model"
    },
    "8": {
      "name": "model.elementary.dbt_source_freshness_results",
      "type": "model",
      "succ": [
        7
      ]
    },
    "9": {
      "name": "model.elementary.dbt_sources",
      "type": "model",
      "succ": [
        7,
        16
      ]
    },
    "10": {
      "name": "model.elementary.alerts_dbt_tests",
      "type": "model"
    },
    "11": {
      "name": "model.elementary.alerts_schema_changes",
      "type": "model"
    },
    "12": {
      "name": "model.elementary.anomaly_threshold_sensitivity",
      "type": "model"
    },
    "13": {
      "name": "model.elementary.metrics_anomaly_score",
      "type": "model",
      "succ": [
        12
      ]
    },
    "14": {
      "name": "model.elementary.data_monitoring_metrics",
      "type": "model",
      "succ": [
        13,
        28,
        32
      ]
    },
    "15": {
      "name": "model.elementary.schema_columns_snapshot",
      "type": "model"
    },
    "16": {
      "name": "model.elementary.dbt_artifacts_hashes",
      "type": "model"
    },
    "17": {
      "name": "model.elementary.dbt_models",
      "type": "model",
      "succ": [
        16,
        5
      ]
    },
    "18": {
      "name": "model.elementary.dbt_tests",
      "type": "model",
      "succ": [
        16
      ]
    },
    "19": {
      "name": "model.elementary.dbt_snapshots",
      "type": "model",
      "succ": [
        16,
        6
      ]
    },
    "20": {
      "name": "model.elementary.dbt_metrics",
      "type": "model",
      "succ": [
        16
      ]
    },
    "21": {
      "name": "model.elementary.dbt_exposures",
      "type": "model",
      "succ": [
        16
      ]
    },
    "22": {
      "name": "model.elementary.dbt_seeds",
      "type": "model",
      "succ": [
        16
      ]
    },
    "23": {
      "name": "model.elementary.dbt_invocations",
      "type": "model",
      "succ": [
        25
      ]
    },
    "24": {
      "name": "model.elementary.dbt_run_results",
      "type": "model",
      "succ": [
        5,
        6,
        32
      ]
    },
    "25": {
      "name": "model.elementary.job_run_results",
      "type": "model"
    },
    "26": {
      "name": "model.elementary.test_result_rows",
      "type": "model"
    },
    "27": {
      "name": "model.elementary.metadata",
      "type": "model"
    },
    "28": {
      "name": "model.elementary.monitors_runs",
      "type": "model",
      "succ": [
        32
      ]
    },
    "29": {
      "name": "operation.elementary.elementary-on-run-start-0",
      "type": "operation"
    },
    "30": {
      "name": "operation.elementary.elementary-on-run-end-0",
      "type": "operation"
    },
    "31": {
      "name": "test.rwde_dbt_demos.not_null_upstream_model_nth_record.e8e6cde494",
      "type": "test"
    },
    "32": {
      "name": "test.rwde_dbt_demos.elementary_volume_anomalies_upstream_model_day.c43182b650",
      "type": "test"
    }
  },
  "with_test_edges": {
    "0": {
      "name": "model.rwde_dbt_demos.downstream_model",
      "type": "model"
    },
    "1": {
      "name": "model.rwde_dbt_demos.upstream_model",
      "type": "model",
      "succ": [
        0,
        31,
        32
      ]
    },
    "2": {
      "name": "model.elementary.alerts_anomaly_detection",
      "type": "model"
    },
    "3": {
      "name": "model.elementary.elementary_test_results",
      "type": "model",
      "succ": [
        2,
        10,
        11,
        26
      ]
    },
    "4": {
      "name": "model.elementary.alerts_dbt_models",
      "type": "model"
    },
    "5": {
      "name": "model.elementary.model_run_results",
      "type": "model",
      "succ": [
        4
      ]
    },
    "6": {
      "name": "model.elementary.snapshot_run_results",
      "type": "model",
      "succ": [
        4
      ]
    },
    "7": {
      "name": "model.elementary.alerts_dbt_source_freshness",
      "type": "model"
    },
    "8": {
      "name": "model.elementary.dbt_source_freshness_results",
      "type": "model",
      "succ": [
        7
      ]
    },
    "9": {
      "name": "model.elementary.dbt_sources",
      "type": "model",
      "succ": [
        7,
        16
      ]
    },
    "10": {
      "name": "model.elementary.alerts_dbt_tests",
      "type": "model"
    },
    "11": {
      "name": "model.elementary.alerts_schema_changes",
      "type": "model"
    },
    "12": {
      "name": "model.elementary.anomaly_threshold_sensitivity",
      "type": "model"
    },
    "13": {
      "name": "model.elementary.metrics_anomaly_score",
      "type": "model",
      "succ": [
        12
      ]
    },
    "14": {
      "name": "model.elementary.data_monitoring_metrics",
      "type": "model",
      "succ": [
        13,
        28,
        32
      ]
    },
    "15": {
      "name": "model.elementary.schema_columns_snapshot",
      "type": "model"
    },
    "16": {
      "name": "model.elementary.dbt_artifacts_hashes",
      "type": "model"
    },
    "17": {
      "name": "model.elementary.dbt_models",
      "type": "model",
      "succ": [
        16,
        5
      ]
    },
    "18": {
      "name": "model.elementary.dbt_tests",
      "type": "model",
      "succ": [
        16
      ]
    },
    "19": {
      "name": "model.elementary.dbt_snapshots",
      "type": "model",
      "succ": [
        16,
        6
      ]
    },
    "20": {
      "name": "model.elementary.dbt_metrics",
      "type": "model",
      "succ": [
        16
      ]
    },
    "21": {
      "name": "model.elementary.dbt_exposures",
      "type": "model",
      "succ": [
        16
      ]
    },
    "22": {
      "name": "model.elementary.dbt_seeds",
      "type": "model",
      "succ": [
        16
      ]
    },
    "23": {
      "name": "model.elementary.dbt_invocations",
      "type": "model",
      "succ": [
        25
      ]
    },
    "24": {
      "name": "model.elementary.dbt_run_results",
      "type": "model",
      "succ": [
        5,
        6,
        32
      ]
    },
    "25": {
      "name": "model.elementary.job_run_results",
      "type": "model"
    },
    "26": {
      "name": "model.elementary.test_result_rows",
      "type": "model"
    },
    "27": {
      "name": "model.elementary.metadata",
      "type": "model"
    },
    "28": {
      "name": "model.elementary.monitors_runs",
      "type": "model",
      "succ": [
        32
      ]
    },
    "29": {
      "name": "operation.elementary.elementary-on-run-start-0",
      "type": "operation"
    },
    "30": {
      "name": "operation.elementary.elementary-on-run-end-0",
      "type": "operation"
    },
    "31": {
      "name": "test.rwde_dbt_demos.not_null_upstream_model_nth_record.e8e6cde494",
      "type": "test",
      "succ": [
        0
      ]
    },
    "32": {
      "name": "test.rwde_dbt_demos.elementary_volume_anomalies_upstream_model_day.c43182b650",
      "type": "test"
    }
  }
}
I think this is why failures in anomaly tests are not causing downstream SKIPs, but I don’t know enough about how DBT constructs the DAG to know how to fix it. What do you think?
e
That's interesting 🤔 Elementary tests are just dbt generic tests to they should work the same as every other test, but we will most definitely look into this! If it's not too hard can you check if you get the same result with 1 thread instead of 4?
And also thanks for the thorough research and example!!!
j
Absolutely. After changing to 1 thread, it looks like the 
FAIL
in the volume anomalies test still doesn’t create a SKIP in `downstream_model`:
Copy code
root@91b775794082:/dbt# dbt build --threads 1
20:06:31  Running with dbt=1.6.9
20:06:33  Registered adapter: bigquery=1.6.1
20:06:37  Found 29 models, 2 operations, 2 tests, 0 sources, 0 exposures, 0 metrics, 1032 macros, 0 groups, 0 semantic models
20:06:37
20:06:41
20:06:41  Running 1 on-run-start hook
20:06:41  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
20:06:41  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
20:06:41
20:06:41  Concurrency: 1 threads (target='model_dev')
20:06:41
20:06:41  1 of 31 START sql incremental model elementary.data_monitoring_metrics ......... [RUN]
20:06:50  1 of 31 OK created sql incremental model elementary.data_monitoring_metrics .... [MERGE (0.0 rows, 251.5 KiB processed) in 8.32s]
20:06:50  2 of 31 START sql incremental model elementary.dbt_exposures ................... [RUN]
20:07:04  2 of 31 OK created sql incremental model elementary.dbt_exposures .............. [MERGE (0.0 rows, 0 processed) in 14.48s]
20:07:04  3 of 31 START sql incremental model elementary.dbt_invocations ................. [RUN]
20:07:12  3 of 31 OK created sql incremental model elementary.dbt_invocations ............ [MERGE (0.0 rows, 11.7 KiB processed) in 7.51s]
20:07:12  4 of 31 START sql incremental model elementary.dbt_metrics ..................... [RUN]
20:07:28  4 of 31 OK created sql incremental model elementary.dbt_metrics ................ [MERGE (0.0 rows, 0 processed) in 16.24s]
20:07:28  5 of 31 START sql incremental model elementary.dbt_models ...................... [RUN]
20:07:46  5 of 31 OK created sql incremental model elementary.dbt_models ................. [MERGE (0.0 rows, 19.1 KiB processed) in 18.25s]
20:07:46  6 of 31 START sql incremental model elementary.dbt_run_results ................. [RUN]
20:07:54  6 of 31 OK created sql incremental model elementary.dbt_run_results ............ [MERGE (0.0 rows, 717.5 KiB processed) in 7.89s]
20:07:54  7 of 31 START sql incremental model elementary.dbt_seeds ....................... [RUN]
20:08:30  7 of 31 OK created sql incremental model elementary.dbt_seeds .................. [MERGE (0.0 rows, 0 processed) in 35.47s]
20:08:30  8 of 31 START sql incremental model elementary.dbt_snapshots ................... [RUN]
20:08:43  8 of 31 OK created sql incremental model elementary.dbt_snapshots .............. [MERGE (0.0 rows, 0 processed) in 12.89s]
20:08:43  9 of 31 START sql incremental model elementary.dbt_source_freshness_results .... [RUN]
20:08:50  9 of 31 OK created sql incremental model elementary.dbt_source_freshness_results  [MERGE (0.0 rows, 0 processed) in 7.89s]
20:08:50  10 of 31 START sql incremental model elementary.dbt_sources .................... [RUN]
20:09:04  10 of 31 OK created sql incremental model elementary.dbt_sources ............... [MERGE (0.0 rows, 0 processed) in 13.78s]
20:09:04  11 of 31 START sql incremental model elementary.dbt_tests ...................... [RUN]
20:09:20  11 of 31 OK created sql incremental model elementary.dbt_tests ................. [MERGE (0.0 rows, 1.7 KiB processed) in 15.98s]
20:09:20  12 of 31 START sql incremental model elementary.elementary_test_results ........ [RUN]
20:09:27  12 of 31 OK created sql incremental model elementary.elementary_test_results ... [MERGE (0.0 rows, 186.3 KiB processed) in 7.28s]
20:09:28  13 of 31 START sql table model elementary.metadata ............................. [RUN]
20:09:33  13 of 31 OK created sql table model elementary.metadata ........................ [CREATE TABLE (1.0 rows, 0 processed) in 5.07s]
20:09:33  14 of 31 START sql incremental model elementary.schema_columns_snapshot ........ [RUN]
20:09:41  14 of 31 OK created sql incremental model elementary.schema_columns_snapshot ... [MERGE (0.0 rows, 0 processed) in 8.29s]
20:09:41  15 of 31 START sql table model demo_rwde_obs.upstream_model .................... [RUN]
20:09:46  15 of 31 OK created sql table model demo_rwde_obs.upstream_model ............... [CREATE TABLE (14.5k rows, 0 processed) in 4.66s]
20:09:46  16 of 31 START sql view model elementary.metrics_anomaly_score ................. [RUN]
20:09:47  16 of 31 OK created sql view model elementary.metrics_anomaly_score ............ [CREATE VIEW (0 processed) in 1.11s]
20:09:47  17 of 31 START sql view model elementary.monitors_runs ......................... [RUN]
20:09:48  17 of 31 OK created sql view model elementary.monitors_runs .................... [CREATE VIEW (0 processed) in 0.92s]
20:09:48  18 of 31 START sql view model elementary.job_run_results ....................... [RUN]
20:09:49  18 of 31 OK created sql view model elementary.job_run_results .................. [CREATE VIEW (0 processed) in 0.93s]
20:09:49  19 of 31 START sql view model elementary.model_run_results ..................... [RUN]
20:09:50  19 of 31 OK created sql view model elementary.model_run_results ................ [CREATE VIEW (0 processed) in 1.03s]
20:09:50  20 of 31 START sql view model elementary.snapshot_run_results .................. [RUN]
20:09:51  20 of 31 OK created sql view model elementary.snapshot_run_results ............. [CREATE VIEW (0 processed) in 1.04s]
20:09:51  21 of 31 START sql view model elementary.alerts_dbt_source_freshness ........... [RUN]
20:09:52  21 of 31 OK created sql view model elementary.alerts_dbt_source_freshness ...... [CREATE VIEW (0 processed) in 1.05s]
20:09:52  22 of 31 START sql view model elementary.dbt_artifacts_hashes .................. [RUN]
20:09:53  22 of 31 OK created sql view model elementary.dbt_artifacts_hashes ............. [CREATE VIEW (0 processed) in 1.51s]
20:09:53  23 of 31 START sql view model elementary.alerts_anomaly_detection .............. [RUN]
20:09:54  23 of 31 OK created sql view model elementary.alerts_anomaly_detection ......... [CREATE VIEW (0 processed) in 0.83s]
20:09:54  24 of 31 START sql view model elementary.alerts_dbt_tests ...................... [RUN]
20:09:56  24 of 31 OK created sql view model elementary.alerts_dbt_tests ................. [CREATE VIEW (0 processed) in 1.07s]
20:09:56  25 of 31 START sql view model elementary.alerts_schema_changes ................. [RUN]
20:09:56  25 of 31 OK created sql view model elementary.alerts_schema_changes ............ [CREATE VIEW (0 processed) in 0.88s]
20:09:56  26 of 31 START sql incremental model elementary.test_result_rows ............... [RUN]
20:10:03  26 of 31 OK created sql incremental model elementary.test_result_rows .......... [MERGE (0.0 rows, 739.2 KiB processed) in 6.73s]
20:10:03  27 of 31 START test not_null_upstream_model_nth_record ......................... [RUN]
20:10:11  27 of 31 PASS not_null_upstream_model_nth_record ............................... [PASS in 7.63s]
20:10:11  28 of 31 START sql view model elementary.anomaly_threshold_sensitivity ......... [RUN]
20:10:12  28 of 31 OK created sql view model elementary.anomaly_threshold_sensitivity .... [CREATE VIEW (0 processed) in 0.96s]
20:10:12  29 of 31 START test elementary_volume_anomalies_upstream_model_day ............. [RUN]
20:10:14  Elementary: Started running data monitors on: TL-8GNR8WCZD4NX2KHR92X6.DEMO_RWDE_OBS.UPSTREAM_MODEL
20:10:24  Elementary: Finished running data monitors on: TL-8GNR8WCZD4NX2KHR92X6.DEMO_RWDE_OBS.UPSTREAM_MODEL
20:10:30  29 of 31 FAIL 1 elementary_volume_anomalies_upstream_model_day ................. [FAIL 1 in 18.31s]
20:10:30  30 of 31 START sql view model elementary.alerts_dbt_models ..................... [RUN]
20:10:31  30 of 31 OK created sql view model elementary.alerts_dbt_models ................ [CREATE VIEW (0 processed) in 1.31s]
20:10:31  31 of 31 START sql table model demo_rwde_obs.downstream_model .................. [RUN]
20:10:36  31 of 31 OK created sql table model demo_rwde_obs.downstream_model ............. [CREATE TABLE (14.5k rows, 340.9 KiB processed) in 4.23s]
20:10:36
20:10:36  Running 1 on-run-end hook
20:10:59  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
20:10:59  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
20:10:59
20:10:59
20:10:59  Finished running 14 incremental models, 3 table models, 12 view models, 2 tests, 2 hooks in 0 hours 4 minutes and 21.68 seconds (261.68s).
20:10:59
20:10:59  Completed with 1 error and 0 warnings:
20:10:59
20:10:59  Failure in test elementary_volume_anomalies_upstream_model_day (models/data/upstream_model.yml)
20:10:59    Got 1 result, configured to fail if != 0
20:10:59
20:10:59    compiled Code at target/compiled/rwde_dbt_demos/models/data/upstream_model.yml/elementary_volume_anomalies_upstream_model_day.sql
20:10:59
20:10:59    See test failures:
  -------------------------------------------------------------------------------------------------------------
  select * from `tl-8gnr8wczd4nx2khr92x6`.`rwde_dbt_demo_test`.`elementary_volume_anomalies_upstream_model_day`
  -------------------------------------------------------------------------------------------------------------
20:10:59
20:10:59  Done. PASS=30 WARN=0 ERROR=1 SKIP=0 TOTAL=31
The 
graph_summary.json
looks the same as well. 
not_null_upstream_model_nth_record
has 
downstream_model
as an explicit child, but 
elementary_volume_anomalies_upstream_model_day
does not
In case it is helpful to reproduce on your end, here is my model definition for 
upstream_model
. It creates gaussian distributed number of records per day with mean 500, and then hard codes the record count for the last day to 40 to trigger the volume anomaly
Copy code
{% set training_days_back = 30 %}
{% set mu_records_per_day = 500 %}
{% set sigma_records_per_day = 5 %}
{% set anomalous_volume = 40 %}
{% set training_prediction_accuracy = 0.80 %}
{% set anomalous_prediction_accuracy = 0.2 %}

WITH date_spine AS (
  SELECT day
       , ROUND({{ mu_records_per_day }} + {{ sigma_records_per_day }} *(SQRT(2*ABS(LOG(RAND()))))*COS(2*ACOS(-1)*RAND())) AS n_records
  FROM UNNEST(
    GENERATE_DATE_ARRAY(
      DATE_SUB(CURRENT_DATE(), INTERVAL {{ training_days_back }} DAY),
      DATE_SUB(CURRENT_DATE(), INTERVAL 2 DAY),
      INTERVAL 1 DAY
    )
  ) day
), individual_records AS (
  SELECT day
      , GENERATE_ARRAY(1, n_records) AS record_array
  FROM date_spine
), good_data AS (
  SELECT day
      , nth_record
      , CAST(RAND() <= {{ training_prediction_accuracy }} AS INT) AS correct_prediction
  FROM individual_records, individual_records.record_array AS nth_record
), anomalous_data AS (
  SELECT DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY) AS day
       , nth_record
       , CAST(RAND() <= {{ anomalous_prediction_accuracy }} AS INT) AS correct_prediction
  FROM UNNEST(GENERATE_ARRAY(1, {{ anomalous_volume }})) AS nth_record
)
SELECT *
FROM good_data
UNION ALL
SELECT *
FROM anomalous_data
ORDER BY day, nth_record
e
yes very useful, thank you 🙏 We will look into this
lastly @joshuah.kapilivsky can you tell me what Elementary dbt package version you're using?
j
Yep, looks like we’re a bit behind, using 
0.10.3
We’re using elementary in several of our production projects, so there’s a bit of inertia for upgrading with each new version, but we’re probably due for an update on our end, especially if it provides this SKIP functionality
e
Not sure yet but will check if it's related
Hi @joshuah.kapilivsky, Happy to update that we found the issue! 🐕 In short, since our anomaly tests are referencing Elementary tables, and those weren't build yet on that 
dbt build
execution, the tests could not run yet, and something in the calculation of the dependency graph was incorrect. To fix the issue those explicit references needed to be removed, so that the test will only depend on 
upstream_moedl
and nothing else. The fix will be released in the following version, but you can view the fix here if you're interested, and you can install our dbt package using the commit hash in the PR as a temporary fix until we release the version. Thank you so much, for both surfacing the issue and providing all the information needed to help us investigate 🙏
j
Amazing, thanks for tracking this down! I really appreciate it elementary is a really great tool that makes my job so much easier, excited to see what you all add next!
e
Happy to hear! BTW, if you have any thoughts on features we should add you're always welcome to schedule a call 🙂
thankyou 1
33 Views
Open in Slack
PreviousNext
Powered by