https://linen.dev logo
Join Slack
Powered by
Hello, I'm trying to implement some tests in my `m...
# community-support
j
Hello, I'm trying to implement some tests in my 
models
but im 
consistently
receiving this kind of error:
Copy code
dbt was unable to infer all dependencies for the model "elementary_volume_anomalies_scout_table1_".
  This typically happens when ref() is placed within a conditional block.
  
  To fix this, add the following hint to the top of the model "elementary_volume_anomalies_table1_":
  
  -- depends_on: {{ ref('monitors_runs') }}
  
  > in macro test_volume_anomalies (macros/edr/tests/test_volume_anomalies.sql)
This happens for 
volume_anomalies
, 
freshness_anomalies
, 
all columns anomalies
, basically everything. I'm using the BigQuery connector. I can't share the data because it's company data, but the issue is exactly as described this thread where @Maayan Salom helped, but unfortunately it was on a private conversation.
version: 2
models:
- name: scout_application_equipment
tests:
- elementary.schema_changes:
tags: ["elementary"]
config:
severity: warn
- elementary.freshness_anomalies:
timestamp_column: "consumer_etl_date_time"
time_bucket:
period: "hour"
count: 3
tags: ["elementary"]
config:
severity: warn
- elementary.volume_anomalies:
timestamp_column: "consumer_etl_date_time"
tags: ["elementary"]
dbt test -s tag:elementary
21:07:10  Running with dbt=1.6.3
21:07:11  Registered adapter: bigquery=1.6.5
21:07:12  Found 159 models, 2 operations, 89 tests, 84 sources, 0 exposures, 0 metrics, 1278 macros, 0 groups, 0 semantic models
21:07:12
21:07:14
21:07:14  Running 1 on-run-start hook
21:07:14  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
21:07:14  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
21:07:14
21:07:14  Concurrency: 5 threads (target='dev')
21:07:14
21:07:14  1 of 3 START test elementary_freshness_anomalies_scout_application_equipment_hour__3__consumer_etl_date_time  [RUN]
21:07:14  2 of 3 START test elementary_schema_changes_scout_application_equipment_ ....... [RUN]
21:07:14  3 of 3 START test elementary_volume_anomalies_scout_application_equipment_consumer_etl_date_time  [RUN]
21:07:15  3 of 3 ERROR elementary_volume_anomalies_scout_application_equipment_consumer_etl_date_time  [ERROR in 0.04s]
21:07:15  2 of 3 ERROR elementary_schema_changes_scout_application_equipment_ ............ [ERROR in 0.04s]
21:07:15  1 of 3 ERROR elementary_freshness_anomalies_scout_application_equipment_hour__3__consumer_etl_date_time  [ERROR in 0.04s]
21:07:15
21:07:15  Running 1 on-run-end hook
21:08:41  1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
21:08:41  1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
21:08:41
21:08:41
21:08:41  Finished running 3 tests, 2 hooks in 0 hours 1 minutes and 28.02 seconds (88.02s).
21:08:41
21:08:41  Completed with 3 errors and 0 warnings:
21:08:41
21:08:41    Compilation Error in test elementary_volume_anomalies_scout_application_equipment_consumer_etl_date_time (models/ingestion_layer/schema.yml)
dbt was unable to infer all dependencies for the model "elementary_volume_anomalies_scout_application_equipment_consumer_etl_date_time".
This typically happens when ref() is placed within a conditional block.
To fix this, add the following hint to the top of the model "elementary_volume_anomalies_scout_application_equipment_consumer_etl_date_time":
-- depends_on: {{ ref('monitors_runs') }}
> in macro test_volume_anomalies (macros/edr/tests/test_volume_anomalies.sql)
> called by test elementary_volume_anomalies_scout_application_equipment_consumer_etl_date_time (models/ingestion_layer/schema.yml)
21:08:41
21:08:41    Compilation Error in test elementary_schema_changes_scout_application_equipment_ (models/ingestion_layer/schema.yml)
dbt was unable to infer all dependencies for the model "elementary_schema_changes_scout_application_equipment_".
This typically happens when ref() is placed within a conditional block.
To fix this, add the following hint to the top of the model "elementary_schema_changes_scout_application_equipment_":
-- depends_on: {{ ref('schema_columns_snapshot') }}
> in macro test_schema_changes (macros/edr/tests/test_schema_changes.sql)
> called by test elementary_schema_changes_scout_application_equipment_ (models/ingestion_layer/schema.yml)
21:08:41
21:08:41    Compilation Error in test elementary_freshness_anomalies_scout_application_equipment_hour__3__consumer_etl_date_time (models/ingestion_layer/schema.yml)
dbt was unable to infer all dependencies for the model "elementary_freshness_anomalies_scout_application_equipment_hour__3__consumer_etl_date_time".
This typically happens when ref() is placed within a conditional block.
To fix this, add the following hint to the top of the model "elementary_freshness_anomalies_scout_application_equipment_hour__3__consumer_etl_date_time":
-- depends_on: {{ ref('monitors_runs') }}
> in macro test_freshness_anomalies (macros/edr/tests/test_freshness_anomalies.sql)
> called by test elementary_freshness_anomalies_scout_application_equipment_hour__3__consumer_etl_date_time (models/ingestion_layer/schema.yml)
21:08:41
21:08:41  Done. PASS=0 WARN=0 ERROR=3 SKIP=0 TOTAL=3
elementary version 0.12.1
@Noy Arie hello! Can you or someone from #C02CTC89LAX please help me out?
n
Hey Can you please run 
dbt deps
and share the output?
j
@Noy Arie The issue was solved when I updated dbt to 
1.7.1
, but now all my reports are empty: 1 - I run 
dbt run -s elementary
, dataset and tables are created successfully as incremental models as expected. 2 - I have 
elementary
in my profile. 3 - I run 
dbt test
4 - 
elementary_test_results
has values. 5 - I run 
edr monitor report
dbt deps
Copy code
dbt deps
20:19:20  Running with dbt=1.7.1
20:19:21  Updating lock file in file path: /Users/jalvi/Desktop/fst-repos/fst-data-systems-pipeline/fst_csw_data_pipeline/package-lock.yml
20:19:21  Installing dbt-labs/dbt_utils
20:19:22  Installed from version 1.1.1
20:19:22  Up to date!
20:19:22  Installing calogica/dbt_expectations
20:19:22  Installed from version 0.8.5
20:19:22  Updated version available: 0.10.1
20:19:22  Installing elementary-data/elementary
20:19:23  Installed from version 0.12.1
20:19:23  Up to date!
20:19:23  Installing calogica/dbt_date
20:19:23  Installed from version 0.7.2
20:19:23  Updated version available: 0.10.0
20:19:23  
20:19:23  Updates available for packages: ['calogica/dbt_expectations', 'calogica/dbt_date']
edr monitor report
Copy code
2023-11-14 15:22:48 — INFO — Running with edr=0.12.0
2023-11-14 15:22:55 — INFO — edr (0.12.0) and Elementary's dbt package (0.12.1) are compatible.
2023-11-14 15:22:57 — INFO — Elementary's database and schema: '"bcs-field-solutions-sbx.elementary"'
2023-11-14 15:22:57 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:10 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models", "macro_args": {"exclude_elementary": true}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:16 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_sources", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:20 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_exposures", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:27 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_runs", "macro_args": {"days_back": 7, "exclude_elementary": true}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:32 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_dbt_models_test_coverage", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:36 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_nodes_depends_on_nodes", "macro_args": {"exclude_elementary": true}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:41 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocation", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-14 15:23:45 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocations_data", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
profiles.yml
Copy code
elementary:
  outputs:
    default:
      type: bigquery
      project: "bcs-field-solutions-sbx"
      dataset: "elementary"
      method: service-account 
      keyfile: /Users/jalvi/Desktop/credentials/serviceKey.json
      threads: 5
all tabs from the report are empty
n
edr report
and 
edr monitor
are two different commands. Can you run the report command only and share the result? Can you also run 
edr monitor debug
?
j
For sure @Noy Arie 
edr report
Copy code
________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - <https://bit.ly/slack-elementary>

2023-11-15 07:50:34 — INFO — Running with edr=0.12.0
2023-11-15 07:50:41 — INFO — edr (0.12.0) and Elementary's dbt package (0.12.1) are compatible.
2023-11-15 07:50:44 — INFO — Elementary's database and schema: '"bcs-field-solutions-sbx.elementary"'
2023-11-15 07:50:44 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:50:55 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models", "macro_args": {"exclude_elementary": true}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:01 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_sources", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:05 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_exposures", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:11 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_runs", "macro_args": {"days_back": 7, "exclude_elementary": true}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:16 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_dbt_models_test_coverage", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:20 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_nodes_depends_on_nodes", "macro_args": {"exclude_elementary": true}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:25 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocation", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
2023-11-15 07:51:29 — INFO — Running dbt --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocations_data", "macro_args": {}} --project-dir /opt/homebrew/lib/python3.11/site-packages/elementary/monitor/dbt_project
edr monitor
I'm not using slack or alerts
Copy code
edr monitor
    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - <https://bit.ly/slack-elementary>

2023-11-15 07:52:13 — INFO — Running with edr=0.12.0
Traceback (most recent call last):
  File "/opt/homebrew/bin/edr", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/elementary/cli/cli.py", line 67, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1635, in invoke
    rv = super().invoke(ctx)
         ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/elementary/monitor/cli.py", line 313, in monitor
    config.validate_monitor()
  File "/opt/homebrew/lib/python3.11/site-packages/elementary/config/config.py", line 213, in validate_monitor
    raise InvalidArgumentsError(
elementary.exceptions.exceptions.InvalidArgumentsError: Either a Slack token and a channel or a Slack webhook is required.
edr monitor debug
Copy code
edr monitor debug
    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - <https://bit.ly/slack-elementary>

2023-11-15 07:53:11 — INFO — Running with edr=0.12.0
Connected to the Elementary db and schema successfully
n
Cool thanks! Is the new report still empty? Just want to make sure, is this your schema and db name: 
bcs-field-solutions-sbx.elementary
? Did you run tests in the past 7 days? And last thing, can you take a look at 
elementary_test_results
table, do you see new records )from the last run)?
27 Views
Open in Slack
PreviousNext
Powered by