After upgrading elementary-data to version dbt

0.11.2

(the same for

0.12.0

) we’re getting the following error

14:41:37  Running with dbt=1.5.1
14:41:37  Unable to do partial parsing because saved manifest not found. Starting full parse.
14:42:03  Found 319 models, 191 tests, 1 snapshot, 1 analysis, 1135 macros, 2 operations, 2 seed files, 58 sources, 104 exposures, 0 metrics, 0 groups
14:42:03  
14:43:02  
14:43:02  Running 1 on-run-start hook
14:43:02  1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
14:43:02  1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
14:43:02  
14:43:02  Concurrency: 4 threads (target='prod')
14:43:02  
14:43:02  1 of 3 START sql view model raw.raw_events_batched ............................. [RUN]
14:43:04  1 of 3 OK created sql view model raw.raw_events_batched ........................ [OK in 2.07s]
14:43:04  2 of 3 START sql incremental model dwh.agg_real_time_metrics ................... [RUN]
14:48:49  2 of 3 OK created sql incremental model dwh.agg_real_time_metrics .............. [OK in 344.05s]
14:48:49  3 of 3 START test agg_real_time_metrics__expect_exposure_columns_exist_in_table  [RUN]
14:48:49  3 of 3 START test agg_real_time_metrics__expect_exposure_columns_exist_in_table  [RUN]
14:48:52  3 of 3 PASS agg_real_time_metrics__expect_exposure_columns_exist_in_table ...... [PASS in 2.98s]
14:48:52  Running 1 on-run-end hook
14:49:10  
14:49:10  Finished running 1 view model, 1 incremental model, 1 test, 1 hook in 0 hours 7 minutes and 6.48 seconds (426.48s).
14:49:10  Encountered an error:
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.IllegalArgumentException: Can not create a Path from an empty string)
pyspark.sql.utils.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.IllegalArgumentException: Can not create a Path from an empty string)

Does anyone else facing with the same error, and what is the solution?

Hey @Almaz Murzabekov, thanks for reaching out! Could you try on the latest

0.13.2

version? What version did you update from?

Hey @Erik Zaadi We’ve tried again with these versions, but wit the same error

elementary-data-0.13.2
elementary package 0.13.0

@michael böhm

fyi: another colleague currently tries to exactly reproduce this. we will update this thread with the (hopefully) conclusive results.

hi @Almaz Murzabekov, mind sharing your

dbt.log

(can be here on in a DM, whichever way suits you best)?

dbt.log.txt

looks like an issue in your hook, if the model is a view you try to run an empty string (which spark seems to not like) you can try changing it to something like

"{{ 'analyze table ' ~ this ~ ' compute statistics' if model.config.materialized != 'view' else 'select 1' }}"

thx Ofek. why would that only be a problem when running elementary with a version newer than ours?

honestly - I'm not sure, this looks like an issue unrelated to elementary? any chance that there was another change you did in parallel? (eg - update dbt/dbt adapter, changes in this hook, etc)

no, update was the only change. but let us try with a modfied hook nevertheless to see the result.

Oh, i was under the impression that you already tried it and it worked out... 😞 Waiting on the results 👀