https://linen.dev logo
Join Slack
Powered by
Good afternoon <#C02CTC89LAX|> Recently (after up...
# community-support
g
Good afternoon #C02CTC89LAX Recently (after upgrading to elementary 13.2) we are getting a python exception now and then when the dbt posthook should start running (it is hook 2 of 2). It appears randomly on runs and then after some runs it will disappear again but come back a couple of days later. int() argument must be a string, a bytes-like object or a real number, not 'NoneType' Any clues on a potential cause and fix for this problem? Thanks of any insights! 111759 1 of 2 START hook: MXA_DI.on-run-end.0 ......................................... [RUN] 111759 1 of 2 OK hook: MXA_DI.on-run-end.0 ............................................ [OK in 0.00s] 111814 Database error while running on-run-end 111814 111814 Finished running 4 incremental models, 18 view models, 28 table models, 2 hooks in 0 hours 15 minutes and 37.89 seconds (937.89s). 111814 111814 Completed with 1 error and 0 warnings: 111814 111814 on-run-end failed, error: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
d
Hey @Gijs Jansen, Can you share dbg.log file ?
g
I have shared the log via DM
For transparency sake, putting the logs where the error occurs below 
[0m_11:18:11.411869_ [debug] [MainThread]: On master: /* {"app": "dbt", "dbt_version": "1.6.9", "dbt_databricks_version": "1.6.6", "databricks_sql_connector_version": "2.9.3", "profile_name": "MXA_DI", "target_name": "prod", "connection_name": "master"} */
delete from 
mxa_di_elementary
.
dbt_models
where metadata_hash is null or metadata_hash in (select metadata_hash from 
mxa_di_elementary
.
dbt_models__tmp_20240201111806997989
); [0m`_111814.055086_ [debug] [MainThread]: SQL status: OK in 2.640000104904175 seconds` [0m`_111814.059745_ [info ] [MainThread]: Database error while running on-run-end` [0m`_111814.061082_ [debug] [MainThread]: On master: ROLLBACK` [0m`_111814.062176_ [debug] [MainThread]: Databricks adapter: NotImplemented: rollback` [0m`_111814.063220_ [debug] [MainThread]: On master: Close` [0m`_111814.117846_ [debug] [MainThread]: Connection 'master' was properly closed.` [0m`_111814.119050_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_fct_ga4_doms_session' was properly closed.` [0m`_111814.120088_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_fct_ga4_doms_pageview' was properly closed.` [0m`_111814.121059_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_dim_ga4_page_type' was properly closed.` [0m`_111814.122001_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_dim_ga4_doms_traffic_source' was properly closed.` [0m`_111814.122948_ [debug] [MainThread]: Connection 'model.MXA_DI.fct_ga4_doms_ecommerce_event_item' was properly closed.` [0m`_111814.123911_ [debug] [MainThread]: Connection 'model.MXA_DI.fct_ga4_doms_session_funnel' was properly closed.` [0m`_111814.124856_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_fct_ga4_doms_session_funnel' was properly closed.` [0m`_111814.125786_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_fct_ga4_doms_session_aggregation' was properly closed.` [0m`_111814.126711_ [debug] [MainThread]: Connection 'model.MXA_DI.rpt_fct_ga4_doms_ecommerce_event_item' was properly closed.` [0m`_111814.127651_ [debug] [MainThread]: Connection 'model.MXA_DI.fct_ga4_doms_pageview' was properly closed.` [0m`_111814.128648_ [debug] [MainThread]: Connection 'model.MXA_DI.fct_ga4_doms_session' was properly closed.` [0m`_111814.129585_ [debug] [MainThread]: Connection 'model.MXA_DI.fct_ga4_doms_session_aggregation' was properly closed.` [0m`_111814.130859_ [info ] [MainThread]:` [0m`_111814.131987_ [info ] [MainThread]: Finished running 4 incremental models, 18 view models, 28 table models, 2 hooks in 0 hours 15 minutes and 37.89 seconds (937.89s).` [0m`_111814.142764_ [debug] [MainThread]: Command end result` [0m`_111814.293990_ [info ] [MainThread]:` [0m`_111814.295282_ [info ] [MainThread]: [31mCompleted with 1 error and 0 warnings:[0m` [0m`_111814.296330_ [info ] [MainThread]:` [0m`_111814.297333_ [error] [MainThread]: on-run-end failed, error:` int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
d
Interesting. I'm wondering if upgrading the dbt-core version will solve it. Can you try ?
g
I know that on the latest dbt-databricks/dbt-core combo the same can occur. Because the first time I started to run into this exception was when I bumped both elementary (from 12.1 to 13.2) and dbt-databricks to the latest version. Then I ran into multiple issues (solved by running elementary with --full-refresh) but this one stayed. Then we rolled back to 1.6 and the issue disappeared. But now it's back on 1.6
n
Hey @Gijs Jansen, jumping in here 🙂 I’ll try to dig deeper, but just to understand you fully - are you saying it’s happening in the dbt post run hook or the elementary one? Also - can you try and think if there are any differences between the run? For example different vars, different envs, maybe in one of them you’re running an additional command? Asking just since you’re saying this happens “from time to time” and not consistently, which might be caused by various of reasons Thanks!
g
Thank you Noa! It seems like bumping dbt-databricks and elementary to latest has solved it.
🙌 1
31 Views
Open in Slack
PreviousNext
Powered by