Hey all, I;m having issues with a bigquery sink.

[2022-03-23, 14:43:03 UTC] {pipeline.py:84} INFO - sink wrote workunit container-urn:li:container:714a46eb68a1eb8ba6308cf73b33190a-to-urn:li:dataset:(urn:li:dataPlatform:bigquery,dw.analytics_245627.page_views,PROD)
[2022-03-23, 14:43:03 UTC] {pipeline.py:92} ERROR - failed to write record with workunit dw.analytics_245627.page_views with Expecting value: line 1 column 1 (char 0) and info {}
[2022-03-23, 14:43:03 UTC] {pipeline.py:84} INFO - sink wrote workunit container-urn:li:container:714a46eb68a1eb8ba6308cf73b33190a-to-urn:li:dataset:(urn:li:dataPlatform:bigquery,dw.analytics_245627038.sessions,PROD)
[2022-03-23, 14:43:03 UTC] {pipeline.py:92} ERROR - failed to write record with workunit dw.analytics_245627038.sessions with Expecting value: line 1 column 1 (char 0) and info {}
[2022-03-23, 14:43:04 UTC] {pipeline.py:84} INFO - sink wrote workunit container-urn:li:container:714a46eb68a1eb8ba6308cf73b33190a-to-urn:li:dataset:(urn:li:dataPlatform:bigquery,dw.analytics_245627038.user_detail_events,PROD)
[2022-03-23, 14:43:04 UTC] {pipeline.py:84} INFO - sink wrote workunit dw.analytics_245627038.user_detail_events

In this log it's visible that it sometimes error out. How would I best approach this to debug this issue and what could be the reason that some sinks fail?

you can write to file and/or running with debug logs if you can more

datahub --debug ingest ...

Thanks!

Awesome

is there a way to turn debug mode to off while running pipeline.run() ?