Hi, I am trying to ingest data from bigquery. I d...
# ingestion
m
Hi, I am trying to ingest data from bigquery. I deployed datahub with Helm. My ingestion always fail when it's almost done. So I've tried in local on my computer and it worked. The error seems to indicate that was due to a property debug, which is only call by logger in the code. Any tips ?
Copy code
'Pipeline finished with failures\n'
           'Exception ignored in: <function BigQuerySource.__del__ at 0x7f28cd00b310>\n'
           'Traceback (most recent call last):\n'
           '  File '
           '"/tmp/datahub/ingest/venv-c3fa8196-827d-40f8-9fad-fda6aa952cfe/lib/python3.9/site-packages/datahub/ingestion/source/sql/bigquery.py", '
           'line 913, in __del__\n'
           "AttributeError: 'NoneType' object has no attribute 'debug'\n",
           "2022-06-30 09:49:45.230371 [exec_id=c3fa8196-827d-40f8-9fad-fda6aa952cfe] INFO: Failed to execute 'datahub ingest'",
           '2022-06-30 09:49:45.230718 [exec_id=c3fa8196-827d-40f8-9fad-fda6aa952cfe] INFO: Caught exception EXECUTING '
           'task_id=c3fa8196-827d-40f8-9fad-fda6aa952cfe, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 119, in execute_task\n'
           '    self.event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 81, in run_until_complete\n'
           '    return f.result()\n'
           '  File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result\n'
           '    raise self._exception\n'
           '  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step\n'
           '    result = coro.send(None)\n'
           '  File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 115, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}