lively-businessperson-84910
04/18/2023, 8:09 PM~~~~ Execution Summary - RUN_INGEST ~~~~
Execution finished with errors.
{'exec_id': 'b7fe3905-6cb1-4c7b-9fb7-06f2e516826a',
'infos': ['2023-04-18 19:28:43.814912 INFO: Starting execution for task with name=RUN_INGEST',
"2023-04-18 19:28:49.881382 INFO: Failed to execute 'datahub ingest'",
'2023-04-18 19:28:49.883130 INFO: Caught exception EXECUTING task_id=b7fe3905-6cb1-4c7b-9fb7-06f2e516826a, name=RUN_INGEST, '
'stacktrace=Traceback (most recent call last):\n'
' File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 122, in execute_task\n'
' task_event_loop.run_until_complete(task_future)\n'
' File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
' return future.result()\n'
' File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 231, in execute\n'
' raise TaskError("Failed to execute \'datahub ingest\'")\n'
"acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
'errors': []}
~~~~ Ingestion Logs ~~~~
Obtaining venv creation lock...
Acquired venv creation lock
venv setup time = 0
This version of datahub supports report-to functionality
datahub ingest run -c /tmp/datahub/ingest/b7fe3905-6cb1-4c7b-9fb7-06f2e516826a/recipe.yml --report-to /tmp/datahub/ingest/b7fe3905-6cb1-4c7b-9fb7-06f2e516826a/ingestion_report.json
[2023-04-18 19:28:45,463] INFO {datahub.cli.ingest_cli:173} - DataHub CLI version: 0.10.2
[2023-04-18 19:28:45,532] INFO {datahub.ingestion.run.pipeline:204} - Sink configured successfully. DataHubRestEmitter: configured to talk to
http://datahub-gms:8080
[2023-04-18 19:28:47,467] WARNING {py.warnings:109} - /tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/ratelimiter.py:127: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
__aexit__ = asyncio.coroutine(__exit__)
[2023-04-18 19:28:47,791] ERROR {datahub.entrypoints:195} - Command failed: Failed to find a registered source for type bigquery: 'str' object is not callable
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 119, in _add_init_error_context
yield
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 214, in __init__
source_class = source_registry.get(source_type)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 173, in get
tp = self._ensure_not_lazy(key)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 117, in _ensure_not_lazy
plugin_class = import_path(path)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 48, in import_path
item = importlib.import_module(module_name)
File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/source/bigquery_v2/bigquery.py", line 57, in <module>
from datahub.ingestion.source.bigquery_v2.lineage import (
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/source/bigquery_v2/lineage.py", line 39, in <module>
from datahub.utilities.bigquery_sql_parser import BigQuerySQLParser
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/utilities/bigquery_sql_parser.py", line 6, in <module>
from datahub.utilities.sql_parser import SqlLineageSQLParser, SQLParser
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/utilities/sql_parser.py", line 9, in <module>
from datahub.utilities.sql_lineage_parser_impl import SqlLineageSQLParserImpl
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/utilities/sql_lineage_parser_impl.py", line 8, in <module>
from sqllineage.core.holders import Column, SQLLineageHolder
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/sqllineage/__init__.py", line 41, in <module>
_monkey_patch()
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/sqllineage/__init__.py", line 35, in _monkey_patch
_patch_updating_lateral_view_lexeme()
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/sqllineage/__init__.py", line 24, in _patch_updating_lateral_view_lexeme
if regex("LATERAL VIEW EXPLODE(col)"):
TypeError: 'str' object is not callable
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/entrypoints.py", line 182, in main
sys.exit(datahub(standalone_mode=False, **kwargs))
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 379, in wrapper
raise e
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 334, in wrapper
res = func(*args, **kwargs)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/utilities/memory_leak_detector.py", line 95, in wrapper
return func(ctx, *args, **kwargs)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 187, in run
pipeline = Pipeline.create(
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 328, in create
return cls(
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 211, in __init__
with _add_init_error_context(
File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "/tmp/datahub/ingest/venv-bigquery-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 121, in _add_init_error_context
raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to find a registered source for type bigquery: 'str' object is not callable
DataHub CLI version: 0.10.2
Python version: 3.10.10 (main, Feb 8 2023, 053204) [Clang 14.0.0 (clang-1400.0.29.202)]lively-cat-88289
04/18/2023, 8:09 PMnice-rose-9274
04/19/2023, 1:07 AMlively-businessperson-84910
04/19/2023, 5:24 AMbrainy-intern-50400
04/19/2023, 10:18 AMdatahub.ingestion.run.pipeline.PipelineInitError: Failed to find a registered source for type metabase: 'str' object is not callablelively-businessperson-84910
04/19/2023, 12:49 PMfamous-barista-37580
04/19/2023, 3:09 PMfamous-barista-37580
04/19/2023, 3:27 PMfew-air-34037
04/20/2023, 6:34 AMpowerbi_1 | from sqllineage.runner import LineageRunner
powerbi_1 | File "/usr/local/lib/python3.10/site-packages/sqllineage/__init__.py", line 41, in <module>
powerbi_1 | _monkey_patch()
powerbi_1 | File "/usr/local/lib/python3.10/site-packages/sqllineage/__init__.py", line 35, in _monkey_patch
powerbi_1 | _patch_updating_lateral_view_lexeme()
powerbi_1 | File "/usr/local/lib/python3.10/site-packages/sqllineage/__init__.py", line 24, in _patch_updating_lateral_view_lexeme
powerbi_1 | if regex("LATERAL VIEW EXPLODE(col)"):
powerbi_1 | TypeError: 'str' object is not callable
powerbi_1 |
powerbi_1 | The above exception was the direct cause of the following exception:
powerbi_1 |
powerbi_1 | Traceback (most recent call last):
powerbi_1 | File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
powerbi_1 | return _run_code(code, main_globals, None,
powerbi_1 | File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
powerbi_1 | exec(code, run_globals)
powerbi_1 | File "/usr/local/app/src/yle/datahub/data_loader.py", line 101, in <module>
powerbi_1 | main()
powerbi_1 | File "/usr/local/app/src/yle/datahub/data_loader.py", line 85, in main
powerbi_1 | pipeline = Pipeline.create(pipeline_config)
powerbi_1 | File "/usr/local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 308, in create
powerbi_1 | return cls(
powerbi_1 | File "/usr/local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 191, in __init__
powerbi_1 | with _add_init_error_context(
powerbi_1 | File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
powerbi_1 | self.gen.throw(typ, value, traceback)
powerbi_1 | File "/usr/local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 117, in _add_init_error_context
powerbi_1 | raise PipelineInitError(f"Failed to {step}: {e}") from e
powerbi_1 | datahub.ingestion.run.pipeline.PipelineInitError: Failed to find a registered source for type redshift: 'str' object is not callablefew-air-34037
04/20/2023, 7:09 AMfew-air-34037
04/20/2023, 7:39 AMacryl-datahub==X.X:X
...
sqlparse==0.4.3
sqlfluff==2.0.3ancient-policeman-73437
04/20/2023, 12:00 PMfew-air-34037
04/20/2023, 12:01 PMrefined-energy-76018
04/20/2023, 5:41 PMrefined-energy-76018
04/20/2023, 10:16 PMhundreds-airline-29192
04/22/2023, 11:59 AMhundreds-airline-29192
04/22/2023, 12:00 PMrapid-accountant-13127
04/25/2023, 2:47 AMdatahub docker quickstart ?refined-energy-76018
04/25/2023, 2:54 AM- UI_INGESTION_DEFAULT_CLI_VERSION=0.10.2.1 under datahub-gms: environment:hundreds-airline-29192
04/25/2023, 6:32 AMflat-petabyte-80502
04/25/2023, 12:36 PMancient-guitar-42996
04/26/2023, 9:06 AMhundreds-airline-29192
04/27/2023, 1:49 AMastonishing-answer-96712
05/02/2023, 6:08 PMaloof-gpu-11378
05/02/2023, 6:21 PMcurved-planet-99787
05/08/2023, 2:08 PMgray-shoe-75895
05/08/2023, 6:46 PMcurved-planet-99787
05/09/2023, 5:32 AMlate-smartphone-6255
05/10/2023, 4:00 AMUI_INGESTION_DEFAULT_CLI_VERSION=0.10.2.2 in dockerfile and start it, but the CLI version still be 0.10.2.1, anyone know how to fix it?gentle-hamburger-31302
05/10/2023, 10:48 AMlate-smartphone-6255
05/11/2023, 6:38 AMaloof-gpu-11378
05/11/2023, 6:41 AMfew-air-34037
05/31/2023, 6:54 AM