Hey, getting this error while trying to ingest Metabase 0.41.2 (version as suggested in the docs - https://datahubproject.io/docs/generated/ingestion/sources/metabase/#compatibility) I have created a local metabase instance at localhost:3000 with the yaml recipe - source: type: metabase config: connect_uri: 'localhost:3000' username: admin_username password: admin_password DataHub CLI version: 0.10.2.3 Python version: 3.9.6 (default, Mar 10 2023, 201638) [Clang 14.0.3 (clang-1403.0.22.14.1)] Need a nudge in what I might be doing wrong here?

~~~~ Execution Summary - RUN_INGEST ~~~~
Execution finished with errors.
{'exec_id': '0de3d15c-4e8d-45bf-8877-46e9c8c66de8',
 'infos': ['2023-05-14 08:45:07.246534 INFO: Starting execution for task with name=RUN_INGEST',
           "2023-05-14 08:45:11.476157 INFO: Failed to execute 'datahub ingest'",
           '2023-05-14 08:45:11.486188 INFO: Caught exception EXECUTING task_id=0de3d15c-4e8d-45bf-8877-46e9c8c66de8, name=RUN_INGEST, '
           'stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 122, in execute_task\n'
           '    task_event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
           '    return future.result()\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 231, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
 'errors': []}

~~~~ Ingestion Logs ~~~~
Obtaining venv creation lock...
Acquired venv creation lock
venv setup time = 0
This version of datahub supports report-to functionality
datahub --debug ingest run -c /tmp/datahub/ingest/0de3d15c-4e8d-45bf-8877-46e9c8c66de8/recipe.yml --report-to /tmp/datahub/ingest/0de3d15c-4e8d-45bf-8877-46e9c8c66de8/ingestion_report.json
[2023-05-14 08:45:08,814] DEBUG    {datahub.telemetry.telemetry:219} - Sending init Telemetry
[2023-05-14 08:45:10,004] DEBUG    {datahub.telemetry.telemetry:248} - Sending telemetry for function-call
[2023-05-14 08:45:10,417] INFO     {datahub.cli.ingest_cli:173} - DataHub CLI version: 0.10.2
[2023-05-14 08:45:10,582] DEBUG    {datahub.ingestion.sink.datahub_rest:116} - Setting env variables to override config
[2023-05-14 08:45:10,582] DEBUG    {datahub.ingestion.sink.datahub_rest:118} - Setting gms config
[2023-05-14 08:45:10,583] DEBUG    {datahub.ingestion.run.pipeline:203} - Sink type datahub-rest (<class 'datahub.ingestion.sink.datahub_rest.DatahubRestSink'>) configured
[2023-05-14 08:45:10,583] INFO     {datahub.ingestion.run.pipeline:204} - Sink configured successfully. DataHubRestEmitter: configured to talk to <http://datahub-gms:8080>
[2023-05-14 08:45:10,595] DEBUG    {datahub.ingestion.run.pipeline:278} - Reporter type:file,<class 'datahub.ingestion.reporting.file_reporter.FileReporter'> configured.
[2023-05-14 08:45:10,630] DEBUG    {datahub.telemetry.telemetry:248} - Sending telemetry for function-call
[2023-05-14 08:45:11,034] ERROR    {datahub.entrypoints:195} - Command failed: Failed to find a registered source for type metabase: 'str' object is not callable
Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 119, in _add_init_error_context
    yield
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 214, in __init__
    source_class = source_registry.get(source_type)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 173, in get
    tp = self._ensure_not_lazy(key)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 117, in _ensure_not_lazy
    plugin_class = import_path(path)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/api/registry.py", line 48, in import_path
    item = importlib.import_module(module_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/source/metabase.py", line 10, in <module>
    from sqllineage.runner import LineageRunner
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/sqllineage/__init__.py", line 41, in <module>
    _monkey_patch()
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/sqllineage/__init__.py", line 35, in _monkey_patch
    _patch_updating_lateral_view_lexeme()
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/sqllineage/__init__.py", line 24, in _patch_updating_lateral_view_lexeme
    if regex("LATERAL VIEW EXPLODE(col)"):
TypeError: 'str' object is not callable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/entrypoints.py", line 182, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 379, in wrapper
    raise e
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 334, in wrapper
    res = func(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/utilities/memory_leak_detector.py", line 95, in wrapper
    return func(ctx, *args, **kwargs)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 187, in run
    pipeline = Pipeline.create(
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 328, in create
    return cls(
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 211, in __init__
    with _add_init_error_context(
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 121, in _add_init_error_context
    raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to find a registered source for type metabase: 'str' object is not callable
[2023-05-14 08:45:11,040] DEBUG    {datahub.entrypoints:197} - DataHub CLI version: 0.10.2 at /tmp/datahub/ingest/venv-metabase-0.10.2/lib/python3.10/site-packages/datahub/__init__.py
[2023-05-14 08:45:11,040] DEBUG    {datahub.entrypoints:200} - Python version: 3.10.10 (main, Mar 14 2023, 03:08:22) [GCC 10.2.1 20210110] at /tmp/datahub/ingest/venv-metabase-0.10.2/bin/python3 on Linux-5.15.49-linuxkit-aarch64-with-glibc2.31
[2023-05-14 08:45:11,040] DEBUG    {datahub.entrypoints:205} - GMS config {'models': {}, 'patchCapable': True, 'versions': {'linkedin/datahub': {'version': 'v0.10.2', 'commit': '0fa983adc7370862371b4c0786aac0e3b81a563a'}}, 'managedIngestion': {'defaultCliVersion': '0.10.2', 'enabled': True}, 'statefulIngestionCapable': True, 'supportsImpactAnalysis': True, 'timeZone': 'GMT', 'telemetry': {'enabledCli': True, 'enabledIngestion': False}, 'datasetUrnNameCasing': False, 'retention': 'true', 'datahub': {'serverType': 'quickstart'}, 'noCode': 'true'}

Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ button ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? ✅ button Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!

@famous-waitress-64616 might be able to provide some guidance for you here!

Hi @brave-room-48783, when you run ingestion via the UI (rather than the CLI), you may not run ingestion with the same CLI version as you have installed locally. It looks from the logs that you're running ingestion with version

v0.10.2

which has a dependency issue with

sqllineage

. Please update to a more recent version, ideally

0.10.2.3

. You can do this on UI ingestion by editing your ingestion source, in the "4 - Finish Up" tab, open up the

> Advanced

panel and provide a manual

CLI Version

.

Hey @famous-waitress-64616 I did that but I am still getting the same error

Can you print out the error logs from the new failed run please?

exec-urn_li_dataHubExecutionRequest_d626d1fb-8593-4865-a817-e9f3a11fe872.log

Yes, this is a different issue than the original issue. Note that we are making a POST request to /api/session, while going to that url on your browser will make a GET request, which it seems is not supported as per https://www.metabase.com/docs/latest/api/session. While your local metabase instance is running, can you try the

curl

as described on https://www.metabase.com/learn/administration/metabase-api#authenticate-your-requests-with-a-session-token (but with your username and password)

Hey @famous-waitress-64616 after running that curl command I am unable to run ingestion for metabase. Tried refreshing, reingesting, restarting containers, restarting system.

Hi Samarth, sorry I was not clear. I just want to see the output of that curl command to help debugging -- it is not meant to fix ingestion.

Got this

{"id":"aeeab220-4740-4510-8269-026810e04e4e"}*%*

@famous-waitress-64616

Hmm I'm not sure what's going on here. Can you send the full recipe you're using for ingestion? We may be able to help you better at office hours tomorrow, if you can make it.