Hi all I successfully set up datahub with docker but failing DataHub #troubleshoot

Hi all, I successfully set up datahub with docker...

eager-vase-41681

12/20/2022, 12:18 PM

Hi all, I successfully set up datahub with docker, but failing to ingest the sample data. Elasticsearch and datahub-gms have different ports assigned compared to default settings.

eager-vase-41681

12/20/2022, 12:19 PM

These are the running datahub containers

eager-vase-41681

12/20/2022, 12:20 PM

$ datahub docker ingest-sample-data

Copy code

Downloading sample data...
Starting ingestion...
[2022-12-20 13:19:39,255] ERROR    {datahub.entrypoints:207} - Command failed: Failed to set up framework context: Failed to instantiate a valid DataHub Graph instance
Traceback (most recent call last):
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/api/common.py", line 62, in __init__
    self.graph = DataHubGraph(datahub_api) if datahub_api is not None else None
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/graph/client.py", line 74, in __init__
    self.test_connection()
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/emitter/rest_emitter.py", line 168, in test_connection
    raise ConfigurationError(message)
datahub.configuration.common.ConfigurationError: Unable to connect to <http://localhost:8080/config> with status_code: 404. Please check your configuration and make sure you are talking to the DataHub GMS (usually <datahub-gms-host>:8080) or Frontend GMS API (usually <frontend>:9002/api/gms).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 151, in __init__
    self.ctx = PipelineContext(
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/api/common.py", line 66, in __init__
    raise Exception(
Exception: Failed to instantiate a valid DataHub Graph instance

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/entrypoints.py", line 165, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 347, in wrapper
    raise e
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 299, in wrapper
    res = func(*args, **kwargs)
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/cli/docker_cli.py", line 914, in ingest_sample_data
    pipeline = Pipeline.create(recipe)
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 313, in create
    return cls(
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 160, in __init__
    self._raise_initialization_error(e, "Failed to set up framework context")
  File "/home/admin086/.local/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 129, in _raise_initialization_error
    raise PipelineInitError(f"{msg}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to set up framework context: Failed to instantiate a valid DataHub Graph instance

faint-actor-78390

12/20/2022, 1:28 PM

Hi, Several month ago , I have the same issue solved it with manuel ingestion

bootstrap_mce.json

eager-vase-41681

12/21/2022, 2:43 PM

Thanks @faint-actor-78390 for your hint. Can you guide me how to trigger the manual ingestion?

faint-actor-78390

12/21/2022, 3:21 PM

Hi something like datahub ingest -c recipe.dhub.yaml , Don't remember exactly. I found that here https://github.com/datahub-project/datahub/tree/master/metadata-ingestion unfortunately the file I send you is json and examples use a yaml file and I gave you a json ...

7 Views

Open in Slack

Previous Next