Hi Team, I am trying to set up a pipeline to inges...
# all-things-deployment
g
Hi Team, I am trying to set up a pipeline to ingest DBT metadata into datahub through a lambda function. However, I am getting the below error. Does anyone know how to resolve it? I am using a layer which has 'acryl-datahub[dbt]' package installed. below is the pipeline step:
Copy code
from datahub.ingestion.run.pipeline import Pipeline
 pipeline = Pipeline.create(
        {
            "source": {
                "type": "dbt",
                "config": {
                    "manifest_path": "/tmp/manifest.json",
                    "catalog_path": "/tmp/catalog.json",
                    "sources_path": "/tmp/sources.json",
                    "target_platform": "databricks",
                    "load_schemas": True
                },
            },
            "sink": {
                "type": "datahub-rest",
                "config": {"server": "http://<masked>:8080"},
            },
        }
    )
for more context, I am able to run dbt ingestion from within the datahub instance
Is there any other package required for using within lambda?