square-bird-94136
09/15/2022, 12:27 PM~~~~ Execution Summary ~~~~
RUN_INGEST - {'errors': [],
'exec_id': 'fe59f987-686e-4078-8f83-eb1ddf63fc2f',
'infos': ['2022-09-15 12:22:12.253433 [exec_id=fe59f987-686e-4078-8f83-eb1ddf63fc2f] INFO: Starting execution for task with name=RUN_INGEST',
'2022-09-15 12:22:48.478154 [exec_id=fe59f987-686e-4078-8f83-eb1ddf63fc2f] INFO: stdout=venv setup time = 0\n'
'This version of datahub supports report-to functionality\n'
'datahub ingest run -c /tmp/datahub/ingest/fe59f987-686e-4078-8f83-eb1ddf63fc2f/recipe.yml --report-to '
'/tmp/datahub/ingest/fe59f987-686e-4078-8f83-eb1ddf63fc2f/ingestion_report.json\n'
'[2022-09-15 12:22:34,221] INFO {datahub.cli.ingest_cli:182} - DataHub CLI version: 0.8.44.2\n'
'[2022-09-15 12:22:34,243] INFO {datahub.ingestion.run.pipeline:175} - Sink configured successfully. DataHubRestEmitter: configured '
'to talk to <http://datahub-datahub-gms:8080>\n'
'[2022-09-15 12:22:46,903] ERROR {datahub.entrypoints:192} - \n'
'Traceback (most recent call last):\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 196, in __init__\n'
' self.source: Source = source_class.create(\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source/sql/bigquery.py", line 989, in create\n'
' config = BigQueryConfig.parse_obj(config_dict)\n'
' File "pydantic/main.py", line 521, in pydantic.main.BaseModel.parse_obj\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/source_config/sql/bigquery.py", line 69, in __init__\n'
' super().__init__(**data)\n'
' File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__\n'
'pydantic.error_wrappers.ValidationError: 1 validation error for BigQueryConfig\n'
'include_view_lineage\n'
' extra fields not permitted (type=value_error.extra)\n'
'\n'
'The above exception was the direct cause of the following exception:\n'
'\n'
'Traceback (most recent call last):\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/cli/ingest_cli.py", line 197, in run\n'
' pipeline = Pipeline.create(\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 317, in create\n'
' return cls(\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 202, in __init__\n'
' self._record_initialization_failure(\n'
' File "/usr/local/lib/python3.9/site-packages/datahub/ingestion/run/pipeline.py", line 129, in _record_initialization_failure\n'
' raise PipelineInitError(msg) from e\n'
'datahub.ingestion.run.pipeline.PipelineInitError: Failed to configure source (bigquery)\n'
'[2022-09-15 12:22:46,903] ERROR {datahub.entrypoints:195} - Command failed: \n'
'\tFailed to configure source (bigquery) due to \n'
"\t\t'1 validation error for BigQueryConfig\n"
'include_view_lineage\n'
" extra fields not permitted (type=value_error.extra)'.\n"
'\tRun with --debug to get full stacktrace.\n'
"\te.g. 'datahub --debug ingest run -c /tmp/datahub/ingest/fe59f987-686e-4078-8f83-eb1ddf63fc2f/recipe.yml --report-to "
"/tmp/datahub/ingest/fe59f987-686e-4078-8f83-eb1ddf63fc2f/ingestion_report.json'\n",
"2022-09-15 12:22:48.478380 [exec_id=fe59f987-686e-4078-8f83-eb1ddf63fc2f] INFO: Failed to execute 'datahub ingest'",
'2022-09-15 12:22:48.478596 [exec_id=fe59f987-686e-4078-8f83-eb1ddf63fc2f] INFO: Caught exception EXECUTING '
'task_id=fe59f987-686e-4078-8f83-eb1ddf63fc2f, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
' File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 123, in execute_task\n'
' task_event_loop.run_until_complete(task_future)\n'
' File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete\n'
' return future.result()\n'
' File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 168, in execute\n'
' raise TaskError("Failed to execute \'datahub ingest\'")\n'
"acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}
Execution finished with errors.square-bird-94136
09/15/2022, 12:27 PMsource:
type: bigquery
config:
include_table_lineage: false
credential:
private_key_id: '${private-key-id}'
project_id: '${project_id}'
client_email: '${client_email}'
private_key: '${private-key}'
client_id: '${client_id}'
profiling:
enabled: false
project_id: '${project_id}'
include_view_lineage: false
stateful_ingestion:
enabled: false
pipeline_name: 'urn:li:dataHubIngestionSource:75869f85-d290-476a-a359-a7a6e6015ebc'square-bird-94136
09/15/2022, 12:27 PMbitter-tent-3827
09/15/2022, 1:10 PMsquare-bird-94136
09/15/2022, 5:10 PMgray-shoe-75895
09/15/2022, 8:53 PM