late-bear-87552
01/27/2022, 5:11 AM[2022-01-27 10:53:32,306] INFO {datahub.cli.ingest_cli:81} - Starting metadata ingestion
[2022-01-27 10:53:35,470] INFO {datahub.ingestion.source.sql.bigquery:224} - Built lineage map containing 0 entries.
[2022-01-27 10:53:38,136] INFO {datahub.ingestion.run.pipeline:79} - sink wrote workunit regal-reporter-240416.data_test.test_test_tesst
[2022-01-27 10:53:41,299] INFO {datahub.ingestion.run.pipeline:79} - sink wrote workunit regal-reporter-240416.test_datahub_big.test_datahub_big
[2022-01-27 10:53:42,221] INFO {datahub.cli.ingest_cli:83} - Finished metadata ingestion
Source (bigquery) report:
{'workunits_produced': 2,
'workunit_ids': ['regal-reporter-240416.data_test.test_test_tesst', 'regal-reporter-240416.test_datahub_big.test_datahub_big'],
'warnings': {},
'failures': {},
'tables_scanned': 2,
'views_scanned': 0,
'entities_profiled': 0,
'filtered': [],
'soft_deleted_stale_entities': [],
'query_combiner': None}
Sink (datahub-rest) report:
{'records_written': 2,
'warnings': [],
'failures': [],
'downstream_start_time': datetime.datetime(2022, 1, 27, 10, 53, 38, 113867),
'downstream_end_time': datetime.datetime(2022, 1, 27, 10, 53, 41, 299106),
'downstream_total_latency_in_seconds': 3.185239}big-carpet-38439
01/28/2022, 12:27 AMcurl -X POST '<http://localhost:8080/entities?action=search>' \
--data '{
"input": "*",
"entity": "dataset",
"start": 0,
"count": 1000
}'big-carpet-38439
01/28/2022, 12:27 AM