tried running ingestion using command for bigquery...
# ingestion
l
tried running ingestion using command for bigquery got below result on console but i dont see these datasets on UI.
Copy code
[2022-01-27 10:53:32,306] INFO     {datahub.cli.ingest_cli:81} - Starting metadata ingestion
[2022-01-27 10:53:35,470] INFO     {datahub.ingestion.source.sql.bigquery:224} - Built lineage map containing 0 entries.
[2022-01-27 10:53:38,136] INFO     {datahub.ingestion.run.pipeline:79} - sink wrote workunit regal-reporter-240416.data_test.test_test_tesst
[2022-01-27 10:53:41,299] INFO     {datahub.ingestion.run.pipeline:79} - sink wrote workunit regal-reporter-240416.test_datahub_big.test_datahub_big
[2022-01-27 10:53:42,221] INFO     {datahub.cli.ingest_cli:83} - Finished metadata ingestion

Source (bigquery) report:
{'workunits_produced': 2,
 'workunit_ids': ['regal-reporter-240416.data_test.test_test_tesst', 'regal-reporter-240416.test_datahub_big.test_datahub_big'],
 'warnings': {},
 'failures': {},
 'tables_scanned': 2,
 'views_scanned': 0,
 'entities_profiled': 0,
 'filtered': [],
 'soft_deleted_stale_entities': [],
 'query_combiner': None}
Sink (datahub-rest) report:
{'records_written': 2,
 'warnings': [],
 'failures': [],
 'downstream_start_time': datetime.datetime(2022, 1, 27, 10, 53, 38, 113867),
 'downstream_end_time': datetime.datetime(2022, 1, 27, 10, 53, 41, 299106),
 'downstream_total_latency_in_seconds': 3.185239}
b
What happens if you issue a "*" search query to datahub backend? eg.
Copy code
curl -X POST '<http://localhost:8080/entities?action=search>' \
--data '{
    "input": "*",
    "entity": "dataset",
    "start": 0,
    "count": 1000
}'
Do you see any of these datasets?