square-house-62981
01/24/2023, 2:21 PMsource:
type: powerbi
config:
scan_timeout: 60000
extract_reports: true
tenant_id: <tenant_id>
workspace_id_pattern:
allow:
- '.*'
env: DEV
client_id: <client_id>
client_secret: '********'
extract_ownership: true
dataset_type_mapping:
Snowflake: snowflake
sink:
type: datahub-rest
config:
server: '<http://host:8080>'
I get no errors but the result of the ingestion is that I don't have anything in my tenant.
Here is the log output:
~~~~ Execution Summary ~~~~
RUN_INGEST - {'errors': [],
'exec_id': 'd2313617-911c-4a97-8d25-0679f414c08e',
'infos': ['2023-01-24 14:28:14.879955 [exec_id=d2313617-911c-4a97-8d25-0679f414c08e] INFO: Starting execution for task with name=RUN_INGEST',
'2023-01-24 14:28:20.932698 [exec_id=d2313617-911c-4a97-8d25-0679f414c08e] INFO: stdout=Obtaining venv creation lock...\n'
'Acquired venv creation lock\n'
'venv setup time = 0\n'
'This version of datahub supports report-to functionality\n'
'datahub ingest run -c /tmp/datahub/ingest/d2313617-911c-4a97-8d25-0679f414c08e/recipe.yml --report-to '
'/tmp/datahub/ingest/d2313617-911c-4a97-8d25-0679f414c08e/ingestion_report.json\n'
'[2023-01-24 14:28:17,021] INFO {datahub.cli.ingest_cli:165} - DataHub CLI version: 0.9.6\n'
'[2023-01-24 14:28:17,075] INFO {datahub.ingestion.run.pipeline:179} - Sink configured successfully. DataHubRestEmitter: configured '
'to talk to <http://host:8080>\n'
'/tmp/datahub/ingest/venv-powerbi-0.9.6/lib/python3.10/site-packages/datahub/ingestion/source/powerbi/powerbi.py:740: '
'ConfigurationWarning: env is deprecated and will be removed in a future release. Please use platform_instance instead.\n'
' config = PowerBiDashboardSourceConfig.parse_obj(config_dict)\n'
'[2023-01-24 14:28:17,295] INFO {datahub.ingestion.source.powerbi.proxy:211} - Trying to connect to '
'<https://login.microsoftonline.com/tenant_id>\n'
'[2023-01-24 14:28:17,295] INFO {datahub.ingestion.source.powerbi.proxy:326} - Generating PowerBi access token\n'
'[2023-01-24 14:28:17,382] INFO {datahub.ingestion.source.powerbi.proxy:340} - Generated PowerBi access token\n'
'[2023-01-24 14:28:17,383] INFO {datahub.ingestion.source.powerbi.proxy:213} - Able to connect to '
'<https://login.microsoftonline.com/tenant_id>\n'
'[2023-01-24 14:28:17,522] INFO {datahub.ingestion.source.powerbi.proxy:211} - Trying to connect to '
'<https://login.microsoftonline.com/tenant_id>\n'
'[2023-01-24 14:28:17,522] INFO {datahub.ingestion.source.powerbi.proxy:326} - Generating PowerBi access token\n'
'[2023-01-24 14:28:17,608] INFO {datahub.ingestion.source.powerbi.proxy:340} - Generated PowerBi access token\n'
'[2023-01-24 14:28:17,608] INFO {datahub.ingestion.source.powerbi.proxy:213} - Able to connect to '
'<https://login.microsoftonline.com/tenant_id>\n'
'[2023-01-24 14:28:17,609] INFO {datahub.ingestion.run.pipeline:196} - Source configured successfully.\n'
'[2023-01-24 14:28:17,610] INFO {datahub.cli.ingest_cli:120} - Starting metadata ingestion\n'
'[2023-01-24 14:28:17,611] INFO {datahub.ingestion.source.powerbi.powerbi:765} - PowerBi plugin execution is started\n'
'[2023-01-24 14:28:17,611] INFO {datahub.ingestion.source.powerbi.proxy:690} - Request to get groups endpoint '
'URL=<https://api.powerbi.com/v1.0/myorg/groups>\n'
'[2023-01-24 14:28:17,989] INFO {datahub.ingestion.reporting.file_reporter:52} - Wrote SUCCESS report successfully to '
"<_io.TextIOWrapper name='/tmp/datahub/ingest/d2313617-911c-4a97-8d25-0679f414c08e/ingestion_report.json' mode='w' encoding='UTF-8'>\n"
'[2023-01-24 14:28:17,989] INFO {datahub.cli.ingest_cli:133} - Finished metadata ingestion\n'
'\n'
'Cli report:\n'
"{'cli_version': '0.9.6',\n"
" 'cli_entry_location': '/tmp/datahub/ingest/venv-powerbi-0.9.6/lib/python3.10/site-packages/datahub/__init__.py',\n"
" 'py_version': '3.10.9 (main, Dec 8 2022, 01:35:40) [GCC 10.2.1 20210110]',\n"
" 'py_exec_path': '/tmp/datahub/ingest/venv-powerbi-0.9.6/bin/python3',\n"
" 'os_details': 'Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.31',\n"
" 'mem_info': '68.89 MB'}\n"
'Source (powerbi) report:\n'
"{'events_produced': '0',\n"
" 'events_produced_per_sec': '0',\n"
" 'event_ids': [],\n"
" 'warnings': {},\n"
" 'failures': {},\n"
" 'dashboards_scanned': '0',\n"
" 'charts_scanned': '0',\n"
" 'filtered_dashboards': [],\n"
" 'filtered_charts': [],\n"
" 'start_time': '2023-01-24 14:28:17.169216 (1.19 seconds ago).',\n"
" 'running_time': '1.19 seconds'}\n"
'Sink (datahub-rest) report:\n'
"{'total_records_written': '0',\n"
" 'records_written_per_second': '0',\n"
" 'warnings': [],\n"
" 'failures': [],\n"
" 'start_time': '2023-01-24 14:28:17.069492 (1.29 seconds ago).',\n"
" 'current_time': '2023-01-24 14:28:18.360552 (now).',\n"
" 'total_duration_in_seconds': '1.29',\n"
" 'gms_version': 'v0.9.6',\n"
" 'pending_requests': '0'}\n"
'\n'
' Pipeline finished successfully; produced 0 events in 1.19 seconds.\n',
"2023-01-24 14:28:20.932903 [exec_id=d2313617-911c-4a97-8d25-0679f414c08e] INFO: Successfully executed 'datahub ingest'"],
'structured_report': '{"cli": {"cli_version": "0.9.6", "cli_entry_location": '
'"/tmp/datahub/ingest/venv-powerbi-0.9.6/lib/python3.10/site-packages/datahub/__init__.py", "py_version": "3.10.9 (main, Dec '
'8 2022, 01:35:40) [GCC 10.2.1 20210110]", "py_exec_path": "/tmp/datahub/ingest/venv-powerbi-0.9.6/bin/python3", "os_details": '
'"Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.31", "mem_info": "68.89 MB"}, "source": {"type": "powerbi", "report": '
'{"events_produced": "0", "events_produced_per_sec": "0", "event_ids": [], "warnings": {}, "failures": {}, '
'"dashboards_scanned": "0", "charts_scanned": "0", "filtered_dashboards": [], "filtered_charts": [], "start_time": "2023-01-24 '
'14:28:17.169216 (now).", "running_time": "0.82 seconds"}}, "sink": {"type": "datahub-rest", "report": '
'{"total_records_written": "0", "records_written_per_second": "0", "warnings": [], "failures": [], "start_time": "2023-01-24 '
'14:28:17.069492 (now).", "current_time": "2023-01-24 14:28:17.989186 (now).", "total_duration_in_seconds": "0.92", '
'"gms_version": "v0.9.6", "pending_requests": "0"}}}'}
Execution finished successfully!
The thing is, I'm sure there are a lot of reports, dashboards, datasets, etc.
Could you help me to guess what I am doing wrong?
Thank you in advance!!!!!!!!!!yay bearbusy-dream-34673
01/27/2023, 1:00 PMbusy-dream-34673
01/30/2023, 2:58 PMsquare-house-62981
04/13/2023, 8:07 AMcareful-doctor-13356
11/07/2023, 7:37 PM