Got the error when ingestion from superset I can ingest some DataHub #ingestion

Got the error when ingestion from superset. I can ...

damp-ambulance-34232

10/29/2021, 3:59 AM

Got the error when ingestion from superset. I can ingest some dashboard but not all

Copy code

File "/usr/local/lib/python3.6/dist-packages/datahub/entrypoints.py", line 91, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
File "/usr/lib/python3/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/datahub/cli/ingest_cli.py", line 58, in run
    pipeline.run()
File "/usr/local/lib/python3.6/dist-packages/datahub/ingestion/run/pipeline.py", line 125, in run
    for wu in self.source.get_workunits():
File "/usr/local/lib/python3.6/dist-packages/datahub/ingestion/source/superset.py", line 339, in get_workunits
    yield from self.emit_dashboard_mces()
File "/usr/local/lib/python3.6/dist-packages/datahub/ingestion/source/superset.py", line 250, in emit_dashboard_mces
    dashboard_data
File "/usr/local/lib/python3.6/dist-packages/datahub/ingestion/source/superset.py", line 213, in construct_dashboard_from_api_data
    position_data = json.loads(raw_position_data)
File "/usr/lib/python3.6/json/__init__.py", line 348, in loads
    'not {!r}'.format(s.__class__.__name__))

TypeError: the JSON object must be str, bytes or bytearray, not 'NoneType'

damp-ambulance-34232

10/29/2021, 4:58 AM

And something wrong with dashboardUrl value, mising a number, the right port is 8099, and datahub ingest to 809

Copy code

wu = MetadataWorkUnit(id='urn:li:dashboard:(superset,23)', metadata=MetadataChangeEventClass({'auditHeader': None, 'proposedS
           napshot': DashboardSnapshotClass({'urn': 'urn:li:dashboard:(superset,23)', 'aspects': [DashboardInfoClass({'customProper
           ties': {}, 'externalUrl': None, 'title': 'Shop_huonght', 'description': '', 'charts': ['urn:li:chart:(superset,165)', 'u
           rn:li:chart:(superset,167)', 'urn:li:chart:(superset,164)', 'urn:li:chart:(superset,161)', 'urn:li:chart:(superset,168)'
           , 'urn:li:chart:(superset,169)', 'urn:li:chart:(superset,166)'], 'lastModified': ChangeAuditStampsClass({'created': Audi
           tStampClass({'time': 1634350969722, 'actor': 'urn:li:corpuser:huonght37', 'impersonator': None}), 'lastModified': AuditS
           tampClass({'time': 1634350969722, 'actor': 'urn:li:corpuser:huonght37', 'impersonator': None}), 'deleted': None}), 'dash
           boardUrl': '<http://superset-host:809/superset/dashboard/23/>', 'access': None, 'lastRefreshed': None})]}), 'proposedDel
           ta': None, 'systemMetadata': SystemMetadataClass({'lastObserved': 1635481603901, 'runId': 'superset-2021_10_29-11_26_43'
           , 'properties': None})}), treat_errors_as_warnings=False)

mammoth-bear-12532

10/29/2021, 6:12 AM

@damp-ambulance-34232 you might need to upgrade your python package

mammoth-bear-12532

10/29/2021, 6:13 AM

this was a bug that was fixed recently

damp-ambulance-34232

10/29/2021, 6:38 AM

@mammoth-bear-12532 My version DataHub CLI version: 0.8.15.7 Python version: 3.6.9 (default, Jan 26 2021, 153300) [GCC 8.4.0]

damp-ambulance-34232

10/29/2021, 7:07 AM

@mammoth-bear-12532 Upgrade datahub to lastest version and still got the error DataHub CLI version: 0.8.16.1 Python version: 3.6.9 (default, Jan 26 2021, 153300) [GCC 8.4.0]

3 Views

Open in Slack

Previous Next