https://datahubproject.io logo
Join Slack
Powered by
Hi Everyone , Getting this error when ingesting Ch...
# ingestion
h
Hi Everyone , Getting this error when ingesting Charts/Dashboards from Superset :
Copy code
File "/root/.venvs/airflow/lib/python3.8/site-packages/datahub/ingestion/source/sql/sql_common.py", line 62, in get_platform_from_sqlalchemy_uri
    if sqlalchemy_uri.startswith("bigquery"):
AttributeError: 'NoneType' object has no attribute 'startswith'
[2022-03-17 21:03:07,554] {taskinstance.py:1525} INFO - Marking task as FAILED. dag_id=metadata_ingestion_dag, task_id=ingest_metadata, execution_date=20220317T205757, start_date=20220317T210304, end_date=20220317T210307
[2022-03-17 21:03:07,623] {local_task_job.py:146} INFO - Task exited with return code 1
o
Looks like it's failing to resolve the platform. Can you verify the database name is correct and reachable via 
{connect_uri}/api/v1/dataset/{datasource_id}
h
@orange-night-91387 I am not sure I understand. Trying to ingest the charts and Dashboards metadata in Datahub. Which Database are you referring to here ?
Few of the Dashboards got ingested and then the pipeline failed with the above error
Copy code
[2022-03-21 14:24:35,443] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,21)
[2022-03-21 14:24:35,459] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,20)
[2022-03-21 14:24:35,472] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,19)
[2022-03-21 14:24:35,490] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,18)
[2022-03-21 14:24:35,504] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,14)
[2022-03-21 14:24:35,518] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,9)
[2022-03-21 14:24:35,532] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,4)
[2022-03-21 14:24:35,547] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,12)
[2022-03-21 14:24:35,561] {pipeline.py:76} INFO - sink wrote workunit urn:li:dashboard:(superset,17)
@orange-night-91387 - Can we specifically ingest the charts ?
o
This is for when pulling the chart source see superset.py line 263. We reach out to the chart's underlying datasource_id:
Copy code
datasource_id = chart_data.get("datasource_id")
        datasource_urn = self.get_datasource_urn_from_id(datasource_id)
The issue seems to be that the chart specifies an underlying database source, but that source is not reachable
Specifically that no 
sqlalchemy_uri
comes back in the response so a NoneType gets passed into a function expecting a string. This should be hardened against failure, but the platform type will not be able to be determined without that response and creating the linked dataset urn for the datasource will fail
Open in Slack
PreviousNext
Powered by