Getting the following error when trying to set up ...
# troubleshooth
handsome-football-66174
08/20/2021, 5:38 PMGetting the following error when trying to set up airflow with datahub:
Following - https://datahubproject.io/docs/metadata-ingestion/#lineage-with-airflow
m
mammoth-bear-12532
08/20/2021, 6:11 PM
@handsome-football-66174 are you using the
lineage_backendh
handsome-football-66174
08/20/2021, 6:15 PM@mammoth-bear-12532 - Yes using lineage backend option. On step
airflow connections add --conn-type 'datahub_rest' 'datahub_rest_default' --conn-host 'http://localhost:8080'
handsome-football-66174
08/20/2021, 6:16 PM@mammoth-bear-12532 - Any prerequisites for this ?
handsome-football-66174
08/23/2021, 5:08 PM@mammoth-bear-12532 - Never mind - Had to run
airflow db init
May be this can be added for people new to airflow
m
mammoth-bear-12532
08/23/2021, 6:27 PM
Thanks for letting us know @handsome-football-66174. Is that something to add as part of pre-requisites (e.g. how to set up Airflow locally)?
h
handsome-football-66174
08/25/2021, 2:50 PMYes Had to execute airflow db init, to initialize the airflow database.
handsome-football-66174
08/25/2021, 2:57 PM@mammoth-bear-12532 - How to connect to a centrally running Airflow instance ? Any steps on this ?
handsome-football-66174
08/25/2021, 7:16 PMAlso with the locally installed airflow getting this - File "/home/airflow/.venv/airflow/lib64/python3.7/site-packages/airflow/configuration.py", line 470, in getimport
f'The object could not be loaded. Please check "{key}" key in "{section}" section. '
airflow.exceptions.AirflowConfigException: The object could not be loaded. Please check "backend" key in "lineage" section. Current value: "datahub_provider.lineage.datahub.DatahubLineageBacken
m
mammoth-bear-12532
08/25/2021, 7:24 PM
@handsome-football-66174 how are you installing the local airflow?
h
handsome-football-66174
08/25/2021, 8:35 PM22 Views