Hi guys, when integrating with airflow, after defining inlets and outlets how should check it from the datahub side? I tried checking the logs after running the DAG and it does not show any datahub related messages. This is the code i did for defining inlets and outlets;

datahub_lineage_task_1 = BashOperator(

task_id="extract_data",

dag= dag,

inlets=[Dataset("mysql", "extract_sql.dag")],

outlets=[Dataset("s3", "test-project-100/project_100/table_01")],

bash_command="echo Dummy Task 1",

)

datahub_lineage_task_2 = BashOperator(

task_id="load_data",

dag= dag,

inlets=[Dataset("s3", "test-project-100/project_100/table_01")],

outlets=[Dataset("bigquery", "project-test.tb_bq_datahub")],

bash_command="echo Dummy Task 2",

)

inlets/outlets look fine, can you check in your Task logs if you see anything Datahub related logs?

i checked the DAG logs after running the DAG but i couldnt find any datahub related logs

Do you use the Airflow plugin and if you use it, please make sure

core.lazy_load_plugins : False

is disabled. https://datahubproject.io/docs/lineage/airflow/#using-datahubs-airflow-lineage-plugin-new

yeah that is done as well

Can you see the plugin on the plugin page of Airflow?

yeah i can see the datahub plugin

when making the connection with the rest HOOK, can you tell me what should be passed as the connection host

host should be your gms host, password should be your generated token

alright

you can generate one on datahub -> https://datahubproject.io/docs/authentication/personal-access-tokens/

Hi @dazzling-judge-80093. can i know if there is a way to test the connection from airflow to datahub?

the only way if you run task

i ran the DAG with the tasks but i dont see the datahub related logs in the task logs.