Can someone point me to the DAG of <https://demo.d...
# ingestion
r
Can someone point me to the DAG of this airflow task on the demo site.
h
Based on the urn
urn:li:dataJob:(urn:li:dataFlow:(airflow,orders_load,PROD),orders_etl_job1)
the flow name should be
urn:li:dataFlow:(airflow,orders_load,PROD)
-> https://demo.datahubproject.io/pipelines/urn:li:dataFlow:(airflow,orders_load,PROD)/Documentation?is_lineage_mode=false
you can also click on the "Details" button in the upper right corner, and the go to the Pipeline tab
r
Yes but i can not see the source code of the DAG. View in airflow button takes me to the airflow website instead. Do we have the source code of this DAG/Pipeline.
s
@rich-policeman-92383 It seems that particular DAG is from an older deployment. The source is not in the hosted airflow for now. But you can find other DAGs here https://airflow.demo.datahubproject.io/home
Was there anything specific that you were trying to figure out from the DAG code @rich-policeman-92383?
r
@square-activity-64562 Thanks for sharing the code. I was looking for how to define Data_Job entity in DAG and how many other entities are available apart from the dataset entity. Data_Job Entity
s
The metadata related to Airflow jobs in datahub comes through lineage configured in airflow https://datahubproject.io/docs/lineage/airflow We have some examples if you are trying to do it through code https://datahubproject.io/docs/lineage/sample_code