Hi <@U01GZEETMEZ> , I would like to the test the a...
# ingestionc
colossal-furniture-76714
07/27/2021, 3:52 PMHi @gray-shoe-75895 , I would like to the test the airflow Integration or in other words I want to produce lineage based on airflow DAGs. I have followed the doc thoroughly, but I'm not sure if the connection is already working. Is there a way to check it? I have a DAG that runs fine, but I do not see any data in datahub
g
gray-shoe-75895
07/27/2021, 6:42 PMIt should produce some logs if it was actually executed
gray-shoe-75895
07/27/2021, 6:44 PMYou can find them in the airflow UI, under the logs for each task
gray-shoe-75895
07/27/2021, 6:45 PMAdditionally, if it failed to initialize the airflow lineage backend, you’ll see an error message in the logs of the airflow scheduler / worker process
c
colossal-furniture-76714
08/02/2021, 3:10 PMI get the following error:
Copy code
[2021-07-27 15:16:13,995] {base.py:78} INFO - Using connection to: id: datahub_rest_default. Host: <http://localhost:8080>, Port: None, Schema: , Login: , Password: None, extra: None
[2021-07-27 15:16:14,013] {base.py:78} INFO - Using connection to: id: datahub_rest_default. Host: <http://localhost:8080>, Port: None, Schema: , Login: , Password: None, extra: None
[2021-07-27 15:16:14,027] {datahub.py:122} ERROR - ('Unable to emit metadata to DataHub GMS', {'message': "HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /entities?action=ingest (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4d289c5128>: Failed to establish a new connection: [Errno 111] Connection refused',))"})
[2021-07-27 15:16:14,027] {datahub.py:123} INFO - Supressing error because graceful_exceptions is set