Hi Team, I am trying to ingest airflow metadata in...
# troubleshootl
little-breakfast-38102
11/18/2022, 7:46 PMHi Team, I am trying to ingest airflow metadata into datahub and need help resolving following error(attached screen shot).
We are on datahub 0.8.45, installed acryl-datahub-airflow-plug==0.9.2.2.
Attached screen shots of Task, datahub airflow operator import, datahub connector, airflow config additions.
Added
[core]
lazy_load_plugins = False
@dazzling-judge-80093, @gray-shoe-75895
g
gray-shoe-75895
11/21/2022, 2:44 AMThe log message just says that airflow failed to fetch logs from the worker
gray-shoe-75895
11/21/2022, 2:44 AMPlease work that out and post the actual error from the task here
l
little-breakfast-38102
11/21/2022, 2:47 AM@gray-shoe-75895, Thanks will do
little-breakfast-38102
11/29/2022, 1:37 AMHi @gray-shoe-75895, I am facing below connection error when my task tried to emit metadata.
Attached screen shot of Task and error message. Appreciate any help.
d
dazzling-judge-80093
11/29/2022, 10:56 AMCan you try to check if you can access datahub from the pod/node where you run your task?
l
little-breakfast-38102
11/29/2022, 11:49 PM@dazzling-judge-80093 running into SSL error, working internally to import the certs to Airflow dockerfile. Appreciate if you can share any insights.
little-breakfast-38102
12/01/2022, 3:54 PM@dazzling-judge-80093/ @gray-shoe-75895 got past ssl, I am getting 401. We have GMS token as password, and https://ingress/url as host. Attached is the screen shot. Any insight on what's missing
little-breakfast-38102
12/01/2022, 3:54 PMmessage has been deleted
d
dazzling-judge-80093
12/01/2022, 3:56 PMhmm, if the token is exactly set as password then it should be fine
dazzling-judge-80093
12/01/2022, 3:56 PMPlease, can you doublecheck if the token is correct?
dazzling-judge-80093
12/01/2022, 3:57 PMHere is an example how you can test your token from curl -> https://datahubproject.io/docs/authentication/personal-access-tokens/#using-personal-access-tokens
l
little-breakfast-38102
12/01/2022, 4:03 PMWill try curl, token is good, while creating airflow connection should I use username?
d
dazzling-judge-80093
12/01/2022, 4:03 PMNo, username doesn’t need
l
little-breakfast-38102
12/01/2022, 6:00 PM@dazzling-judge-80093 created a new PAT after creating a policy with extended permissions( all privileges) to the user. When ran CURL with HTTP returned nothing, more like success. When ran CURL with HTTPS got below SSL error.
little-breakfast-38102
12/01/2022, 7:33 PM@dazzling-judge-80093 , locally ran curl with the new PAT I created after assigning elevated privileges to the user. I am seeing 404.
Service: /api/gms/entities?action=ingest
2 Views