Hello DataHub, I am getting an error when I try t...
# troubleshoot
p
Hello DataHub, I am getting an error when I try to ingest a file into datahub GMS via the CLI. It seems that the issue is related to configuration (it's the first time we are trying to do this). The command I am issuing is:
Copy code
python3 -m datahub ingest -c source.yml
The log is attached as
cli-error-log.txt
my .datahubenv looks something like this:
Copy code
gms:
  server: https://<<<gms-host>>>.us-east-1.elb.amazonaws.com:8080
  token: <<<token>>>
And I can curl the following URL successfully:
Copy code
curl http://<<<gms-host>>>.us-east-1.elb.amazonaws.com:8080/config
{
  "models" : { },
  "patchCapable" : true,
  "versions" : {
    "linkedin/datahub" : {
      "version" : "v0.10.0",
      "commit" : "cf1e627e55431fc69d72918b2bcc3c5f3a1d5002"
    }
  },
  "managedIngestion" : {
    "defaultCliVersion" : "0.10.0",
    "enabled" : true
  },
  "statefulIngestionCapable" : true,
  "supportsImpactAnalysis" : true,
  "telemetry" : {
    "enabledCli" : true,
    "enabledIngestion" : false
  },
  "datasetUrnNameCasing" : false,
  "retention" : "true",
  "datahub" : {
    "serverType" : "prod"
  },
  "noCode" : "true"
}
I looked at this post: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#your-proxy-appears-to-only-use-http-and-not-https In my setup, there are no env vars setup for HTTP_PROXY or HTTPS_PROXY. The error happens when trying to access the /config endpoint and says
try changing your proxy URL to be HTTP
GMS is installed in a kubernetes pod, in a production environment and we are in a VPN while running the above commands. Thanks !
📖 1
🔍 1
l
Hey there 👋 I'm The DataHub Community Support bot. I'm here to help make sure the community can best support you with your request. Let's double check a few things first: ✅ There's a lot of good information on our docs site: www.datahubproject.io/docs, Have you searched there for a solution? ✅ It's not uncommon that someone has run into your exact problem before in the community. Have you searched Slack for similar issues? Did you find a solution to your issue? ❌ Sorry you weren't able to find a solution. I'm sending you some tips on info you can provide to help the community troubleshoot. Whenever you feel your issue is solved, please react ✅ to your original message to let us know!
p
I found this post : https://datahubproject.io/docs/authentication/introducing-metadata-service-authentication/#now-that-i-can-make-authent[…]-service-which-should-i-use Could the issue be that I need to configure the HTTP_PROXY and HTTPS_PROXY env vars to point to the DataHub Frontend Proxy?
b
Just taking a stab, I see that the sink in your datahubenv is specified as https//...8080. Maybe you should try http instead? Cos nothing I see here seems to indicate that your deployment is running on https.
p
Thanks xL - I will try that in the morning !
a
Keep us posted!