Hi everyone.

Issue

My Ingestion status is Failed. I attach log (screenshot). Also when I try to execute 'datahub docker ingest-sample-data' I get error: "ConfigurationError: Unable to connect to http://localhost:8080/config with status_code: 407. Please check your configuration and make sure you are talking to the DataHub GMS (usually <datahub-gms-host>:8080) or Frontend GMS API (usually <frontend>:9002/api/gms)."

Question

As I can understand I need to set Proxy for Datahub network. How can I do this? May be there is some related manual or FAQ

Did you deploy datahub? how did you set up your deployment?

Try with Python 3.8 or a higher version of CLI. Might be due to python 3.9.10 and CLI not released for that Python version in

0.8.26.6

. We had a temporary restriction starting

0.8.25

which has been removed now

@green-football-43791 I have used standard docker-compose.yml file

Can you please share with me the exact CLI command that you are running and the full output of the CLI logs when it fails?

@square-activity-64562 I use standard docker-compose.yml Datahub is started by CLI "datahub docker quickstart --quickstart-compose-file /home/user/datahub/config/docker-compose/docker-compose.yml" 1 issue: CLI: datahub-docker ingest-sample-data Logs are attached in txt file As so ExitCode is 407 so I think the problem is in proxy. 2 issue: I have added MsSQL Source in Ingestion via UI. When I try to Execute it, I get error as you could see on the screenshot above (in my first message) You could see multiple warnings like that:

'WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by '
           "'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5644fd4c40>, 'Connection to <http://pypi.org|pypi.org> timed out. "
           "(connect timeout=15)')': /simple/pip/\n"

After many reconnections to pypi.org you could see that there are no required libs for datahub.

The file that you sent me does not have any

pypi

errors.

I use ip adress of our server not

localhost

Datahub is accessable when I use link

<http://192.168.xx.xx:9002/ingestion>

So this doesn't work Recipe for MsSQL Source and logs from UI are attached. JFYI:To access Ithernet on our server, I need authorization through a proxy

@billowy-jewelry-4209 Let me ask someone as I am not familiar with adding proxy for the

actions

container

If I add environment variables for proxy for `actions`container so I get "N/A" status in the UI while Execute Ingestion and 407 error in logs for

actions

container:

user@server:~/path$ docker exec -it datahub_datahub-actions_1 /bin/bash -c env | grep proxy
ftp_proxy=<http://user_proxy:pass_proxy@proxy_server>:proxy_port/
https_proxy=<http://user_proxy:pass_proxy@proxy_server>:proxy_port/
http_proxy=<http://user_proxy:pass_proxy@proxy_server>:proxy_port/

user@server:~/path$ docker logs datahub_datahub-actions_1
2022/03/16 10:34:44 Waiting for: <http://datahub-gms:8080/health>
2022/03/16 10:34:44 Received 407 from <http://datahub-gms:8080/health>. Sleeping 1s
2022/03/16 10:34:45 Received 407 from <http://datahub-gms:8080/health>. Sleeping 1s
...

Can you share the config file that you created? That would be helpful for others in the community too

@billowy-jewelry-4209 This is really helpful! Thank you! I would love to understand how you configured this. Was the issue that your server could not connect to PyPi? Because you were on an internal network that doesn't directly allow access to pip?

@big-carpet-38439 You are qiute right. I have created file `pip.conf`:

[global]
proxy=<http://user:password@server>:port/

And then added it in

actions

container via `docker-compose.yml`:

datahub-actions:
    depends_on:
    ...
    environment:
    ...
    volumes:
    - ./pip.conf:/home/datahub/.pip/pip.conf

Pip can use both

/etc/pip.conf

(system config) and `~/.pip/pip.conf`(local user config) Issue: My server have access allow access to pip only via proxy authentification. So without proxy settings actions container couldn't reach the pypi servers.