Hi team, I kept getting the "Network is unreachable" error when trying to ingest BigQuery. My company uses proxy for outbound connection. I've set proxy in both linux os and docker and tested the connection fine. But still can't reach out to BQ in Datahub. Does BQ connector support the use of proxy? I've even included this line in the docker-compose file for the container datahub-frontend-react and still doesn't work:

"JAVA_OPTS=-Xms512m -Xmx512m -Dhttp.port=9002 -Dhttp.proxyHost=<IP> -Dhttp.proxyPort=<port> -Dhttps.proxyHost=<IP> -Dhttps.proxyPort=<port> -Dhttp.nonProxyHosts=localhost -Dconfig.file=datahub-frontend/conf/application.conf -Djava.security.auth.login.config=datahub-frontend/conf/jaas.conf -Dlogback.configurationFile=datahub-frontend/conf/logback.xml -Dlogback.debug=false -Dpidfile.path=/dev/null"

BQ ingestion error:

The error was: Deadline of 600.0s exceeded while calling target function, last exception: HTTPSConnectionPool(host='<http://oauth2.googleapis.com|oauth2.googleapis.com>', port=443): Max retries exceeded with url: /token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3e9cbd7310>: Failed to establish a new connection: [Errno 101] Network is unreachable')) Stacktrace: Traceback (most recent call last):\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/connection.py\", line 174, in _new_conn\n    conn = connection.create_connection(\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n    raise err\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n    sock.connect(sa)\nOSError: [Errno 101] Network is unreachable\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n    httplib_response = self._make_request(\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 386, in _make_request\n    self._validate_conn(conn)\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 1042, in _validate_conn\n    conn.connect()\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/connection.py\", line 363, in connect\n    self.sock = conn = self._new_conn()\n  File \"/usr/local/lib/python3.10/site-packages/urllib3/connection.py\", line 186, in _new_conn\n    raise NewConnectionError(\nurllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f3e9cbd7310>: Failed to establish a new connection: [Errno 101] Network is unreachable\n\n

(edited)

Good to hear you figured it out 🙂