A complete solution for open data platforms, enterprise data catalogs, data lakes and data management. Open source, mature, fully-featured and production ready.

DataHub

Hi Datahub

I am trying to build datahub frontend in an air gapped environment that allows to reach internet only via proxy.
To build the frontend i have specified the proxy on <https://github.com/linkedin/datahub/blob/master/docker/datahub-frontend/Dockerfile#L23|line> but gradlew is unable to reach <http://registry.npmjs.org|registry.npmjs.org>.

Also while building the gms similarly the build fails with error could not find tools.jar.

It is possible this was not tested with proxy. Some changes may be required in <https://github.com/linkedin/datahub/blob/master/datahub-web-react/build.gradle#L4>
Something like <https://github.com/srs/gradle-node-plugin/issues/63#issuecomment-293698448>

Which node/yarn repository urls are required to build datahub-frontend?
We need these to whitelist on our proxy.

Hi there. Currently we fetch node from here: <https://nodejs.org/dist/v16.8.0/>

That being said, when we run `yarn install` during the build of datahub-frontend, we do go and download many required packages from NPM registry in order to create a bundle which can be deployed. Therefore, in order to build this container locally, you'd either need an internal NPM registry with all of the dependencies (shown <https://github.com/linkedin/datahub/blob/master/datahub-web-react/package.json|here>) which will continually need to change as new dependencies are added, or you'll need to permit outbound access to NPM via whitelist (recommended)

image.png

Any other urls apart from the one showed in this error:

So this is fetching gradle dependencies, which are also required to complete a build.

There are many such dependencies that are pulled from repositories like maven, gradle plugins repository

This file shows the repositories we pull from for java artifacts: v

<https://github.com/linkedin/datahub/blob/master/repositories.gradle>

okay noted for gradle.
Please suggest if any url missing in below list for frontend:
<https://registry.yarnpkg.com/>
<https://registry.npmjs.org/>