I have been trying to build the containers with

COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -p datahub build

but I keep getting the error

UNABLE_TO_GET_ISSUER_CERT_LOCALLY

when it tries to build the datahub-frontend container. I tried changing the registry to the proxy that we need to use at my company through

.npmrc

and

.yarnrc

files, but I am still seeing

<https://registry.npmjs.org/yarn>

in the error log. How can I change that to my proxy so that it will build?

Not super familiar with working with NPM behind a proxy, does this help in any way? https://stackoverflow.com/questions/7559648/is-there-a-way-to-make-npm-install-the-command-to-work-behind-proxy

@some-glass-26087 any ideas?

@faint-painting-38451 My theory so far (till you tell me my assumptions below are wrong): Some companies run their own registries so they can do security scanning, version locking, etc to protect the pipeline (there have been lots of dependency / registry attacks to try and back door into things so this is understandable). My guess is that this local address may be protected via SSL, but likely a self-signed cert from your IT org. And even though you yourself can surf there without error that's likely because you have your company's CA bundle installed that allows this through. But

npm/yarn

(and lots of other tools) by default don't want to talk to talk to sites with self-signed certs. And anything building inside of docker wouldn't have the same company certs that you likely have on your machine. So you change the registry url, but npm is not liking it. A few ways to address: 1 You can see in the link from John above that you can tell npm to not be strict about bad ssl. I'm not a fan of this. I'm also not a fan of self-signed certs -- with LetsEncrypt and other options, cost usually isn't a barrier any longer but lots of IT teams aren't up to speed on those. So even though I'm not a fan of this, you can try setting

strict-ssl=false

in .npmrc and .yarnrc since you said you're already editing those files and rebuilding. There's also a way to feed that in via the build.gradle file but I'm not familiar enough with gradle to quite know where to put that (but would be happy to try and figure it out with you). 2 You can ask your IT org if you can have the CA bundle and cert info for that internal proxy/registry. Then we can figure out how to pass it / include it with the npm/yarn calls. This is what I've had to do several times to get around internal self-signed certs (though that's for python projects but I'm sure we can figure it out with npm/yarn as well). 3 You can ask your IT org to get with the program and put a real SSL cert on that device as I'm surprised it hasn't caused more issues honestly. 4 If you don't know if any of the above is true or not, ask a bit and come back with some more info if my assumptions were incorrect and that proxy/registry does have a legit cert. We may have to think about this in a different way. If so, maybe I can throw together some sort of test or experiment to help narrow down what the cause is.

@some-glass-26087 Thanks for the detailed writeup, but it looks like we bumped into this issue: https://github.com/node-gradle/gradle-node-plugin/issues/196 so I ended up looking at the setup they suggest here https://github.com/node-gradle/gradle-node-plugin/blob/master/docs/faq.md#how-do-i-specify-a-registry-for-the-npm-setup-task but I have little experience with Gradle and it turns out that, in this case, I got it to work by adding that

doFirst

block for

tasks.yarnSetup

instead of

tasks.npmSetup

. Also, adding this

tasks.yarnSetup

block to the root build.gradle file doesn't work since it can't find the yarnSetup task there, which I guess is because it has to happen in the file where

apply plugin: 'com.github.node-gradle.node'

is called so I managed to make it work by adding this

tasks.yarnSetup

block to

datahub-web-react/build.gradle

. I'd hate to have to maintain a patched version of this file internally, so would you be able to propose a better way of configuring this without having to change any code? Adjusting it via an environment variable would be great.

Wow! Great debugging there and getting to the root issue. I wasn't even considering gradle interacting with npm/yarn being a culprit, but was confused why you'd have networking issues with npm but not other things the build is pulling down. If you don't mind can you create an issue on the datahub repo? Gradle is out of my wheelhouse so I may have to point some other people at this for their thoughts. Also: can you clarify if this was because of using a custom registry or because you needed to pass other changes from npmrc/yarnrc for your proxy (like ignoring ssl, etc)? For instance in those examples you linked its all about having a custom registry. But since you were talking about a proxy in the original post I just want to be clear. If you have a version of your edited build.gradle file you could share that would be awesome too. Once again thanks for catching and doing such thorough debugging.

@some-glass-26087 Sure thing, I created this issue here: https://github.com/linkedin/datahub/issues/3349 We only need to use a custom registry URL for npm/yarn for any project which needs either/both of them. Please let me know if you need more details.