https://datahubproject.io logo
Join Slack
Powered by
# all-things-deployment
  • b

    brief-insurance-68141

    09/23/2021, 12:04 AM
    it have error, which server host name I need to use?
  • b

    brief-insurance-68141

    09/23/2021, 12:04 AM
    datahub-datahub-gms:8080?
  • s

    some-glass-26087

    09/30/2021, 6:47 PM
    Kind of tricky, eh? For instance, I can't even run the command you pasted above as it complains that "docker build requires exactly 1 argument". Without "-f" it then tries to build the Dockerfile in ./docker/datahub-frontend, but the build context becomes that folder, not the root. Which is why
    gradlew
    is missing. What you want to run is this:
    sudo docker build -t updated_datahub_frontend_react -f ./docker/datahub-frontend/Dockerfile .
    Note the final
    .
    there -- that's giving the build context as the datahub root folder, and -f is specifying which Dockerfile to use (I tried just the directory for frontend, but it complains that it is a directory and failed to read dockerfile). It's still building on my machine (and my 13" macbook's fans are buzzing like crazy) but it doesn't fail on a missing
    gradlew
    like it did before.
    ❤️ 1
    🎉 1
  • s

    some-glass-26087

    09/30/2021, 6:47 PM
    @some-cricket-23089 ^ that's for you. 🙂
  • s

    some-cricket-23089

    10/01/2021, 6:27 AM
    @some-glass-26087 Thankyou so much for the detailed explanation with fix in the command. I also updated some of thee ask from @green-football-43791 in #ui. Let me try this command and let you know how it works now .
  • s

    some-cricket-23089

    10/01/2021, 12:28 PM
    @some-glass-26087 @green-football-43791 i tried the above fix it started to process now but now its failing with below error ..
    > Task :datahub-web-react:yarnInstall
    info Visit <https://yarnpkg.com/en/docs/cli/install> for documentation about this command.
    error /datahub-src/datahub-web-react/node_modules/cypress: Command failed.
    Exit code: 1
    Command: node index.js --exec install
    Arguments:
    Directory: /datahub-src/datahub-web-react/node_modules/cypress
    Output:
    Installing Cypress (version: 7.3.0)
    [12:21:54]  Downloading Cypress     [started]
    [12:25:19]  Downloading Cypress     [failed]
    The Cypress App could not be downloaded.
    Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: <https://on.cypress.io/proxy-configuration>
    Otherwise, please check network connectivity and try again:
    ----------
    URL: <https://download.cypress.io/desktop/7.3.0?platform=linux&arch=x64>
    Error: read ECONNRESET
    ----------
    Platform: linux (Alpine Linux - 3.14.2)
    Cypress Version: 7.3.0
    > Task :datahub-web-react:yarnInstall FAILED
    Note: Some input files use or override a deprecated API.
    Note: Recompile with -Xlint:deprecation for details.
    Note: Some input files use unchecked or unsafe operations.
    > Task :gms:impl:compileJava
    Note: Recompile with -Xlint:unchecked for details.
    FAILURE: Build failed with an exception.
    * What went wrong:
    Execution failed for task ':datahub-web-react:yarnInstall'.
    > Process 'command '/datahub-src/datahub-web-react/.gradle/yarn/yarn-v1.22.0/bin/yarn'' finished with non-zero exit value 1
    * Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
    * Get more help at <https://help.gradle.org>
    Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
    Use '--warning-mode all' to show the individual deprecation warnings.
    See <https://docs.gradle.org/5.6.4/userguide/command_line_interface.html#sec:command_line_warnings>
    BUILD FAILED in 13m 20s
    82 actionable tasks: 82 executed
    The command '/bin/sh -c cd datahub-src && ./gradlew :datahub-frontend:dist -PenableEmber=${ENABLE_EMBER} -x test -x yarnTest -x yarnLint     && cp datahub-frontend/build/distributions/datahub-frontend.zip ../datahub-frontend.zip     && cd .. && rm -rf datahub-src && unzip datahub-frontend.zip' returned a non-zero code: 1
  • s

    some-glass-26087

    10/01/2021, 3:56 PM
    @some-cricket-23089 This might be a longshot but can you tell what ip range your machine is on for its private internal address? How you do so will depend on your OS so if you can't figure it out, let me know your OS. One thing I've had happen a few times to me is that docker defaults to using the
    172.17.0.0/16
    subnet and if your work's network (I'm assuming you're in office, but perhaps not) uses the same subnet for their internal networking (like my company did), then containers can break being able to reach the net. Now if OTHER steps in the build are able to execute and download things just not from Cypress -- I'm not sure but will keep thinking on it. For some debugging, share the output of this:
    docker network ls
    and
    docker network inspect bridge
    This may not be the issue, I just want to eliminate ip conflicts as the source of not being able to reach out as I've had that happen to me twice with other projects.
  • s

    some-glass-26087

    10/01/2021, 4:14 PM
    Also to make sure we get a total rebuild and no previous layer is somehow giving us an issue, let's do the rebuild with no cache and pulling all recent images we need:
    docker build --no-cache --pull -t updated_datahub_frontend_react -f ./docker/datahub-frontend/Dockerfile .
  • s

    some-glass-26087

    10/01/2021, 4:15 PM
    This will take a little to run of course -- my macbook sounds like a jet engine right now after running it.
  • s

    some-cricket-23089

    10/01/2021, 6:32 PM
    @some-glass-26087 I tried again with the "--no-cache --pull" it's downloading the other packages like I pasted the log of downloading the gradle jar but broken with cypress
    ---> Running in 5bf8a68cb80d
    Downloading <https://services.gradle.org/distributions/gradle-5.6.4-bin.zip>
    .........................................................................................
    Welcome to Gradle 5.6.4!
    Here are the highlights of this release:
    - Incremental Groovy compilation
    - Groovy compile avoidance
    - Test fixtures for Java projects
    - Manage plugin versions via settings script
    For more details see <https://docs.gradle.org/5.6.4/release-notes.html>
    Configuration on demand is an incubating feature.
  • s

    some-cricket-23089

    10/01/2021, 6:33 PM
    Also Am using Ubuntu OS And below are the output for the command you shared above
  • s

    some-cricket-23089

    10/01/2021, 6:33 PM
    docker network ls
    NETWORK ID     NAME                  DRIVER    SCOPE
    cc7265d1a79b   astroef35b4_airflow   bridge    local
    dfb6dc300d3e   bridge                bridge    local
    b15d992f6557   datahub_network       bridge    local
    275733a3855a   host                  host      local
    6678e02b563f   none                  null      local
  • s

    some-cricket-23089

    10/01/2021, 6:34 PM
    sudo docker network inspect bridge
    [
    {
    "Name": "bridge",
    "Id": "dfb6dc300d3e986c381d90c662922e1b5178cdfc01b8a8607e7347e028038942",
    "Created": "2021-10-01T23:43:27.654904458+05:30",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": false,
    "IPAM": {
    "Driver": "default",
    "Options": null,
    "Config": [
    {
    "Subnet": "172.17.0.0/16",
    "Gateway": "172.17.0.1"
    }
    ]
    },
    "Internal": false,
    "Attachable": false,
    "Ingress": false,
    "ConfigFrom": {
    "Network": ""
    },
    "ConfigOnly": false,
    "Containers": {},
    "Options": {
    "com.docker.network.bridge.default_bridge": "true",
    "com.docker.network.bridge.enable_icc": "true",
    "com.docker.network.bridge.enable_ip_masquerade": "true",
    "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
    "com.docker.network.bridge.name": "docker0",
    "com.docker.network.driver.mtu": "1500"
    },
    "Labels": {}
    }
    ]
  • s

    some-glass-26087

    10/01/2021, 6:54 PM
    Okay, I'll try to look at this a bit later. You're right, it does seem like that first run of gradlew goes out and fetches newest gradle (I think) -- which means you ARE getting out the net via the container itself. I reran the command too and having no issues with Cypress and you said you can get to it manually too outside the container, so doubtful that is an issue now. For clarity's sake, let's look at the
    datahub_network
    as well:
    sudo docker network inspect datahub_network
  • s

    some-cricket-23089

    10/02/2021, 1:35 AM
    @some-glass-26087 Thankyou so much your time. below is the out for datahub_network
    [
    {
    "Name": "datahub_network",
    "Id": "b15d992f65574d1c5713b00ff149dabb69d0368c965f0d3de1691aefb663e26e",
    "Created": "2021-05-06T19:09:08.48414495+05:30",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": false,
    "IPAM": {
    "Driver": "default",
    "Options": null,
    "Config": [
    {
    "Subnet": "172.18.0.0/16",
    "Gateway": "172.18.0.1"
    }
    ]
    },
    "Internal": false,
    "Attachable": true,
    "Ingress": false,
    "ConfigFrom": {
    "Network": ""
    },
    "ConfigOnly": false,
    "Containers": {},
    "Options": {},
    "Labels": {
    "com.docker.compose.network": "datahub_network",
    "com.docker.compose.project": "datahub",
    "com.docker.compose.version": "1.29.1"
    }
    }
    ]
  • s

    some-glass-26087

    10/02/2021, 10:57 PM
    My pleasure @some-cricket-23089 I love helping people and hope to do more of it as I dive deeper into DataHub. Thanks for the network inspections. We'll sit on those for now -- it's so odd that your image build seems to be able to reach out and download some things (newer gradlew for instance) but not get to Cypress. So I suggest we try a smaller experiment. Throw together a small Dockerfile and package.json file in there that loads Cypress as a dependency too. And then have you fire it up as a test. I just want to see if it still breaks. I like trying to break down a problem into its smallest testable piece. What I hope happens is it ALSO fails, because that would mean we can focus in on why that domain is broken for you and maybe we can try some more things there. If it doesn't fail, then that makes it all the more mysterious haha. I have a bit of a busy weekend but you can give it a try if my instructions above make sense. If not, I'll throw that little test Dockerfile/project together and send it to you so we can try it next.
  • s

    some-cricket-23089

    10/03/2021, 4:44 PM
    @some-glass-26087 I will test it tomorrow and will let you know. I need to ready some package.json nd Dockerfile . If you have that handy with you please share .
  • s

    some-glass-26087

    10/03/2021, 11:15 PM
    @some-cricket-23089 I threw this together for you: https://github.com/thebouv/testcypress
  • s

    some-glass-26087

    10/03/2021, 11:17 PM
    Clone that and give the docker build command a try. The source is pretty simple. I kept it minimal. Basically just used yarn to init project, add cypress. Dockerfile just runs yarn install. Let's see what happens. Like I said previously I hope it fails for you in the same way as datahub because then we know it is something weird with your docker builds being able to reach out to fetch cypress. If it DOESN'T fail, try the datahub-frontend build again and let's see what happens.
  • s

    some-cricket-23089

    10/04/2021, 5:40 AM
    @some-glass-26087 The repo you shared with me is working fine.
  • s

    some-cricket-23089

    10/04/2021, 5:41 AM
    And the good news is that I again tried the datahub repo and that also worked fine. without any issue. Now I can build the docker image without any issue.
  • s

    some-cricket-23089

    10/04/2021, 5:42 AM
    The only thing I changed is used another high-speed internet and it worked well without any issue. So possibly it was the internet speed issue.
  • l

    little-megabyte-1074

    10/05/2021, 3:48 PM
    has renamed the channel from "docker-k8s" to "all-things-deployment"
  • l

    little-megabyte-1074

    10/05/2021, 3:49 PM
    set the channel topic: Channel to discuss all-things-deploying DataHub
  • l

    little-megabyte-1074

    10/05/2021, 3:49 PM
    set the channel description: Channel to discuss all-things-deploying DataHub
  • l

    little-megabyte-1074

    10/05/2021, 3:51 PM
    Hi folks! Just a heads up that I renamed this channel to be more encompassing of how we deploy DataHub, not only limited to Docker & K8s teamwork
    🙌 1
  • c

    calm-morning-92759

    10/29/2021, 8:13 AM
    @big-carpet-38439 thank you for your respone. That helps a lot. I found the extraEnvs as part of the Helm configuration for the frontend: The values overview for the frontend can be found here: https://github.com/acryldata/datahub-helm/tree/master/charts/datahub/subcharts/datahub-frontend
  • b

    bulky-intern-2942

    11/01/2021, 3:18 PM
    Thanks @green-football-43791!
  • s

    silly-translator-73123

    11/09/2021, 8:17 AM
    can i add user using UI now?
  • s

    silly-translator-73123

    11/09/2021, 9:29 AM
    anyone know this error?
1...4950515253Latest