https://datahubproject.io logo
Join Slack
Powered by
Running into an issue with the quickstart script w...
# getting-started
q
Running into an issue with the quickstart script where I am getting Java library initialization failed - unable to allocate file descriptor table - out of memory===> ENV Variables ... and similar from Zookeeper, Broker, and Schema Registry. Obviously this is preventing the app from properly launching. Fedora in VMware Fusion, 12gb ram, 40gb HDD, 4 cpu cores. From what I understand docker should have all the resources it needs available to it since i’m running in a linux host. I did try a couple of things. I was able to work around this one time by setting the default ulimit nofile for docker.service to something like 65535. However on a new fresh vm that workaround is no longer working. I’ve also tried editing the compose file and using a custom version to provide more memory to the JVM via the -Xmx parameter for the relevant containers. Neither have seemed to solve my issue this time around.
............. mysql is up-to-date elasticsearch is up-to-date Starting zookeeper ... Starting mysql-setup ... datahub-gms is up-to-date Starting zookeeper ... done Starting mysql-setup ... done Starting elasticsearch-setup ... done Starting broker ... done Starting schema-registry ... done kafka-setup is up-to-date ............. Unable to run quickstart - the following issues were detected: - kafka-setup is still running - schema-registry is not running - broker is not running - datahub-gms is still starting - zookeeper is not running
Last bit of output in the terminal for reference
g
We’re not quite sure what’s going on here, but I believe people have found success following the answer from here https://stackoverflow.com/a/69864234/5004662
Somehow the docker processes have too many files open? by chance, did you have anything else running in docker at the same time?
q
No. It was a fresh VM with only the QuickStart containers
However. It seems like setting ulimits per container in the compose file has helped. I’ve been able to replicate the issue and the fix 4-5 times now.
I’m new to docker and learning as I’m working through this but I ran into the MySQL 5.7 container leaking memory on Fedora and that lead me into the ulimits solution from a GitHub thread on the fedora MySQL container
g
That’s quite strange - haven’t seen mysql leak memory before
Mind linking the github thread?
q
Yeah I’ll have to go find it again. I’ll link it when I do
https://github.com/docker-library/mysql/issues/579#issuecomment-1074069882 here is the comment in particular that sent me down that path.
QuickStart runs perfectly fine in a Debian based *nix distro with the same specs. It’s leading me to believe it’s something related to the default system ulimits of Debian vs RHEL systems.
Until I move into our enterprise test environment I can’t say for sure with RHEL itself but the same solution solved the same issue in both fedora 36 and CentOS Stream 9
g
Got it, thanks for the link
2 Views
Open in Slack
PreviousNext
Powered by