https://gradle.com/ logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
k
I tried to pass the arguments many different ways but didn't succeed.. It grows even above 1.5G.. I already spent almost 2 days on it and have no idea how to debug / profile it
e
which column? assuming this is htop, VIRT and SHR are higher, but that's expected - that includes stuff outside the program's control such as the operating system caching the libraries and files. RES is within expectations - the JVM takes additional space for the JIT code cache
c
connect to it with jvisualvm or equivalent to see threadcount and the reserved / used sizes for each memory space segment.
k
thanks for tips, I ll try tomorrow and I ll let you know 🙂
v
c
Are you running any tests? I mean, they would create new forks and may have different memory constraints. (Or any other task that can fork the VM)
k
No, I do not run any tests in this specifc case and it only calls kotlin compiler which spawns new jvm instance.. I tried to run jvisualvm as suggested and it shows that the daemon really uses just something around 530M, so I was confused by the htop RES column, which still shows something around 9000-1000M .. so what is the reminder used for? is there some way to controll it? I tried to adjust the 
-XX:ReservedCodeCacheSize
to 
6400K
but that doesn't have significat impact..
the problem is that bitbucket pipelines only allows us to use either 4 or 8 GB of ram for each pipeline step.. and I need to run gradle build (daemon, gradle client and kotlin compiler and npm/node build executor) .. if it reaches the memory limit, it is killed, so I need to set some hard limits for the whole process
c
check the sizes of the non-heap pool in jvisualvm as well, they all add up to the resident memory size.
k
@Chris Lee do you mean the metaspace? It currently says 94MB used out of 293M max available .. which is weird because I specified 
-XX:MaxMetaspaceSize=256m
so it shouldn't be more 293 .. but still it doesn't give the total resident memory size
c
metaspace is part of the non-heap memory, as is code cache. the thread count * the stack size adds in there as well. Here’s an overview of the memory breakdown.
Can also set 
-XX:ReservedCodeCacheSize
to reduce memory, though this can come at a significant performance penalty. for server-class configurations the ergonomics sets this to 240m, for client-class configurations 48m.
Copy code
240m - Code Cache
256m - Metaspace
512m - Heap
xxx  - threads (thread count * stack size)
---
~1GB (max)

As only the max sizes are set it's up to the JVM when to allocate more memory; usually a good idea to set the minimum and maximum memory sizes to the same to avoid surprises (and fragmentation) when the JVM subequently attempts to allocate memory
-set InitialCodeCacheSize and ReservedCodeCacheSize to the same value
-set MetaspaceSize and MaxMetaspaceSize to the same value
-set -Xms and -Xmx to the same value

It can help to add `-XX:+AlwaysPreTouch` to force all memory pages to be initialized at startup (vs on allocation at a later time)
k
@Chris Lee thanks for all your input.. so now I tried to execute it using the following command:
Copy code
./gradlew -Dorg.gradle.jvmargs='-Xverify:none -XX:ReservedCodeCacheSize=240m -Xss1m -Xmx512m -Xms512m -XX:+AlwaysPreTouch -XX:MaxMetaspaceSize=256m -Dkotlin.daemon.jvm.options=-Xmx256m' --max-workers=1 --no-build-cache --no-configuration-cache distributionWithSsoTest testClasses;
and visualVM shows 512M heap, 256M meta space, 89 threads So my expectation for resident memory would be: 512M heap 256M meta space 240M code cache (not shown in visaulvm) 89*1 = 89M thread stack which is 1097M in total, but htop shows ~1520M so I assume those remaining 423M are used by shared libraries? Is there a way how to monitor it as well?
Seems like the LIB is only ~34M if I interpret it correctly.. Also, I m confused because I would expect that RES = CODE + DATA + LIB ... but it is actually less than DATA itself... O.o
ok, you can disregard my last confusion, I found that DATA includes the sets that doesn't have to be loaded into physical memory https://superuser.com/questions/1703495/data-memory-bigger-than-resident-memory-as-shown-in-htop
so I only need to figure out what are those remaining 423M
the native memory tracking shows that the whole process has committed 1,15G
Copy code
root@9adabf8bda0c:/# jcmd 8993 VM.native_memory summary
8993:

Native Memory Tracking:

(Omitting categories weighting less than 1KB)

Total: reserved=1598761KB, committed=1159449KB
but htop says that the resident memoery is 1.6G
3 Views
Open in Slack
PreviousNext
Powered by