This message was deleted.
# puppet-enterprises
Slackbot
08/26/2022, 3:14 PMThis message was deleted.
c
csharpsteen
08/26/2022, 3:22 PMIt came in with PE 2019.0 and Java 11. Basically, after performance testing, we found that:
• ParallelGC can reduce the maximum heap size below
-Xmxb
bastelfreak
08/26/2022, 3:23 PMthanks, will have a look. We were considering to switch to G1 because of our pxp-agent timeouts
n
NickB
08/26/2022, 3:40 PMIf you try this, let me know how it works out for you. Reevaluating G1 is on our todo list, especially since it lets you do things like
-XX:+UseStringDeduplicationn
nlew
08/26/2022, 5:59 PMI think the long GCs you’re seeing are actually when ParGC is trying to do just that: reduce the heap size below
-Xmxb
bastelfreak
08/26/2022, 6:03 PModd thing is that I have outofmemory logfiles from the puppetserver. they logged the java args and that says it used G1, not parallelGC
bastelfreak
08/26/2022, 6:04 PMmhmmm
bastelfreak
08/26/2022, 6:05 PMwhat if someone did some hiera cleanup, G1 was configured but changed by accident to parallelGC
bastelfreak
08/26/2022, 6:05 PMI need to check the git history on monday...
n
nlew
08/26/2022, 6:06 PMThe private ticket you linked describes G1 causing out of memory errors, and switching to ParGC to avoid them.
b
bastelfreak
08/26/2022, 6:07 PMif that in turn kills the pcp-broker, I choose G1 and throw more memory at it :D
n
nlew
08/26/2022, 6:07 PMMaybe that’s why it has so much memory allocated now?
b
bastelfreak
08/26/2022, 6:07 PMyeah
bastelfreak
08/26/2022, 6:09 PMwe reduced heap today on two compilers to 16G and on the other two to 20G, but saw timeouts again on a box with 20G. we increasedheap there to 31G
bastelfreak
08/26/2022, 6:09 PMalso I think default gc log retention is a bit.. not optimal. thise are 16 files with 65kb each. thats around 20min of history per file. we increased that as well and will check that + flight recorder on monday
bastelfreak
08/29/2022, 2:33 PMto come back to this: We've less used HEAP with G1GC (not significant). After 4 hours with G1GC we had no single FullGC/no delay in the expired CRL check from the pcp-broker
hooray 1
n
NickB
09/21/2022, 5:31 PM@bastelfreak Have you tried
-XX:UseStringDeduplicationb
bastelfreak
09/21/2022, 5:32 PMno we didn't set that
bastelfreak
09/21/2022, 5:33 PMI can take a look when I'm back in the office in two weeks. would that result in smaller HEAP?
n
NickB
09/21/2022, 5:33 PMIf you were inclined to give it a try, I’d be very interested 🙂
NickB
09/21/2022, 5:34 PMIt might. On Java 11, I’m not sure that G1 actually frees RAM back up, but it might be more efficient. Or it might not and cause increased pause times.
b
bastelfreak
09/21/2022, 5:35 PMI still would like to understand why we see so many full GCs that it breaks the pxp-agents. I haven't noticed it in other environments
bastelfreak
09/21/2022, 5:36 PMbut maybe that's also a common problem but where we just run so many tasks/plans that we notice it
n
NickB
09/21/2022, 5:36 PMYeah, kind of odd
b
bastelfreak
09/21/2022, 5:37 PMyup
bastelfreak
09/21/2022, 5:41 PMhttps://dzone.com/articles/usestringdeduplication
25 percent of Java applications memory is filled up with strings.? 13.5 percent are duplicate strings in Java applications.
Average string length is 45 characters.having that data for puppetserver/puppetdb would be interesting
n
NickB
09/21/2022, 5:48 PMYeah, for sure. I feel like it will probably help, but it would be good to measure in a real-world environment.
2 Views