https://www.puppet.com/community logo
Join Slack
Powered by
# puppet-enterprise
  • n
    we did run an 
    puppet infrastructure tune
    a little while ago
  • n
    Copy code
    puppet_enterprise::console_services::socket_timeout: 0
puppet_enterprise::profile::database::shared_buffers: 12931MB
puppet_enterprise::puppetdb::command_processing_threads: 3
puppet_enterprise::profile::puppetdb::java_args:
  Xms: 6465m
  Xmx: 6465m
puppet_enterprise::profile::orchestrator::jruby_max_active_instances: 4
puppet_enterprise::profile::orchestrator::java_args:
  Xms: 6465m
  Xmx: 6465m
puppet_enterprise::profile::console::java_args:
  Xms: 1024m
  Xmx: 1024m
puppet_enterprise::master::puppetserver::jruby_max_active_instances: 8
puppet_enterprise::profile::master::java_args:
  Xms: 8192m
  Xmx: 8192m
puppet_enterprise::master::puppetserver::reserved_code_cache: 768m
puppet_infrastructure_tune::metadata:
  pe_version: 2019.8.12
  tune_date: '2022-10-25'
  • s
    You might want to bump to 2021.7.3. There was a bug previously that caused some slowdown in the console that was resolved in that release.
  • n
    is that LTS?
  • b
    yes
  • n
    hmm.. ok
  • n
    @spp are you referring to this note?
    Copy code
    Issue that caused an unexpected increase in CPU usage is resolved
In PE 2021.7.1, 2021.7.2, and 2023.0, an issue with Puppet Server caused an unexpected increase in central processing unit (CPU) usage in some environments. CPU usage continued to grow and some operations took longer than expected until the Puppet Server service was restarted. This issue is resolved in PE 2023.1 and 2021.7.3.
  • s
    That wasn't the specific issue I was thinking of. Actually, looking back into the ticket, it doesn't match your use case, so probably isn't relevant. My comment was probably a red herring.
  • b
    but still: latest is greatest, so update :)
  • n
    the other one I am thinking that you referred to is
    Copy code
    Improved performance for the each, map, and filter functions in the Puppet language
Previously, the Puppet language built-in functions each, map, and filter showed poor performance and consumed unnecessary resources when run on JRuby software. The issue was resolved to enhance performance.
  • n
    but this seems to be more for the puppet code itself and not fact rules
  • n
    agree @bastelfreak will look into upgrading. though we are only 1 bugfix version out 😉
  • c
    2021.7.3 has a fairly large set of performance fixes. Including several to node group matching.
  • s
    The one I was thinking of wasn't mentioned specifically in the release notes. It had to do with node selection on tasks being slow.
  • n
    ah ok..
  • n
    thanks @csharpsteen, I couldn't really see much in the release notes though. Do you have any additional information I could be given?
  • r
    Seems that after the last of RHEL patches our Primary Servers aren't happy. Keep dropping connections to agents and Impact Analysis in CD4PE appears to fail from not being able to complete. Have had to crank up memory resources to let it work again.
  • v
    Documentation doesn't mention if any cleanup needs to be done after switching to lockless code deploys
  • v
  • s
    This message was deleted.
    b
    v
    • 3
    • 25
  • b
    morning, is there an option to increase the filesync client timeout in puppetserver/orchestrator?
    Copy code
    2023-05-25T08:58:47.151+02:00 ERROR [clojure-agent-send-off-pool-92594] [p.e.file-sync-errors] File Sync failure during sync or fetch phase: Unable to get latest-commits from server (<https://primary:8140/file-sync/v1/latest-commits>).
java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-97304 [ACTIVE]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:98)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:40)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:506)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
        at java.base/java.lang.Thread.run(Thread.java:829)
    Or is that the generic http client timeout?
  • g
    you can adjust 
    timeouts_deploys
    or 
    timeouts_sync
    (or the other 
    timeouts_*
    parameters) https://www.puppet.com/docs/pe/2021.7/code_mgr_customizing.html#code_manager_parameters however those would affect sync for all file sync clients (ie: compilers too) not just orchestrator
  • b
    aren't those the timeouts in code manager itself?
  • g
    is there a plan run blocking the deploy?
  • b
    no. just sometimes many concurrent deploys and busy hardware 😄
  • b
    the errors sounds to me like a http client timeout, not within code manager
  • b
    and the different timeout options in codemanager are already increased, none of them is at 30 seconds anymore
  • g
    i think timeouts_sync defaults to 300 not 30 (and the others are mostly also in the hundreds)
  • b
    timeouts_fetch and timeouts_hook default to 30s
  • g
    file sync troubleshooting mentions timeouts_sync (or allowing plans alongside code deployments) but no specifics about where the timeouts are: https://www.puppet.com/docs/pe/2021.7/code_mgr_troubleshoot.html#code_deployments_time_out
1...545556...73Latest