Puppet Community

We've talked about precalculating all of the file metadata &amp; checksums during a code deploy for an environment, like how you can run `puppet resource generate` to generate the pcore resource types.

otherwise someone's going to get a real nightmare troubleshooting situation if they don't realize that cache is happening

if there isnt somethi g precalculated to deliver the puppetserver could still do it on the fly (just for the requested file )

right, I am thinking about the opposite problem, someone deploys and then tweaks a file to test a small change (which is a terrible strategy, but it happens)

ah cache invalidation is one of the easiest things in the world <@U020LK7D0JU>!

putting nginx with some lua in front of puppetsetver sounds like a solution for most issues

I'm throwing you out an airlock in my mind right now

nginx could serve all the static files and even calculate the checksums on the fly + cache them and PE even already ships an nginx. and  it can handle TLS offloading way better than the jvm

tbh, if <https://github.com/puppetlabs/docs-archive/blob/main/puppet/4.4/configuration.md#pluginsync> wasn't deprecated, you could just turn off pluginsync and replicate it by having r10k build a tarball on deploy, then manage a file resource and an exec resource to uncompress it in a run stage. :shrug:

Yeah I think all of those have pluses and minuses. Using something-other-than-puppetserver for fileserving is a big win. But the `recurse =&gt; true` behavior complicates things and the response needs to conform to the fileserving REST API

I think the majority of the resources do not use recurse =&gt; true

The tarball approach is nice when deploying a new agent, but you also don't want to download everything again when a single fact changes.

and if they do, that could still be forwarded to puppetserver

<@U1PC2RAH0> ah sorry I mean pluginsync is implemented as a recursive file resource

I used the nginx in front of puppetserver in the past to terminate tls and serve static file resources and that worked really well

I'd prefer to download the whole tarball when a single fact changes over a recursive file resource.

mhm not sure. extracting that on the agent is also expensive

sure, but how often does the extraction happen compared to the recursive checksums that happen every time?

Lifting File Serving out of Ruby and into Java would also help, similar to an NGINX offload but without an external dependency. Last time I looked, it was about 10x faster and there are 10x more threads available if you’re not queuing for the JRuby pool.

Although, OpenTracing helps the most for digging into outliers.

For overall service operation, these metrics should give an idea of how much time is going into File Serving vs. other operations:

<https://github.com/puppetlabs/puppetserver/pull/1689>

your jq examples should be put into the official docs