yep

it's now in an endless restart loop 😄

oh

but I also tried a real stop/start

Have you checked the puppet server conf.d dir. there should be one file with a ruby load path configured. The ruby load path should include the cache lib dir

usually that is available on PE

@bastelfreak ^^^

hey, are lockless code deploys considered experimental in PE 2021.5, or is it stable now? it's marked as experimental in 2021.4 but not in 2021.5 anymore

update to my satellite issue. I get the following trace in puppetserver.log:

2022-05-04T14:20:34.230+02:00 ERROR [qtp4533876-2176] [puppetserver] Puppet Server Error: Could not autoload puppet/reports/satellite: no such file to load -- puppet/util/satellite
org/jruby/RubyKernel.java:974:in `require'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:83:in `require'
/etc/puppetlabs/code/environments/tst_bas0/modules/satellite_pe_tools/lib/puppet/reports/satellite.rb:3:in `<main>'
org/jruby/RubyKernel.java:1009:in `load'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:78:in `load_file'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:182:in `load'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:54:in `block in loaded_instance'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/concurrent/lock.rb:10:in `synchronize'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:49:in `loaded_instance'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:27:in `block in report'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:52:in `block in processors'
org/jruby/RubyArray.java:1809:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:51:in `processors'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:30:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:14:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:316:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:181:in `do_save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:53:in `block in call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:302:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:52:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/server/v3.rb:17:in `block in wrap'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `block in process'
org/jruby/RubyArray.java:1809:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:86:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:69:in `block in with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:65:in `with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:85:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:92:in `respond_to_errors'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:84:in `process'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:69:in `block in handleRequest'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:302:in `override'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:68:in `handleRequest'

the pe-puppet-server.conf contains the correct load paths:

jruby-puppet: {
    # Where the puppet-agent dependency places puppet, facter, etc...
    # Puppet server expects to load Puppet from this location
    ruby-load-path: [
        "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby",
        "/opt/puppetlabs/puppet/cache/lib",
        "/opt/puppetlabs/server/apps/bolt-server/lib/ruby/vendor_ruby"
    ]

    # This setting determines where JRuby will install gems.  It is used for loading gems,
    # and also by the `puppetserver gem` command line tool.
    gem-home: /opt/puppetlabs/server/data/puppetserver/jruby-gems

    # This setting defines the complete "GEM_PATH" for jruby.  If set, it should include
    # the gem-home directory as well as any other directories that gems can be loaded
    # from (including the vendored gems directory for gems that ship with puppetserver)
    gem-path: [
      ${jruby-puppet.gem-home},
      "/opt/puppetlabs/server/data/puppetserver/vendored-jruby-gems",
      "/opt/puppetlabs/puppet/lib/ruby/vendor_gems"
    ]
...

# find /opt/puppetlabs/puppet/ -type f -name satellite.rb
/opt/puppetlabs/puppet/cache/lib/puppet/util/satellite.rb
/opt/puppetlabs/puppet/cache/lib/puppet/indirector/facts/satellite.rb
/opt/puppetlabs/puppet/cache/lib/puppet/reports/satellite.rb

update to my satellite issue. I get the following trace in puppetserver.log:

2022-05-04T14:20:34.230+02:00 ERROR [qtp4533876-2176] [puppetserver] Puppet Server Error: Could not autoload puppet/reports/satellite: no such file to load -- puppet/util/satellite
org/jruby/RubyKernel.java:974:in `require'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:83:in `require'
/etc/puppetlabs/code/environments/tst_bas0/modules/satellite_pe_tools/lib/puppet/reports/satellite.rb:3:in `<main>'
org/jruby/RubyKernel.java:1009:in `load'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:78:in `load_file'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/autoload.rb:182:in `load'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:54:in `block in loaded_instance'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/concurrent/lock.rb:10:in `synchronize'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:49:in `loaded_instance'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/instance_loader.rb:27:in `block in report'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:52:in `block in processors'
org/jruby/RubyArray.java:1809:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:51:in `processors'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:30:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:14:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:316:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:181:in `do_save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:53:in `block in call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:302:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:52:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/server/v3.rb:17:in `block in wrap'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `block in process'
org/jruby/RubyArray.java:1809:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:86:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:69:in `block in with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:65:in `with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:85:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:92:in `respond_to_errors'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:84:in `process'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:69:in `block in handleRequest'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:302:in `override'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:68:in `handleRequest'

the pe-puppet-server.conf contains the correct load paths:

jruby-puppet: {
    # Where the puppet-agent dependency places puppet, facter, etc...
    # Puppet server expects to load Puppet from this location
    ruby-load-path: [
        "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby",
        "/opt/puppetlabs/puppet/cache/lib",
        "/opt/puppetlabs/server/apps/bolt-server/lib/ruby/vendor_ruby"
    ]

    # This setting determines where JRuby will install gems.  It is used for loading gems,
    # and also by the `puppetserver gem` command line tool.
    gem-home: /opt/puppetlabs/server/data/puppetserver/jruby-gems

    # This setting defines the complete "GEM_PATH" for jruby.  If set, it should include
    # the gem-home directory as well as any other directories that gems can be loaded
    # from (including the vendored gems directory for gems that ship with puppetserver)
    gem-path: [
      ${jruby-puppet.gem-home},
      "/opt/puppetlabs/server/data/puppetserver/vendored-jruby-gems",
      "/opt/puppetlabs/puppet/lib/ruby/vendor_gems"
    ]
...

# find /opt/puppetlabs/puppet/ -type f -name satellite.rb
/opt/puppetlabs/puppet/cache/lib/puppet/util/satellite.rb
/opt/puppetlabs/puppet/cache/lib/puppet/indirector/facts/satellite.rb
/opt/puppetlabs/puppet/cache/lib/puppet/reports/satellite.rb

I think it's a bit odd that it refers to

/etc/puppetlabs/code/environments/tst_bas0/modules/satellite_pe_tools/lib/puppet/reports/satellite.rb

and not

/opt/puppetlabs/puppet/cache/lib/puppet/reports/satellite.rb

My settings are the same and it works for me

I wonder if puppetserver is not using the same environment where report processor is installed

why tst_bas0 and not production?

I've the same settings on two PE 2019 box and that works as well. just this new 2021 fails

company policy, PE infra doesn't run in production here

oh, haven't tried 2021 yet

I'm wondering if it's related to the new ruby version in the agent

loadpath from the puppetserver looks also fine:

# puppetserver irb
irb(main):001:0> puts $:
uri:classloader:/puppetserver-lib
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby
/opt/puppetlabs/puppet/cache/lib
/opt/puppetlabs/server/apps/bolt-server/lib/ruby/vendor_ruby
uri:classloader:/META-INF/jruby.home/lib/ruby/2.5/site_ruby
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib
=> nil
irb(main):002:0>

also loading it in irb works 🤔

# puppetserver irb
irb(main):001:0> require 'puppet/util/satellite'
=> true
irb(main):002:0>

I raised https://github.com/puppetlabs/puppetlabs-satellite_pe_tools/issues/179 for this

I raised https://github.com/puppetlabs/puppetlabs-satellite_pe_tools/issues/179 for this

Extensions like termini need to use require_relative to load helper code, because when running in a server context the environment specific directory is not added to the ruby LOAD_PATH

Or resolve the absolute path at runtime like https://github.com/puppetlabs/puppetlabs-splunk_hec/blob/6f2cd7beaddba20015a5d3833dacd79705b2d877/lib/puppet/indirector/facts/splunk_hec.rb#L3

Or resolve the absolute path at runtime like https://github.com/puppetlabs/puppetlabs-splunk_hec/blob/6f2cd7beaddba20015a5d3833dacd79705b2d877/lib/puppet/indirector/facts/splunk_hec.rb#L3

are you aware of any change between puppetserver 6/7 related to the LOAD_PATH?

This message was deleted.

@josh @vchepkov https://github.com/puppetlabs/puppetlabs-satellite_pe_tools/pull/180 the require with an absolute path indeed works on PE 2021

@josh @vchepkov https://github.com/puppetlabs/puppetlabs-satellite_pe_tools/pull/180 the require with ab absolute path indeed works on PE 2021

@josh @vchepkov https://github.com/puppetlabs/puppetlabs-satellite_pe_tools/pull/180 the require with an absolute path indeed works on PE 2021