Puppet Community

depending on the puppetserver version, it does

latest Puppetserver 6 and 7 do that by default, on a new installation

that must be it then as it's a pretty recent 6 install

well, that's annoying, I'll have to work out what to do with these IOT raspberry pi devices, thank you, that's saved me a lot of false effort

I'm not sure if that was backported to Puppet 5. on a quick check I didn't see it at <https://puppet.com/docs/puppet/5.5/release_notes.html>. keep in mind Puppet 5 is EoL. you should really not use it anymore

as a workaround, you could setup the puppetserver without intermediate CA

the fact that tit's EOL and the open source repos are not building anything for later puppet versions on arm apart from ubuntu 20.04 (which is fine for most of my stuff)

yes proper ARM support would be nice :disappointed:

at least for the agent on different architectures

ubuntu 20.04 on arm works and that's useful but that's pretty much 'it'

you can check <#C016WALE1V4|>, some people built their own packages for different ARM architectures

you can check <#C016WALE1V4|pi>, some people built their own packages for different ARM architectures

Hello , I am a beginner in puppet and was trying to see if we can call a puppet function ( or invoke a module) in a bats script. Could someone help clarify if this is possible ?

For custom functions, you might lookup the function's ruby script in `/opt/puppetlabs/puppet/cache/lib/puppet/functions` and call that? Though I wouldn't know how to do that. On top of that, build-in functions as well as functions written in Puppet's own language are not found there.

Thank you for that but I don't think it works :disappointed: It errors out saying unknown function
 `Error: Evaluation Error: Unknown function`  <@U2S1MUUQ7>

Sorry in case I'm violating guidelines, but I want to drop a reference to <https://github.com/puppetlabs/community/discussions/21|my open question on puppetlabs/community GitHub project>. Maybe someone can invest some time?

I don't know why Puppet reports the error in this way. The implementation of the `multipath`class is not mentioned, because it simply is a vehicle to forward the input data to a template, which writes `multipath.conf`. There is no other intelligence applied. Certainly, I can put the sources into the GitHub discussion, if you like.

We’re gonna have to see how and where that `Multipath::Config` data type is used.
As is it now, there’s not enough information for us (well, for me at least) to diagnose the problem.

Is that `Multipath::Config` datatype used to validate a class parameter? If so, which parameter, and how is that class being declared?

Please show actual code snippets of that.

Since this happens with `puppet apply` and not `puppet agent`, I’m guessing the issue is that the data used when declaring the class is either different or incomplete.

Hi <@U112RM52S>, I've posted the `multipath` class definition on the <https://github.com/puppetlabs/community/discussions/21#discussioncomment-2813407|GitHub discussion>.

This happens for `puppet agent`! I mentioned `puppet apply` only cursory.

Yes, this data type mainly has to validate the input data. I don't think I can show you how `multipath` class is declared. Because the master is able to produce the catalog, I'd say that isn't relevant, but I'll still try to provide insights without including all the details, if you insist.

What data is being supplied to the `$config` parameter of the `multipath` class?

Can you explain to us, or show us with code, how and where the `multipath` class is declared?
That’s the missing piece of info we’d need to properly troubleshoot this.

Hi <@U112RM52S>, thank you again for your time. The <https://github.com/puppetlabs/community/discussions/21#discussioncomment-2813508|GitHub discussion> is updated with some information. Please tell me if that is enough.

The error messages indicate that the data coming in does not match the data type’s spec.
Have you tried to verify the values of the data coming in to make sure all the expected keys are present?

Also, where are those variables, `$multipath_defaults` and `$multipath_devices`, defined in relation to where the class is declared?

Also perhaps that custom fact that’s parsed as json is different than you expect?

As I had shown in the initial post of the GitHub discussion, the catalog produced by the master (fetched from API, not applied through `puppet agent`) the class parameters look exactly as the should.

In-code type validation with e.g. `$multipath_devices =~ Array[Multipath::Config::Device]` also returns `true`.

Running `puppet agent -t` and fetching a catalog via the `puppet/v3/catalog` APi are not the same thing though.
The latter is likely using cached facts while the former is producing fresh fact values for the compilation.

Facts are not cache specific to Puppet environments, or are they?

Fact caches are not tied to an environment.

Then this should not be a problem either. `puppet agent` with production environment succeeds and thus sends current facts to the master for caching.

Backing up a bit here, the errors messages clearly indicate that the problem is with the data coming into the class’s `$config` parameter.
Personally, I would start troubleshooting by double checking every piece of input and confirming it is what I expect it to be. Don’t assume anything. Validate everything.

Perhaps try some `notify` resources to print the variables, the fact value, and the `parsejson()` result.

I'm sorry, not to discredit your opinion, but I would not have started this here discussion, if I hadn't done that already. :wink: Tell me what you want to see.

`notify` only works when I'm _not_ applying the `multipath`class, since that aborts `puppet agent`, but I can do that, if you want to see it.

That’s fair, no worries :slightly_smiling_face:

And in that case yeah, perhaps showing the output from some `notify` resources while also commenting out the class declaration would help.
And I’m also interested in seeing the full PUppet run output and the command you’re using the run Puppet.

(assuming it’s okay to share that information publicly, feel free to redact if needed)

I have to apologize, I tried to get a newly compiled catalog via API, after running `puppet agent` in production environment, and that has now failed, too, with the same errors

Ah, well at least the failure is consistent now. That removes some mystery.

Well, I'm uncertain as to how much information is needed. I don't want to just vomit all information for you to search through. There should be little privacy concerns, I'd say.

If you agree, I'd send that in private conversation?

Some of the things I’d like to rule out:
1) You’re running against the Puppetserver you expect, and not some other Puppetserver.
2) You’re running against the environment you expect, and not some other environment with different code.
3) You’re running `puppet agent` in a “normal” way, and not some non-standard way that might interfere with the catalog compilation.

And then on top of that, seeing the value of all those variables used as input to the `$config` parameter via a `notify` resource.

Usually just seeing what command you run (e.g. `sudo puppet agent -t --environment foo`), and the first 10 or so lines of the output will help to answer 1, 2, and 3.

1. I explicitly set `--server` for `puppet agent`. That was necessary for `notify` debugging... :slightly_smiling_face:
2. Yes, I am using the correct feature environment.
3. Counts `puppet agent -t --noop --environment multipath_transition_to_Puppet_data_types --server <http://p6master-03.gridka.de|p6master-03.gridka.de>` normal?

These are the notified parameters
```Notice: /Stage[main]/Gridka_gxfs_nsdserver::Multipath/Notify[Multipath devices]/message: current_value 'absent', should be '[{\'vendor\' =&gt; \'(LSI|NETAPP)\', \'product\' =&gt; \'INF-01-00\', \'product_blacklist\' =&gt; \'Universal Xport\', \'path_grouping_policy\' =&gt; \'group_by_prio\', \'path_selector\' =&gt; \'service-time 0\', \'path_checker\' =&gt; \'rdac\', \'features\' =&gt; \'2 pg_init_retries 50\', \'hardware_handler\' =&gt; \'1 rdac\', \'prio\' =&gt; \'rdac\', \'no_path_retry\' =&gt; 30, \'rr_min_io\' =&gt; 100, \'rr_min_io_rq\' =&gt; 1}]' (noop)
Notice: /Stage[main]/Gridka_gxfs_nsdserver::Multipath/Notify[Multipath multipaths]/message: current_value 'absent', should be '[{\'alias\' =&gt; \'119_912_DDP05_03\', \'wwid\' =&gt; \'36d039ea00009466f0000010f5e30445e\'}, {\'alias\' =&gt; \'119_912_DDP05_04\', \'wwid\' =&gt; \'36d039ea00009466f000001105e304483\'}, {\'alias\' =&gt; \'119_912_DDP06_03\', \'wwid\' =&gt; \'36d039ea00009466f000001115e3044e6\'}, {\'alias\' =&gt; \'119_912_DDP06_04\', \'wwid\' =&gt; \'36d039ea00009466f000001125e30450c\'}, {\'alias\' =&gt; \'119_934_DDP03_01\', \'wwid\' =&gt; \'36d039ea000094291000000ef5e303fff\'}, {\'alias\' =&gt; \'119_934_DDP03_02\', \'wwid\' =&gt; \'36d039ea000094291000000f15e304019\'}, {\'alias\' =&gt; \'119_934_DDP04_01\', \'wwid\' =&gt; \'36d039ea000094291000000f25e304079\'}, {\'alias\' =&gt; \'119_934_DDP04_02\', \'wwid\' =&gt; \'36d039ea000094291000000f45e304093\'}, {\'alias\' =&gt; \'119_934_DDP05_01\', \'wwid\' =&gt; \'36d039ea000094291000000f55e3040f5\'}, {\'alias\' =&gt; \'119_934_DDP01_01\', \'wwid\' =&gt; \'36d039ea000094291000000e95e303f06\'}, {\'alias\' =&gt; \'119_934_DDP05_02\', \'wwid\' =&gt; \'36d039ea000094291000000f75e30410f\'}, {\'alias\' =&gt; \'119_934_DDP06_01\', \'wwid\' =&gt; \'36d039ea000094291000000f85e304172\'}, {\'alias\' =&gt; \'119_934_DDP06_02\', \'wwid\' =&gt; \'36d039ea000094291000000fa5e30418d\'}, {\'alias\' =&gt; \'119_912_DDP01_01\', \'wwid\' =&gt; \'36d039ea00009463d000000fa5e3041e5\'}, {\'alias\' =&gt; \'119_934_DDP01_02\', \'wwid\' =&gt; \'36d039ea000094291000000eb5e303f28\'}, {\'alias\' =&gt; \'119_912_DDP01_02\', \'wwid\' =&gt; \'36d039ea00009463d000000fc5e30420a\'}, {\'alias\' =&gt; \'119_912_DDP02_01\', \'wwid\' =&gt; \'36d039ea00009463d000000fd5e304271\'}, {\'alias\' =&gt; \'119_912_DDP02_02\', \'wwid\' =&gt; \'36d039ea00009463d000000ff5e30428d\'}, {\'alias\' =&gt; \'119_912_DDP03_01\', \'wwid\' =&gt; \'36d039ea00009463d000001005e3042f4\'}, {\'alias\' =&gt; \'119_912_DDP03_02\', \'wwid\' =&gt; \'36d039ea00009463d000001025e304311\'}, {\'alias\' =&gt; \'119_912_DDP04_01\', \'wwid\' =&gt; \'36d039ea00009463d000001035e304378\'}, {\'alias\' =&gt; \'119_912_DDP04_02\', \'wwid\' =&gt; \'36d039ea00009463d000001055e304396\'}, {\'alias\' =&gt; \'119_912_DDP05_01\', \'wwid\' =&gt; \'36d039ea00009463d000001065e3043ff\'}, {\'alias\' =&gt; \'119_912_DDP05_02\', \'wwid\' =&gt; \'36d039ea00009463d000001085e30441c\'}, {\'alias\' =&gt; \'119_912_DDP06_01\', \'wwid\' =&gt; \'36d039ea00009463d000001095e304486\'}, {\'alias\' =&gt; \'119_912_DDP06_02\', \'wwid\' =&gt; \'36d039ea00009463d0000010b5e3044a4\'}, {\'alias\' =&gt; \'119_934_DDP01_03\', \'wwid\' =&gt; \'36d039ea00009477d000001325e303f05\'}, {\'alias\' =&gt; \'119_934_DDP01_04\', \'wwid\' =&gt; \'36d039ea00009477d000001335e303f27\'}, {\'alias\' =&gt; \'119_934_DDP02_01\', \'wwid\' =&gt; \'36d039ea000094291000000ec5e303f86\'}, {\'alias\' =&gt; \'119_934_DDP02_03\', \'wwid\' =&gt; \'36d039ea00009477d000001345e303f7d\'}, {\'alias\' =&gt; \'119_934_DDP02_04\', \'wwid\' =&gt; \'36d039ea00009477d000001355e303f9f\'}, {\'alias\' =&gt; \'119_934_DDP03_03\', \'wwid\' =&gt; \'36d039ea00009477d000001365e303ff6\'}, {\'alias\' =&gt; \'119_934_DDP03_04\', \'wwid\' =&gt; \'36d039ea00009477d000001375e304019\'}, {\'alias\' =&gt; \'119_934_DDP02_02\', \'wwid\' =&gt; \'36d039ea000094291000000ee5e303f9f\'}, {\'alias\' =&gt; \'119_934_DDP04_03\', \'wwid\' =&gt; \'36d039ea00009477d000001385e304071\'}, {\'alias\' =&gt; \'119_934_DDP04_04\', \'wwid\' =&gt; \'36d039ea00009477d000001395e304094\'}, {\'alias\' =&gt; \'119_934_DDP05_03\', \'wwid\' =&gt; \'36d039ea00009477d0000013a5e3040ed\'}, {\'alias\' =&gt; \'119_934_DDP05_04\', \'wwid\' =&gt; \'36d039ea00009477d0000013b5e304111\'}, {\'alias\' =&gt; \'119_934_DDP06_03\', \'wwid\' =&gt; \'36d039ea00009477d0000013c5e30416c\'}, {\'alias\' =&gt; \'119_934_DDP06_04\', \'wwid\' =&gt; \'36d039ea00009477d0000013d5e304190\'}, {\'alias\' =&gt; \'119_912_DDP01_03\', \'wwid\' =&gt; \'36d039ea00009466f000001075e30424a\'}, {\'alias\' =&gt; \'119_912_DDP01_04\', \'wwid\' =&gt; \'36d039ea00009466f000001085e30426f\'}, {\'alias\' =&gt; \'119_912_DDP02_03\', \'wwid\' =&gt; \'36d039ea00009466f000001095e3042ce\'}, {\'alias\' =&gt; \'119_912_DDP02_04\', \'wwid\' =&gt; \'36d039ea00009466f0000010a5e3042f3\'}, {\'alias\' =&gt; \'119_912_DDP03_03\', \'wwid\' =&gt; \'36d039ea00009466f0000010b5e304351\'}, {\'alias\' =&gt; \'119_912_DDP03_04\', \'wwid\' =&gt; \'36d039ea00009466f0000010c5e304377\'}, {\'alias\' =&gt; \'119_912_DDP04_03\', \'wwid\' =&gt; \'36d039ea00009466f0000010d5e3043d7\'}, {\'alias\' =&gt; \'119_912_DDP04_04\', \'wwid\' =&gt; \'36d039ea00009466f0000010e5e3043fd\'}]' (noop)```

This is the code of the notifies
```notify { 'Multipath devices':
  message =&gt; String($multipath_devices),
}
notify { 'Multipath multipaths':
  message =&gt; String($gridka_gxfs_multipath_aliases_struct),
}```

I could try notifying the entire `multipath::config` parameter from within the multipath class. Though I'm not sure whether that is possible...

I was possible!
```Notice: /Stage[main]/Multipath/Notify[{'defaults' =&gt; {'hw_str_match' =&gt; 'yes', 'user_friendly_names' =&gt; 'yes', 'verbosity' =&gt; 2, 'polling_interval' =&gt; 5, 'multipath_dir' =&gt; '/lib64/multipath', 'path_selector' =&gt; 'round-robin 0', 'path_grouping_policy' =&gt; 'failover', 'prio' =&gt; 'const', 'features' =&gt; '0', 'path_checker' =&gt; 'directio', 'failback' =&gt; 'manual', 'rr_min_io' =&gt; 100, 'rr_min_io_rq' =&gt; 1, 'max_fds' =&gt; 1048576, 'rr_weight' =&gt; 'uniform', 'queue_without_daemon' =&gt; 'no', 'flush_on_last_del' =&gt; 'no', 'log_checker_err' =&gt; 'always'}, 'devices' =&gt; [{'vendor' =&gt; '(LSI|NETAPP)', 'product' =&gt; 'INF-01-00', 'product_blacklist' =&gt; 'Universal Xport', 'path_grouping_policy' =&gt; 'group_by_prio', 'path_selector' =&gt; 'service-time 0', 'path_checker' =&gt; 'rdac', 'features' =&gt; '2 pg_init_retries 50', 'hardware_handler' =&gt; '1 rdac', 'prio' =&gt; 'rdac', 'no_path_retry' =&gt; 30, 'rr_min_io' =&gt; 100, 'rr_min_io_rq' =&gt; 1}], 'blacklist' =&gt; {'devnode' =&gt; ['^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*', '^hd[a-z]']}, 'multipaths' =&gt; [{'alias' =&gt; '119_912_DDP05_03', 'wwid' =&gt; '36d039ea00009466f0000010f5e30445e'}, {'alias' =&gt; '119_912_DDP05_04', 'wwid' =&gt; '36d039ea00009466f000001105e304483'}, {'alias' =&gt; '119_912_DDP06_03', 'wwid' =&gt; '36d039ea00009466f000001115e3044e6'}, {'alias' =&gt; '119_912_DDP06_04', 'wwid' =&gt; '36d039ea00009466f000001125e30450c'}, {'alias' =&gt; '119_934_DDP03_01', 'wwid' =&gt; '36d039ea000094291000000ef5e303fff'}, {'alias' =&gt; '119_934_DDP03_02', 'wwid' =&gt; '36d039ea000094291000000f15e304019'}, {'alias' =&gt; '119_934_DDP04_01', 'wwid' =&gt; '36d039ea000094291000000f25e304079'}, {'alias' =&gt; '119_934_DDP04_02', 'wwid' =&gt; '36d039ea000094291000000f45e304093'}, {'alias' =&gt; '119_934_DDP05_01', 'wwid' =&gt; '36d039ea000094291000000f55e3040f5'}, {'alias' =&gt; '119_934_DDP01_01', 'wwid' =&gt; '36d039ea000094291000000e95e303f06'}, {'alias' =&gt; '119_934_DDP05_02', 'wwid' =&gt; '36d039ea000094291000000f75e30410f'}, {'alias' =&gt; '119_934_DDP06_01', 'wwid' =&gt; '36d039ea000094291000000f85e304172'}, {'alias' =&gt; '119_934_DDP06_02', 'wwid' =&gt; '36d039ea000094291000000fa5e30418d'}, {'alias' =&gt; '119_912_DDP01_01', 'wwid' =&gt; '36d039ea00009463d000000fa5e3041e5'}, {'alias' =&gt; '119_934_DDP01_02', 'wwid' =&gt; '36d039ea000094291000000eb5e303f28'}, {'alias' =&gt; '119_912_DDP01_02', 'wwid' =&gt; '36d039ea00009463d000000fc5e30420a'}, {'alias' =&gt; '119_912_DDP02_01', 'wwid' =&gt; '36d039ea00009463d000000fd5e304271'}, {'alias' =&gt; '119_912_DDP02_02', 'wwid' =&gt; '36d039ea00009463d000000ff5e30428d'}, {'alias' =&gt; '119_912_DDP03_01', 'wwid' =&gt; '36d039ea00009463d000001005e3042f4'}, {'alias' =&gt; '119_912_DDP03_02', 'wwid' =&gt; '36d039ea00009463d000001025e304311'}, {'alias' =&gt; '119_912_DDP04_01', 'wwid' =&gt; '36d039ea00009463d000001035e304378'}, {'alias' =&gt; '119_912_DDP04_02', 'wwid' =&gt; '36d039ea00009463d000001055e304396'}, {'alias' =&gt; '119_912_DDP05_01', 'wwid' =&gt; '36d039ea00009463d000001065e3043ff'}, {'alias' =&gt; '119_912_DDP05_02', 'wwid' =&gt; '36d039ea00009463d000001085e30441c'}, {'alias' =&gt; '119_912_DDP06_01', 'wwid' =&gt; '36d039ea00009463d000001095e304486'}, {'alias' =&gt; '119_912_DDP06_02', 'wwid' =&gt; '36d039ea00009463d0000010b5e3044a4'}, {'alias' =&gt; '119_934_DDP01_03', 'wwid' =&gt; '36d039ea00009477d000001325e303f05'}, {'alias' =&gt; '119_934_DDP01_04', 'wwid' =&gt; '36d039ea00009477d000001335e303f27'}, {'alias' =&gt; '119_934_DDP02_01', 'wwid' =&gt; '36d039ea000094291000000ec5e303f86'}, {'alias' =&gt; '119_934_DDP02_03', 'wwid' =&gt; '36d039ea00009477d000001345e303f7d'}, {'alias' =&gt; '119_934_DDP02_04', 'wwid' =&gt; '36d039ea00009477d000001355e303f9f'}, {'alias' =&gt; '119_934_DDP03_03', 'wwid' =&gt; '36d039ea00009477d000001365e303ff6'}, {'alias' =&gt; '119_934_DDP03_04', 'wwid' =&gt; '36d039ea00009477d000001375e304019'}, {'alias' =&gt; '119_934_DDP02_02', 'wwid' =&gt; '36d039ea000094291000000ee5e303f9f'}, {'alias' =&gt; '119_934_DDP04_03', 'wwid' =&gt; '36d039ea00009477d000001385e304071'}, {'alias' =&gt; '119_934_DDP04_04', 'wwid' =&gt; '36d039ea00009477d000001395e304094'}, {'alias' =&gt; '119_934_DDP05_03', 'wwid' =&gt; '36d039ea00009477d0000013a5e3040ed'}, {'alias' =&gt; '119_934_DDP05_04', 'wwid' =&gt; '36d039ea00009477d0000013b5e304111'}, {'alias' =&gt; '119_934_DDP06_03', 'wwid' =&gt; '36d039ea00009477d0000013c5e30416c'}, {'alias' =&gt; '119_934_DDP06_04', 'wwid' =&gt; '36d039ea00009477d0000013d5e304190'}, {'alias' =&gt; '119_912_DDP01_03', 'wwid' =&gt; '36d039ea00009466f000001075e30424a'}, {'alias' =&gt; '119_912_DDP01_04', 'wwid' =&gt; '36d039ea00009466f000001085e30426f'}, {'alias' =&gt; '119_912_DDP02_03', 'wwid' =&gt; '36d039ea00009466f000001095e3042ce'}, {'alias' =&gt; '119_912_DDP02_04', 'wwid' =&gt; '36d039ea00009466f0000010a5e3042f3'}, {'alias' =&gt; '119_912_DDP03_03', 'wwid' =&gt; '36d039ea00009466f0000010b5e304351'}, {'alias' =&gt; '119_912_DDP03_04', 'wwid' =&gt; '36d039ea00009466f0000010c5e304377'}, {'alias' =&gt; '119_912_DDP04_03', 'wwid' =&gt; '36d039ea00009466f0000010d5e3043d7'}, {'alias' =&gt; '119_912_DDP04_04', 'wwid' =&gt; '36d039ea00009466f0000010e5e3043fd'}]}]/message: current_value 'absent', should be '{\'defaults\' =&gt; {\'hw_str_match\' =&gt; \'yes\', \'user_friendly_names\' =&gt; \'yes\', \'verbosity\' =&gt; 2, \'polling_interval\' =&gt; 5, \'multipath_dir\' =&gt; \'/lib64/multipath\', \'path_selector\' =&gt; \'round-robin 0\', \'path_grouping_policy\' =&gt; \'failover\', \'prio\' =&gt; \'const\', \'features\' =&gt; \'0\', \'path_checker\' =&gt; \'directio\', \'failback\' =&gt; \'manual\', \'rr_min_io\' =&gt; 100, \'rr_min_io_rq\' =&gt; 1, \'max_fds\' =&gt; 1048576, \'rr_weight\' =&gt; \'uniform\', \'queue_without_daemon\' =&gt; \'no\', \'flush_on_last_del\' =&gt; \'no\', \'log_checker_err\' =&gt; \'always\'}, \'devices\' =&gt; [{\'vendor\' =&gt; \'(LSI|NETAPP)\', \'product\' =&gt; \'INF-01-00\', \'product_blacklist\' =&gt; \'Universal Xport\', \'path_grouping_policy\' =&gt; \'group_by_prio\', \'path_selector\' =&gt; \'service-time 0\', \'path_checker\' =&gt; \'rdac\', \'features\' =&gt; \'2 pg_init_retries 50\', \'hardware_handler\' =&gt; \'1 rdac\', \'prio\' =&gt; \'rdac\', \'no_path_retry\' =&gt; 30, \'rr_min_io\' =&gt; 100, \'rr_min_io_rq\' =&gt; 1}], \'blacklist\' =&gt; {\'devnode\' =&gt; [\'^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*\', \'^hd[a-z]\']}, \'multipaths\' =&gt; [{\'alias\' =&gt; \'119_912_DDP05_03\', \'wwid\' =&gt; \'36d039ea00009466f0000010f5e30445e\'}, {\'alias\' =&gt; \'119_912_DDP05_04\', \'wwid\' =&gt; \'36d039ea00009466f000001105e304483\'}, {\'alias\' =&gt; \'119_912_DDP06_03\', \'wwid\' =&gt; \'36d039ea00009466f000001115e3044e6\'}, {\'alias\' =&gt; \'119_912_DDP06_04\', \'wwid\' =&gt; \'36d039ea00009466f000001125e30450c\'}, {\'alias\' =&gt; \'119_934_DDP03_01\', \'wwid\' =&gt; \'36d039ea000094291000000ef5e303fff\'}, {\'alias\' =&gt; \'119_934_DDP03_02\', \'wwid\' =&gt; \'36d039ea000094291000000f15e304019\'}, {\'alias\' =&gt; \'119_934_DDP04_01\', \'wwid\' =&gt; \'36d039ea000094291000000f25e304079\'}, {\'alias\' =&gt; \'119_934_DDP04_02\', \'wwid\' =&gt; \'36d039ea000094291000000f45e304093\'}, {\'alias\' =&gt; \'119_934_DDP05_01\', \'wwid\' =&gt; \'36d039ea000094291000000f55e3040f5\'}, {\'alias\' =&gt; \'119_934_DDP01_01\', \'wwid\' =&gt; \'36d039ea000094291000000e95e303f06\'}, {\'alias\' =&gt; \'119_934_DDP05_02\', \'wwid\' =&gt; \'36d039ea000094291000000f75e30410f\'}, {\'alias\' =&gt; \'119_934_DDP06_01\', \'wwid\' =&gt; \'36d039ea000094291000000f85e304172\'}, {\'alias\' =&gt; \'119_934_DDP06_02\', \'wwid\' =&gt; \'36d039ea000094291000000fa5e30418d\'}, {\'alias\' =&gt; \'119_912_DDP01_01\', \'wwid\' =&gt; \'36d039ea00009463d000000fa5e3041e5\'}, {\'alias\' =&gt; \'119_934_DDP01_02\', \'wwid\' =&gt; \'36d039ea000094291000000eb5e303f28\'}, {\'alias\' =&gt; \'119_912_DDP01_02\', \'wwid\' =&gt; \'36d039ea00009463d000000fc5e30420a\'}, {\'alias\' =&gt; \'119_912_DDP02_01\', \'wwid\' =&gt; \'36d039ea00009463d000000fd5e304271\'}, {\'alias\' =&gt; \'119_912_DDP02_02\', \'wwid\' =&gt; \'36d039ea00009463d000000ff5e30428d\'}, {\'alias\' =&gt; \'119_912_DDP03_01\', \'wwid\' =&gt; \'36d039ea00009463d000001005e3042f4\'}, {\'alias\' =&gt; \'119_912_DDP03_02\', \'wwid\' =&gt; \'36d039ea00009463d000001025e304311\'}, {\'alias\' =&gt; \'119_912_DDP04_01\', \'wwid\' =&gt; \'36d039ea00009463d000001035e304378\'}, {\'alias\' =&gt; \'119_912_DDP04_02\', \'wwid\' =&gt; \'36d039ea00009463d000001055e304396\'}, {\'alias\' =&gt; \'119_912_DDP05_01\', \'wwid\' =&gt; \'36d039ea00009463d000001065e3043ff\'}, {\'alias\' =&gt; \'119_912_DDP05_02\', \'wwid\' =&gt; \'36d039ea00009463d000001085e30441c\'}, {\'alias\' =&gt; \'119_912_DDP06_01\', \'wwid\' =&gt; \'36d039ea00009463d000001095e304486\'}, {\'alias\' =&gt; \'119_912_DDP06_02\', \'wwid\' =&gt; \'36d039ea00009463d0000010b5e3044a4\'}, {\'alias\' =&gt; \'119_934_DDP01_03\', \'wwid\' =&gt; \'36d039ea00009477d000001325e303f05\'}, {\'alias\' =&gt; \'119_934_DDP01_04\', \'wwid\' =&gt; \'36d039ea00009477d000001335e303f27\'}, {\'alias\' =&gt; \'119_934_DDP02_01\', \'wwid\' =&gt; \'36d039ea000094291000000ec5e303f86\'}, {\'alias\' =&gt; \'119_934_DDP02_03\', \'wwid\' =&gt; \'36d039ea00009477d000001345e303f7d\'}, {\'alias\' =&gt; \'119_934_DDP02_04\', \'wwid\' =&gt; \'36d039ea00009477d000001355e303f9f\'}, {\'alias\' =&gt; \'119_934_DDP03_03\', \'wwid\' =&gt; \'36d039ea00009477d000001365e303ff6\'}, {\'alias\' =&gt; \'119_934_DDP03_04\', \'wwid\' =&gt; \'36d039ea00009477d000001375e304019\'}, {\'alias\' =&gt; \'119_934_DDP02_02\', \'wwid\' =&gt; \'36d039ea000094291000000ee5e303f9f\'}, {\'alias\' =&gt; \'119_934_DDP04_03\', \'wwid\' =&gt; \'36d039ea00009477d000001385e304071\'}, {\'alias\' =&gt; \'119_934_DDP04_04\', \'wwid\' =&gt; \'36d039ea00009477d000001395e304094\'}, {\'alias\' =&gt; \'119_934_DDP05_03\', \'wwid\' =&gt; \'36d039ea00009477d0000013a5e3040ed\'}, {\'alias\' =&gt; \'119_934_DDP05_04\', \'wwid\' =&gt; \'36d039ea00009477d0000013b5e304111\'}, {\'alias\' =&gt; \'119_934_DDP06_03\', \'wwid\' =&gt; \'36d039ea00009477d0000013c5e30416c\'}, {\'alias\' =&gt; \'119_934_DDP06_04\', \'wwid\' =&gt; \'36d039ea00009477d0000013d5e304190\'}, {\'alias\' =&gt; \'119_912_DDP01_03\', \'wwid\' =&gt; \'36d039ea00009466f000001075e30424a\'}, {\'alias\' =&gt; \'119_912_DDP01_04\', \'wwid\' =&gt; \'36d039ea00009466f000001085e30426f\'}, {\'alias\' =&gt; \'119_912_DDP02_03\', \'wwid\' =&gt; \'36d039ea00009466f000001095e3042ce\'}, {\'alias\' =&gt; \'119_912_DDP02_04\', \'wwid\' =&gt; \'36d039ea00009466f0000010a5e3042f3\'}, {\'alias\' =&gt; \'119_912_DDP03_03\', \'wwid\' =&gt; \'36d039ea00009466f0000010b5e304351\'}, {\'alias\' =&gt; \'119_912_DDP03_04\', \'wwid\' =&gt; \'36d039ea00009466f0000010c5e304377\'}, {\'alias\' =&gt; \'119_912_DDP04_03\', \'wwid\' =&gt; \'36d039ea00009466f0000010d5e3043d7\'}, {\'alias\' =&gt; \'119_912_DDP04_04\', \'wwid\' =&gt; \'36d039ea00009466f0000010e5e3043fd\'}]}' (noop)```

Note here the multipathclass does literally nothing besides notifying the `$config` parameter. It does not manage the `multipath.conf` file.

So that points to the template as the source of the error

Ah interesting. And yeah, good point.
If this passed validation while compiling the class, then it’s not the data coming into the class that’s the problem.

The template iterates through the devices and multipaths deleting the mandatory keys after they've been printed:
```multipaths {
&lt;% @config['multipaths'].sort { |x, y| x['wwid'] &lt;=&gt; y['wwid'] }.each do |multipath| -%&gt;
  multipath {
    &lt;%-# Start each multipath subsection with the wwid. -%&gt;
    wwid &lt;%= multipath.delete('wwid') %&gt;
    &lt;%- multipath.keys.sort.each do |k| -%&gt;
    &lt;%= k %&gt; &lt;%= multipath[k] %&gt;
    &lt;%- end -%&gt;
  }

&lt;% end -%&gt;
}```

This is virtually identical to the version of the production environment.

Huh… that’s interesting. The `multipath.delete('wwid')` part of that template lines up with the “index 0 expects a value for key ‘wwid’” messages.

right, that's the only place the error could be raised from

I wouldn’t think this is possible, but I wonder if the ERB code is mutating the scope in a way brings those changes back up to the Puppet class.

I'll edit the template to show me what data it got, one moment...

One thing I recommend is migrating to an EPP template, rather than an ERB template.
EPP has much safer and saner scope handling, and wouldn’t suffer from this problem, if that’s actually what’s happening.

Yeah, I personally prefer EPP, too. Though I didn't want to invest that time right away.

OK, two things have been verified now:
1. Removing the `multipath.delete('wwid')` calls prevent the error
2. The data supplied to the template looks just as expected  
```devices = [{"vendor"=&gt;"(LSI|NETAPP)", "product"=&gt;"INF-01-00", "product_blacklist"=&gt;"Universal Xport", "path_grouping_policy"=&gt;"group_by_prio", "path_selector"=&gt;"service-time 0", "path_checker"=&gt;"rdac", "features"=&gt;"2 pg_init_retries 50", "hardware_handler"=&gt;"1 rdac", "prio"=&gt;"rdac", "no_path_retry"=&gt;30, "rr_min_io"=&gt;100, "rr_min_io_rq"=&gt;1}]```

If it really is the ERB template messing with the scope and mutating that data, this is an extremely interesting example of hostile ERB code :slightly_smiling_face:

Man, I leave to pick up kids and I come back to a programming lab

The template only aims to write the mandatory hash values first and then the rest, Using the `Ruby Hash.delete()` function seems just perfect for that

heh, sorry to blow up your thread cvquesty :slightly_smiling_face:

and in production it also still works. But there the legacy `validate_xyz()` of the stdlib module are still used.

So knowing this, I’d recommend in order:
1) Not using ERB. Use EPP instead.
2) If you have to use ERB, re-write the Ruby to not alter the original data structure. Either copy it to a new variable or use operations like `filter()` to generate new data structures.

Thank you again for your time, <@U112RM52S>. I have to leave now. I'll follow up on your advice tomorrow. You may want to formulate a proper answer on the GitHub discussion. Or I'll do that when I have the time for it later. CU :wave:

:coffee: _:teacher:Puppet Core Team_ is about to start up in <#CFD8Z9A4T|>

:wave::skin-tone-4: Hi <@U112RM52S>!  You mean during the `apply`, don't resync from the local modulepath?

I’m talking about a scenario where you don’t have the stdlib module locally, but you want to use one of its functions for a `puppet apply`.
And that function was previously pluginsynced to `/opt/puppetlabs/puppet/cache/lib/puppet/`

The easy answer is to just sync the module down locally and use `--modulepath &lt;path_to_local_modules&gt;`, but I’m wondering if it’s possible to *not* have the module locally and to just rely on the pluginsynced plugins.

Can you turn pluginsync off temporarily so it'll just use what's already there?

```$ sudo puppet apply -e "notice(fact('os.release.major'))" --no-pluginsync
Warning: Setting 'pluginsync' is deprecated.
   (location: /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/defaults.rb:1995:in `block in initialize_default_settings!')
Error: Evaluation Error: Unknown function: 'fact'. (line: 1, column: 8) on node nate```

You need to set the libdir to the plugin sync destination iirc