https://www.puppet.com/community logo
Powered by
Title
b

binford2k

05/25/2023, 6:52 PM
relevant to this conversation in that the per-module hiera datadir can be anything you want it to be. When we could have just defined it as 
<module>/data
.
c

csharpsteen

05/25/2023, 6:53 PM
There's actually an important bit of PE configuration behavior that depends on the fact we didn't do that 😒miling-face-but-you-can-see-pain-in-their-eyes:
:lol: 1
The 
puppet_enterpise
module defines its datadir to be 
/etc/puppetlabs/enterprise/conf.d/
so that re-configuration after restore of a backup, or during a classifier outage will mostly Do The Right Thing™ instead of shifting the entire PE install to out-of-box-defaults which is, if nothing else, a great way to slam a performance issue on top of whatever problem you were trying to solve :lolsob:
b

bastelfreak

05/25/2023, 6:56 PM
lol
c

csharpsteen

05/25/2023, 6:57 PM
But, pointing to an absolute path is a lesser evil compared to relative paths or symlinks that lead to hoisting yourself on a petard of your own cleverness.
v

vchepkov

05/25/2023, 6:59 PM
I was always wanted to ask, why installer doesn't honor user's hiera, maybe I miss some step? I observe installer revert custom setting and then puppet agent puts them back
I understand that maybe hierarchy is a challenge, but I keep pe settings in common.yaml
b

binford2k

05/25/2023, 7:00 PM
@csharpsteen tangent: did you know that "petard" is a small bomb and the word etymology is latin for "to fart"?
:nodding: 1
c

csharpsteen

05/25/2023, 7:04 PM
The installer did have a tap into 
common.yaml
, but I think that has become less useful recently as 
puppet infra configure
sandboxes its 
environmentpath
to 
/opt/puppetlabs/server/data/environments/
and uses the 
enterprise
environment --- as a firebreak between sensitive operations like upgrade and the infinite combinations of user module behavior that acceptance tests can't guard against. In general, the usefulness of user Hiera is debatable since the directories don't exist before installation and are some form of circular dependency after.
v

vchepkov

05/25/2023, 7:13 PM
Is there a way to plug it back in?
or we have to deploy pe.conf instead somehow?
unless it's also ignored?
some services are not properly restarted when they are 'reverted' for example - 
puppet_enterprise::profile::plan_executor::versioned_deploys
b

bastelfreak

05/25/2023, 7:16 PM
you can provide your own pe.conf for the pe-installer/peadm
v

vchepkov

05/25/2023, 7:17 PM
If this is the way , sure:mandalorian:
c

csharpsteen

05/25/2023, 7:18 PM
The hierarchy used by 
puppet infra configure
is defined in 
/etc/puppetlabs/enterprise/hiera.yaml
, but I wouldn't recommend modifying it as one of two things is likely to happen depending on whether it is marked as a config file in the packages: • The file gets overwritten on upgrade and unexpected things happen because customized data sources are no longer pulled in. • The file does not get overwritten on upgrade and unexpected things happen because updated defaults are no longer pulled in.
b

bastelfreak

05/25/2023, 7:19 PM
and if you provide a pe.conf the data will be added to the classifier and you won't be able to override it in hiera. so I usually provide a minimal config that just disables metrics/update check
💯 1
then configure code manager, do code deploy, configure the rest via data in hiera
v

vchepkov

05/25/2023, 7:20 PM
enterprise/hiera.yaml does reference common.yaml. but probably it gets ignored because environment is not what normal is
➕ 1
- name: "Common data from Hiera3 user hieradata"
    datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"
    data_hash: yaml_data
    path: "common.yaml"

  - name: "Common data from Hiera5 user data"
    datadir: "/etc/puppetlabs/code/environments/%{environment}/data"
    data_hash: yaml_data
    path: "common.yaml"
b

bastelfreak

05/25/2023, 7:21 PM
do you deploy your code before running the installer?
v

vchepkov

05/25/2023, 7:23 PM
not sure what do you mean? installer is also executed during upgrades, so code is already there
b

bastelfreak

05/25/2023, 7:23 PM
ahh you're talking about upgrades
v

vchepkov

05/25/2023, 7:23 PM
yep
b

bastelfreak

05/25/2023, 7:23 PM
mhm
before you said that I thought our upgrades honoured our hiera data, but now I'm not so sure anymore
v

vchepkov

05/25/2023, 7:25 PM
nope, they don't
c

csharpsteen

05/25/2023, 7:30 PM
sudo crontab -l
should show 
puppet infra recover_configuration
running on a schedule. That tool looks at PuppetDB and tries to determine if any 
puppet_enterprise
parameters are being set via Classifier or user Hiera data. It then writes those values back to 
/etc/puppetlabs/enterprise/conf.d/user_data.conf
and 
enterprise/conf.d/nodes/*.conf
in the case of settings that appear to be different for compilers vs. primary vs. replica vs. database.
puppet infra configure
also runs that same configuration recovery to sync 
enterprise/conf.d/
up before making changes. The 
--no-recover
flag is commonly used to skip this step in situations where PuppetDB is not responding.
v

vchepkov

05/25/2023, 7:33 PM
only puppet_enterprise ?
c

csharpsteen

05/25/2023, 7:40 PM
puppet_enterprise
is the majority consumer of that data. The 
pe_infrastructure
and 
pe_repo
modules also use it.
:gratitude-thank-you: 1
Looks like those other two may specifically only care about 
pe.conf
. 
pe_infrastructure
is mostly active during first install, not sure why 
pe_repo
ignores the other data files --- likely just an oversight.
v

vchepkov

05/25/2023, 7:56 PM
I observed quite a few changing back and forward - for example 
--- /etc/puppetlabs/nginx/conf.d/proxy.conf	2023-02-16 09:48:18.801727023 -0500
+++ /etc/puppetlabs/nginx/conf.d/proxy.conf.augnew	2023-05-04 19:10:35.327820741 -0400
@@ -17,7 +17,7 @@
 ssl_session_cache shared:SSL:50m;
 location /
 {
-proxy_pass <http://127.0.0.1:4430>;
+proxy_pass <http://localhost:4430>;
I think the process misses some hiera that one is driven by puppet_enterprise::plaintext_address, for example and it's not extracted to conf.d
d

Dr Bunsen Honeydew

05/25/2023, 7:56 PM
```- lost a level. (Karma: -109)
v

vchepkov

05/25/2023, 7:57 PM
This goes back and forward 
--- /etc/puppetlabs/r10k/r10k.yaml      2023-02-04 09:46:53.577921459 -0500
+++ /tmp/puppet-file20230504-3697032-knu4j4     2023-05-04 19:08:34.304876302 -0400
@@ -1,8 +1,8 @@
 ---
-cachedir: /opt/puppetlabs/server/data/puppetserver/r10k
+cachedir: /opt/puppetlabs/puppet/cache/r10k
 sources:
   puppet:
-    basedir: /etc/puppetlabs/code/environments
+    basedir: /opt/puppetlabs/server/data/environments
d

Dr Bunsen Honeydew

05/25/2023, 7:57 PM
```- lost a level. (Karma: -110)
b

bastelfreak

05/25/2023, 7:58 PM
mhm interesting
v

vchepkov

05/25/2023, 7:58 PM
I just reboot server now after upgrade is done 🙂 ¯\_(ツ)_/¯
b

bastelfreak

05/25/2023, 7:58 PM
😄
#office-hoursJoin Slack
Powered by