When you have a large fleet of hosts with the same role, but you need wide variance between instances, what would be a better hieradata backend than a git repo with yaml files?

which issues do you have with git or with yaml?

the manually editing part of it 🙂

cant you group those nodes?

Or you may use another facts to split large fleet by groups.. we were using DC, country and env (qa/prod) to do smaller changes

In cases like these we're often done big array of machines

$is_105 = [
  'fqdn1',
  'fqdn2',
]

The other option is to pull version out and assume that the upgrade will be handled outside of Puppet.

profile::maria::server::package: 'installed'

This works fairly well as long as the upgrade window is small, couple of weeks at most. New servers get latest version, current servers don't upgrade, and Puppet doesn't fight with external upgrade process.

I'll clarify what we're all sort of getting at: This isn't a Hiera backend problem. This is a process problem. It sounds like you need more ways to group nodes into smaller chunks (assuming you want to do the upgrade with Puppet rather than outside of Puppet like what ramindk mentioned). Since the servers all have the same role, you don't want to use that info since it's too broad. And grouping by FQDN is tedious because it's too fined grained. As Yury mentioned, a popular middle ground is grouping by geography, tenant, tier, and/or specialization. Normally that'd be done with facts or some bits of data set by an external node classifier (ENC). And you'd add a layer to your hierarchy that uses that middle-ground data to progress through the upgrade.

Nate always following up with these banger summations. :magic:

Thanks, all of that helps. Let me elaborate on why it only kind of helps 🙂 We use the mariadb_version to determine what to put into the apt_source for mariadb. So we can't just go with 'installed' although that would have been very nice, because, like you said: new machines would get the new version from the start. grouping by geolocation or domain has worked for us up until a point, but our largest datacentre has 50 percent of our fleet. I wonder if we can go with a static custom fact where we give servers an upgrade batch number between say, 1 and 10, generated by fqdn_rand. roughly 10 percent of the fleet would be in each batch number, and we use that to group. ie, this week upggrade_batch_0 gets the upgrade, next week upgrade_batch_1, etc

I’d go with batch number fact, why not

thanks for the oppurtunity to bounce some ideas, guys