Puppet Community

from the docs, it seems like if you're using default SRV discovery, you don't need to define anything for `plugin.choria.middleware_hosts` in server config. as long as SRV records exist for broker(s), it should work, right?

probably :slightly_smiling_face: i think so - though SRV records have a bit of a flaw in that they never update till restart, so if you expect to use them for their dynamic nature rather than their just ease of config it might not be too hot

that seems like a bug :slightly_smiling_face:

one I am happy to hear about because I would be almost sure to use SRV when I actually do choria (because my puppet uses srv_domain already)

for things like access to puppetdb and such it updates

but for the nats connection - this is a short coming in the nats client libraries - it doesnt update

and I've tried for years to get people to prioritise this but seems we're one of the v.few/only ones using that

ah, so all the choria-servers need to restart, which they should do via puppet when we remove that middleware line from the config

but what i am getting at is also - even if you update the SRV records, it wont re-resolve them till next restart

okay, so a safer approach may be to use a comma separated list of brokers?

depens on the general stability of your setup yeah

for many that just wont be an issue cos shit never changes

minimal config/SRV discovery is fine by me, but if it's better to define the brokers, also no big deal

does it at least load all of the levels of the SRV record, so if I setup failover to other datacenter systems I can at least restart services to update records?

i think conceptually choria agent falling over to other DCs is probably a mistake

not impossible but you really want to keep things local if possible and multi dc clusters have issues unless they are network wise quite near

oh, also, if you have multiple brokers, but they're not federated, the responses will be different according to the broker, correct?

we've been operating with just a single broker, but i'm thinking we may need to federate to add a broker or two.

if you just want an in-dc cluster of 3 no federation, they peer and form a cluster and have full HA

well, at the moment, it's one broker in a vm on-prem, and one in ec2. we'll most likely end up with 1 or maybe a few in ec2 and none on-prem.

whats the latency between the 2 locatiojns?

haven't checked, but it's not bad. connectivity is via direct connect to the vpc.

ok, yeah 3 brokers one per AZ should be fine

agent-&gt;broker can have latency, broker-&gt;broker for clustering should be low

in theory broker-broker can be like 100ms, but i am not a fan of that