This message was deleted.
# _general
s
This message was deleted.
s
What is the use case...
r
That would be a no.
✔️ 5
l
So, our org uses rehydration with a new AMI every 90 days. I was also told that all large tech organizations re-do their full infra quite frequently and it just seemed drastic to me.
I know we could build DDC 1, 2 and 3, then add 4,5 and 6 and remove 1,2 and 3. I guess for SQL you could build an AOAG with 3, then add 3, then remove 3, etc ongoing, but it just seems like a lot more operational headache than patching monthly
k
Infra as a code.. paging @Dave Brett
#chaosmonkey FTW
l
@Kees Baggerman I've heard of Chaos Monkey - did you intend it to be Chaos MONEY? LOL
🙌 1
k
😇 Slack edit FTW
🙌 1
l
that's what I figured 😛
n
i worked at a $15B public corp, fortune 500 and we never did anything remotely like this
d
We rebuild almost all of it every quarter and patch inbetween. Mainly to validate the process and ensure we don't miss anything 👍🏻
👍 1
w
My lab is rebuilt regularly. I don't know anyone who does that in a prod env.
✔️ 1
h
We essentially do that every year but that is because of HW additions and removals and the way we have fault zones setup. But I wouldn't say on a regular basis for infrastructure.
w
There's quite a few companies that have this as their ultimate goal. Most fail t it though.
s
We rebuild all of our VDA machines as code for each update we need to push (patches, 0-day, new version, new app). But Infra is Citrix Cloud so..
j
Yep, everything every month
c
Only heard of companies rebuilding Citrix to run off fresh free licenses each time so they didn't have to pay.
l
@James Rankin how big of an environment?
j
80,000 users
j
I know @RSRevord @Hal Lange do this.
l
@James Rankin assuming this is fully cloud based - is it using Citrix cloud or AVD? If using onprem citrix, you rebuild the storefront and sql every month as well? Do you simply do a backup of the old site, then restore to the new each month? We have 50K nonpersistent VDAs and it seems like a recipe for issues rebuilding every month for the sake of rebuilding
j
We are in cloud so just have to rebuild cloud connectors. It's not a recipe for issues, it's absolutely awesome. The users never even notice things have changed.
l
OK, makes more sense. I'd gladly redo a cloud connector every month rather than the entire database. What about Storefront? Are using workspace service or onprem storefront?
j
No on-premises infra at all
r
we rebuild but we don't' nuke SQL just keeping the OS fresh, about every 3 years we'll start over completely but thats due to organic growth and changing needs more than a need or anything like that.
l
So for SQL do you just do a backup of running env, and restore to a new server? I was thinking one way would be to use AOAG and have 3 nodes, and cycle new servers in and old out every month - which would keep the site pointed to the same listener
r
we've done both, usually we end up building a new AAG member adding it, and retiring old
we started because we've noticed storefronts tend to just get stupid after abit
and we can rebuild faster than troubleshooting most issues
we also try and run core anywhere we can get away with it. (giving stinkeye to storefront and a few others)
r
Man, We just run that shit into the ground. I guess this is the difference between Enterprise and MSP. We have like 100 environments.
l
For storefront rebuild, do you do backup/restore or do you have a script that builds out every store as you want it?
r
usually backup/restore
problems with automation is unless you push the detlta operational changes via automation those settings get lost.
ideal state all things IaaC automated for changes, but least in our culture its not feasible today.
👍 1