Windows Server 2022 VDA 2311 // Images build by MDT for MCS I have one environment where "suddenly" my MDT created Master Images stopped working. When I start my MDT build, it finishes without (visible) issues as usual. The MCS machine catalog update runs also without problems. But when the clones power up, they never register, as all the important services refuse to start: Server, Workstation, LanmanWorkstation, Netlogon, Citrix Pvs for VMs Agent. I suspect the PvsVmAgeht to be the root cause, but not sure. I have no idea what change could have triggered this, I just altered a software to use a different config file and started a new build. The VDA setup script itself was not altered in weeks. The eventlog is not helpful at all, as it just shows an Error for the service and states that there is no error message for that error. So I have no hint into why it refuses to start suddenly. Anybody an idea, why the services in my clones stopped working? (Screenshot from a working machine)

i like to isolate by just building the image with only the required software...so in your case maybe that's just the OS and Citrix. If it works, you know it's something else.

Thought the same while writing my help request. But it was already 9pm. It will be the first thing I try tomorrow morning.

it's not a trust relationship problem is it? your able to RDP to it using domain creds?

It is. As the PVSVMAGENT and the Netlogon service don’t start, I’m not able to login with any domain user. If I roll the image back to the previous image, the error is gone.

I see a few articles around this. Not sure what might have changed to cause this, but you could try https://support.citrix.com/article/CTX249833/pvsvmagent-service-on-vda-not-updating-the-machine-account-password-locally

I know that article. But as far as I can tell, this dependency is set by default in VDA 2311, so that can't be the reason.

no citrix infra updates at all (site or vda) or hypervisor updates?

No, but i gets more complicated

what about windows updates in the image?

Let me add one fact

and the master is rebuilt every time from scratch? aka WIM gets laid down and then the apps/domain join/etc..

Yes, via MDT from an imported vanilla ISO.

almost sounds like the provscheme got edited some how

Exactly

what if you add a VM to the MC, that one fails too?

On the current snapshot that works or on the future snapshot that I would like to deploy?

current

I can tell you in 5 minutes

wonder if any of these would turn up anything.

That added machine on the current prod catalog with the current snapshot works

could you add the new snapshot to that machine?

jup, just takes a bit longer, as it's a catalog update

yeah

Catalog upgrade is working now

i'm sure you want root cause, but how hard would it be to use a new MC and recreate the machines

After that I can force reboot the added machine

annoying, but at least you ahve a fallback plan

The problem is not the rebuilding, the problem is, what did I do wrong in the ProvSchema(?) to make that happen. I don't want that again in 4 months.

yeah, could always create new MC and new machines and keep the existing stuff there and open up aticket wiht support, that way you can work on it as needed with your schedule

well F

I now do a catalog update to the "old" snapshot" instead of a rollback.

just dont know why new machines would work on old snaps, it uses the same prov scheme, just different snap location

In which Eventviewer? On the VDA there are loads of errors, that all the services don't start. I didn't see anything else useful. But that doesn't mean there isn't. Haven't tried Get-ProvSchemeWarning

lol

Nothing

sigh

It makes no sense

agreed

All snapshots later than 22 March won't start It doesn't matter on a newly created catalog. I don't grab that pattern

wouldn't be a hypervisor permission thing on new snaps would it?

The catalog update works without (visible) issues.

haha, for sure

no, CSP base license with 2 on-prem controllers

well...poop. Could see DaaS stuff changing things and how they work, but if there wasn't any on nprem changes , then i just dont know

fml it works with the old snapshot from february

wth happened around march 22nd

MDT creates me a new machine everytime, every machine has only one snapshot, so I can chose between many machine with one snapshot each No machine ever gets any update. Only new MDT builds.

ah, nice

Well I guess deleting and re-creating it is.

and you tried a new snap on the older machines?

I don't understand. (sorry it's getting quite late)

the MDT VMs built prior to march 22nd, take a new snap on one of those and try to deploy

Well, if I don't boot them up, than a snapshot based on a snapshot would essentially be the same?

yeah

The ISO was imported in November and hasn't changed since

curious, what's the hypervisor?

VMware VSAN 7

i'm pretty much out of ideas on RC, hopefully someone else will chime in, but at least you have a fall bakc plan for now 😞

I have another idea.

Interesting, I did see that, but didn't think that would do anything, especially if a new machine works.

Does the Default Domain Policy Password Policy affect local user accounts?

Nice!! Betting its that as well, especially since it changed.

Re-created the Machine Catalog with MCSIO Drive Letter E: and ResetPassword False last night, and can use every snapshot in it, regardless of Minimum Password Age 1. Issue is therefore solved. @Steve Noel Wish I could send you some positive Karma.

atta boy, way to stick with it!