World of EUC

Not knowing your setup and giving a roundabout answer based on what I see in the space.
Typically this is something with GPO, such as login scripts, drive mapping, Item-level targeting. Something in GPO is my first sign. Not sure how it compared to other VDA and another version of windows. But I am sure you have an understanding of what is was compared to now.

^^ I'd definitely start at GPO, specifically item level targeting/laborious WMI queries holding up the logon.

So yea I should clarify. This is Azure VDI's. multiple machines, various GPOs and OUs. Over the summer the images have been updated to 21H2 and cu5 and other updates

yes looks like GPO. Just moved desktop to OU with blocked inhert and login is instantaneous. There are alot of GPO's to look through :smile:

I feel your pain on that. Where I am at, they went nuts with GPP. They didn't know any better, though, even though they had AppSense LOL

well maybe not GPO's after all. I was testing on writable machine and once the profile got created it stopped using FSLogix. So its something with profile after all :disappointed:

hmmm when did this happen?  Can you test with an older vesion of FSLogix?

Well thats the thing. FLogix was older version this entire time. I just updated it now to see if it will fix it. It did not. I think the issue started at some point during summer and because its slower time no one reported until recently. And in the last few months a few things changed including CU5, Windows patches, etc. But I dont know how this is tied to profile.... :thinking_face:

Following up on this. Control up indicated several scheduled tasks that run on logon - Google Updates, Edge, Office telemetry. Disabling them brought the logon speed back to more or less normal.

Chrome/Edge went to 104 in the last week IIRC.

image.png

There was a registry key for this in past 

I think it was native drive mapping or something 

VDA side since you are modifying the Citrix Workspace App behavior from within your Citrix images.

NativeDriveMapping

<https://www.rorymon.com/blog/mappednetworkdriveissue/>

Ok. I’ll give that a shot. Just to be clear, I don’t want the client drives from the user device to show up in the app session, nor the VDi session. I do want the client drives from the VDi to show up in the app session. 

yep so you change this reg key on your existing Citrix Servers where the VDA is installed and Workspace App, not your actual endpoint device

if you have cache in ram  I think that is harder to tell but you can use some of the Sys internals tools

cache in ram with overflow. and overflow file is filling up cache drive

i've used the nirsoft tool 'folderchangesview' before and it worked well.

Nirsoft truly does have a utility for everything.

Never used it before, but that FileActivityWatch utility may also work well for this case.

FWIW, last time I was bit by this it was <https://support.citrix.com/article/CTX338425> NTFSDisableLastAccessUpdate due to outdated Citrix Optimizer in my seal process

interesting, wish they would have included specific versions

Interesting. I swear that was an old requirement in App Layering.

looks like NTFSDisableLastAccessUpdate is set correctly. :disappointed:

but a bunch of other things are not optimized

yea just checked another machine. 30GB vdiskdif :man-facepalming:

and no changes to C drive either. same size as at boot

cm/branch cache?  That has bitten me when cm clients were updated before

yea checked that too. it was enabled once before but now its disabled.

and yea its totally inconsistent I have been sitting look at the server and zero change to cache. I am guessing this is happening in spurts

AppV? Print Spooler? Tanium?  the folderchangesview program should be able to pin point, especially if it blows up quickly

the slow spurts are harder to see, however you can log the changes in excel format to the write-cache drive or somewhere else

you pointing the shared content store to your writech cache drive?

how did you disable branch cache?  PS:\Disable-BC?  Regedit, service disable?  if so, know that when CM client checks in it may reenable it.  I can check with my CM guy, but I think you need to configure BC in CMC for it to be disabled

I asked the sccm guys to disable it when I saw it was on earlier this year.

<@ULJMUV893> folderchangeview saves the day. its

ha :slightly_smiling_face: they are not. It uses defender so I think killing updates would stop defender upd as well

ah, hmmm, well at least we are on the hunt

also question is why are the updates 10s of GB?

i can see cache filling up in front of my eyes :smile:

is there a 'delta' checkbox for how you receive updates?

<https://docs.microsoft.com/en-us/microsoft-365/security/defender-endpoint/manage-protection-updates-microsoft-defender-antivirus?view=o365-worldwide>

probably not. I think I see what is going on. its failing on a rollup update and looks like it keeps retrying

and there are a bunch of other updates too that are included. Visual Studio even

ok well mystery solved. Now just need to figure out the updates. Thanks again <@ULJMUV893>. Your help was HUGE.

think i remember <@ULC9J5XPB> or <@ULLH5JGFR> talking about Defender best practices for non persistence, kind of a bunch of documents cobbled together if i remember correctly.

yea I had that in the back of my mind as well. but never got around to it since defender by itself never consumed that much space

I downloaded "folderchangesview" about 2 months ago, and security was all over me like stink on poop. it flagged something in their scanners. But yea they have some great tools.

nice!  that's a "walk off win" on a friday!

Yeah, I would definitely change your Defender update settings so that Windows Update and Microsoft update are last in the list. If you're looking for a way to simply prevent windows update from working entirely in a non-persistent image, let me know. I changed methods recently and it has been flawless for me.

<@ULC9J5XPB> definitely interested in redesigning Defender updates in non persistent settings. Let me know how I can obtain some of that knowledge :slightly_smiling_face:

This thread has all of the details, I think: <https://worldofeuc.slack.com/archives/CLSHNTWLW/p1653987969746889|https://worldofeuc.slack.com/archives/CLSHNTWLW/p1653987969746889>

And before Kasper says it, yes, I feel bad about not just writing a blog about it.

If it can wait until Tuesday, I can tell you what I did for Windows Update blocking. Not in the office until then.

I dont see the thread. Probably because Unlock messages prior to June 6th in World of EUC
To view and search all the messages in your workspace’s history, rather than just the 10,000 most recent, upgrade to one of our paid plans.

No worries, I think I have it written up in a Word doc, too. I can grab that on Tuesday.

Gonna write one blog, on Defender, and dedicate it to Kasper.

Hahaha. But it would get a lot of traction for sure. 

Maybe I'll take the plunge and see if I can write one for <http://mycugc.com|mycugc.com>

From your Citrix Delivery Controller, run this script as a scheduled task, it will download the latest defender updates and create a share. Then modify your Citrix VDA GPO to point to that share for Defender updates.

<https://github.com/JonathanPitre/Scripts/tree/master/Get-MicrosoftDefenderUpdates|https://github.com/JonathanPitre/Scripts/tree/master/Get-MicrosoftDefenderUpdates>

*Windows Update blocking:* Aside from disabling the Windows Update service via GPO, I also set the following items:

Basically, you're telling Windows Update to only use loopback for updates. This will prevent Defender updates from coming down, as well, which is why you also need to set the _Define the order of sources..._ setting to both your local fileshare where the updates are downloaded to as well as MMPC, which is Microsoft's dedicated hosting site for Defender updates. It's usually a last resort option, as the downloads are larger in size, but it works fine for this method.

Also worth noting that this setting *must* be baked into your image, as it requires a reboot to take effect. I have it enabled via GPO just for enforcement, but I do bake it in with my sealing script:

```Write-Host "Updating Defender..." -ForegroundColor Yellow -NoNewLine
Start-Process "C:\Program Files\Windows Defender\MpCmdRun.exe" -ArgumentList "-RemoveDefinitions -DynamicSignatures" -Wait -PassThru | Out-Null
Start-Process "C:\Program Files\Windows Defender\MpCmdRun.exe" -ArgumentList "-SignatureUpdate" -Wait -PassThru | Out-Null
Write-Host " Done" -ForegroundColor Green

Write-Host "Scanning the image with Defender..." -ForegroundColor Yellow -NoNewLine
Start-MpScan -ScanType FullScan
Write-Host " Done" -ForegroundColor Green

Write-Host "Configuring Defender..." -ForegroundColor Yellow -NoNewLine
Set-MpPreference -SharedSignaturesPath \\<http://domain.com|domain.com>\Citrix\WDAV\wdav-update
Set-MpPreference -SignatureDefinitionUpdateFileSharesSources \\<http://domain.com|domain.com>\Citrix\WDAV\wdav-update
Set-MpPreference -SignatureDisableUpdateOnStartupWithoutEngine $False
Set-MpPreference -SignatureFallbackOrder 'FileShares|MMPC'
Write-Host " Done" -ForegroundColor Green```

to circle back to the original issue. The KB that kept trying to install over and over somehow corrupted WinSxS. I spent a long time running various dism restores before giving up and copying WinSxS from another image. There are others who are seeing similar problem with the update on reddit and microsoft support

is it only blocked in a session?  Multiple users?  Does it work on local device?

Not related to Citrix or multi user OS. The same behavior on laptop :see_no_evil:

In general it’s only issue with „Ctrl+0” in web browsers (Edge / Chrome). I did change to „Ctrl+9” and it still not working :disappointed:

Haven't tested it but I remember the runone /alternate shell key did that many years ago. I also remember enabling seamless flags to stop showing server side icon/notifications in the clients systemTray. This was many many years ago. May not be related at all, but something to thing about.

Yeah, I seem to recall that seamless flag as well. I think they're all seamless by default now?

Well the published apps are referenced by seamless apps. But Being they have the CAX tool that would tell all seamless flags are not enabled as it per application. Again been a long time since I had to dive Into this. Not trying to steer you wrong.

Correct. Just have to update the strings and make sure the mirror isn't a part of the new string

<https://support.citrix.com/article/CTX280675/migrate-citrix-databases-to-new-database-sql-server-cluster-always-on-or-mirroring|Migrate Citrix databases to new database SQL server (cluster, Always On or Mirroring)>

<https://support.citrix.com/article/CTX318273/how-to-migrate-citrix-database-on-sql-express-to-full-microsoft-sql-server|How to Migrate Citrix Database on SQL Express to Full Microsoft SQL Server>

Yep only null the connection and then new connection to the new sql server per PowerShell. All after export and import the db :sweat_smile:

done :white_check_mark: only difference to other DB migrations is to shut down first the secondary SQL Server :grinning:

not seeing that big performance improvement if storage is SSD or NVME

read I/O is higher, but should be handled easily. just assign enough RAM

I'm just leveraging it as an easy way to have a temp drive. 

great replacement for pvs, same driver is used in the background

depends on the hosting infra -&gt; don't use it on Nutanix

true MCS IO on Nutanix is not exactly supported out of the box, best to avoid it.

Hey Steve, its Global Protect (Palo Alto), cheers

just curious, been hearing some similar complaints with zscaler VPN, i know customers have tickets open with Citrix on the matter.  I'll touch base with them to see if there was any progress.

why don't they do a split tunnel for Citrix traffic?

If they are not on the LAN, the VPN is needed to hit the SF LB. They likely have split tunnel set up but it will be for sending non lan/internet trafic out of the local GW vrs the VPN GW, cheers

the customer that has a ticket open said that citrix has recognized a dev problem in the code.  They were given a private hotfix that solved some of these problems, but there will be more changes.  Don't have any other specifics or support #s at the moment, and quite frankly, i'm not 100% its related to your issue, but it does sound like the right symptoms, where a VPN and Citrix Workspace resets are involved.

Thanks a lot, Steve. I'll get the customer to open a call with Citrix anyway. Worth ruling out at least, cheers

I'm interested about what you could find regarding this issue. It is an issue our users occasionally face, it’s somehow mitigated with the VPN client launching a refresh after the connection is established.

FYI - Looks like Microsoft has finally decided to support M365 on Server 2022.

<https://twitter.com/cstalhood/status/1567512242593177601?s=20&amp;t=2XN4B0XJ-6aA1ot7CPl3jg>

We opened a ticket with Citrix on that.  Still waiting for reaponse

ugh thats frustrating..guess i will have to roll it back to 2203 CU1...how do they release these broken things constantly

No idea.  The worst part is it is so hit and miss if it applies or not... So hard to prove to them as well

we have this issue with Single Session VDA 2203 on our Win10 vm's, Citrix case opened and still investigating

the only thing I cannot get to work for one customer is timezone redirection and it fails on the 6th logon to the server

2206 should be put in the bin. 2203.1 is what I am deploying everywhere - 2206 is a bouquet of shite 

I have that issue, just had it 1 minute ago indeed. Logged back in and now the Citrix GPO is applied and I have changed absolutely nothing. Something is definitely wrong.

Few questions:
• Does it apply any policy when you see the issue? Or is it just not applying the Citrix settings?
• Do you see errors in the event viewer?
• Do you see the same issue with a VM that's been booted for a while? 

when the issue happens, zero Citrix Web Studio policy is applied. I don't see any particular error in event viewer.
Not sure about the vm uptime, something I need to test more thoroughly.

That's definitely weird.. Would be good to check VM Uptime

Our VMs were rebooted on sun/monday so less than a few days uptime

Another weird thing is that even when the Citrix GPO is applied sometimes I don't see it as applied in Citrix Monitor (Director)

Do you have any security agents or proxy running?

For us Citrix studio user policies are not applying on server 2016. Everything is working after a reboot and suddenly for some unknown reason it stopped for newly logged in user and we noticed no rollback.gpf files are created under c:\programdata\Citrix\group policy\&lt;session_ID&gt;

also the user registry hive under user session ID is empty.

<https://support.citrix.com/article/CTX472865/cvad-version-2206-2209-gpss-fail-to-apply>

Oldie - some new stuff out there now also but a good start <https://jkindon.com/designing-profile-management-with-active-active-resource-locations/|https://jkindon.com/designing-profile-management-with-active-active-resource-locations/>

Upm has some neat new features that can help <https://jkindon.com/the-evolution-of-citrix-profile-management/|https://jkindon.com/the-evolution-of-citrix-profile-management/>

Are the users disconnected if you disable BCR entirely?

Found the solution <https://support.citrix.com/article/CTX322696/workspace-app-crash-or-session-disconnect-when-user-logs-on-to-the-vda-with-html5-video-redirection-policy-enabled-and-visits-specific-websites|over at Citrix>. Turns out you can't have HTML5 redirection and BCR active at the same time... Turned that off and now the website opens without users getting disconnected from their sessions.

Isnt there a setting in Storefront to terminate the app if it's closed workspace? I can't recall and will have to look myself.

there are some workspace control settings that allow you to automatically reconnect to sessions at login, but in our case the session is already in an active state so it will not reconnect. Also if you add the disconnect button in storefront it should let you disconnect the session but it doesn't appear to show up on workspace for android

So your issue is instead of the session going disconnected, it stays active. Then not allowing them to reconnect based on restriction.

Sorry man, I have not seen this before. Not much help I know.

I find it interesting you don't allow reconnection to Active sessions.  Our users like this to allow them to move to conference rooms, and such, without even thinking about it.  They just lock the PC, and go home, and reconnect from there, etc

yeah this client shares logins. they are generic accounts. so if you allow reconnections, they will steal each other all day long

so ccasau123 is the account, and that account may be used by multiple workstations

and they may not know that another user is using that account

thanks <@ULLH5JGFR> that is the setting I have configured, but in the workspace app for android, it doesnt display and doesnt function. only the receiver for web site

The only time I see it is when the GPO is not applied for the RDS Licensing server in the RDSH/XA area here " HKLM\SOftware\Polices\Microsoft\Windows NT\Terminal Services

And it is for my latest case. I have a theory that once the grace period expires legitimately, something else is set that causes RDS to assume it is unlicensed rather than look at that registry key.

I have also seen a properly licensed XenApp server pop up a message about it not being licensed for the local Administrator account, but not domain users or administrators.

My "permanent" fix is to make deleting the TimeBomb reg value a step in the image update process.

Use bis-f and this killed the timebomb reg key :sweat_smile:

Oh no! :fearful: Being part of BIS-F, that tells me this is common and deleting the key is the best remedy.

BIS-F deletes the key? Or you saying add it to do that?

one step of bis-f is killing this reg key. I can send u a screenshot with this Part later ray :sweat_smile:

Thanks man. I never acutally knew it did that. Or I never paid attention

Well, I learned something new today. Guess it's time to log out ha

This has been a plague for us too. Happened on 2012R2 and 2016 and I'm going to assume 2019 (although we haven't got any VDA's in production yet). We had to delete the key as part of the PVS reseal process and with App Layering we have to do the same on the Platform before it runs out. We've opened tickets with Microsoft and their suggestion was to create a cluster for our RDS license servers. This doesn't address the issue. I'd love to hear someone figured this out. Deleting the key within 120 days seems to be the only fix.

For those of us with this problem, does your image have this reg value set to 1? [HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server] Name=fDenyTSConnectionsBackup REG_DWORD 1

I have 4 images and my problem child has this set to 1. My other 3 have 0. My google-fu finds nothing about this value.

<https://docs.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-terminalservices-localsessionmanager-fdenytsconnections|https://docs.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/microsoft-windows-terminalservices-localsessionmanager-fdenytsconnections>

<https://www.top-password.com/blog/enable-remote-desktop-with-registry-powershell-or-command-prompt/|https://www.top-password.com/blog/enable-remote-desktop-with-registry-powershell-or-command-prompt/>

Not just fDenyTSConnections, but there's another value called fDenyTSConnectionsBackup

yes, we have the same behavior.
with TS-Server 2016, at the moment Server 2019 is good.
we do a complete restaging of the system, all strange to me.

but as i can see, solution is to delete TimeBomb reg value.
thx

<@ULLH5JGFR>, I have Server 2016 images. I don't have anything else to compare it to if it's there for 2019 and 2022. But the value exists on all 4 images I manage.

I just don't know what it does. If it is benign or operates as some sort of fall back for RDS. It bugs me that deleting a reg value that should have no bearing on a RDS server once it is licensed fixes a license issue every 120 days.

Looks good to me. I just did this as well. I exported the MCS full clone user assignment as well to know who was assigned to what MCS dedicated machine

1. no, actually It prevents issues
2. yes

Just don't rely on the power management when LHC is enabled. If the machines reboot, they go in MM. ( or use to)

it depends if you use pooled random desktops or not. Pooled random doesn’t get any benefit from lhc because there’s no cached user to machine mapping. assigned desktops are cached.

We use lhc with random pools, you just want to make sure to change the config for reusemachineswithoutrebootinoutagemode to true.  It can be done at a site level or by delivery group.

Can you get back in to a pooled random group after logging off?

I always thought it was just established sessions 

Only by this I thought 
<https://docs.citrix.com/en-us/citrix-virtual-apps-desktops/manage-deployment/local-host-cache.html#application-and-desktop-support|https://docs.citrix.com/en-us/citrix-virtual-apps-desktops/manage-deployment/local-host-cache.html#application-and-desktop-support>

I must have got confused with something else, I thought ot was an issue with multi session too

I'd forgotten about the cross zone launch thing, that would suck for roaming users in an outage 

Glad I read the link you posted <@ULLH5JGFR>. I know in cloud they have it so you can make the setting site wide or at least the default for all DGs allowing it, but it looks like onprem you still have to set it per DG.

Looks like site and DG and it will resume Shutdownafteruse even with the outage command update (ReuseMachinesWithoutShutdownInOutageAllowed $true)

doesnt work in 2016. only in 2019. When you run in compatibility it uses the older style notifications.

Nice one - thanks, Danny. We have set the the only user complaining and he'ss good. Thanks for confirming :slightly_smiling_face:

yea we had the same thing here but luckily no one cared too much. We did end up going to 2019 and users were definitely happy it was working. I was trying to find an article to back up my claims but failing to do so atm.

VDA GUI Fehler.jpg

What was the orginal VDA install you had before the upgrade? Use that to install it again, then use Control panel to remove it, then VDA cleanup tool. I have ran into this since 7.15 from time to time.

HI Jonas I asked support and they said this was not a recent change.  But if you run the vda install from the command line with /VIRTUALMACHINE then it should show you the normal prompts

Thanks for the suggestions! I will test them out and keep you up to date about it.

<@ULLH5JGFR> unfortunately your suggestions didn't work in my environment. But I was able to install the VDA by using the /VIRTUALMACHINE parameter which <@UM04QUUSJ> suggested. Thanks again for your help

we had the same behavior with Moonshot cartridges and opened an incident. they dont recognise that the underlying system is hardware. Only command line installation works.

unfortunately the case was opened with my old account and company, but if you open an new incident, they should find this.
hope this helps, best regards
Thorsten

Ok, thanks <@U03P5EVA0R3> . Glad you got it though. Sorry for the extra work I added to you.

interestingly logging into the same server via RDP and opening the App-v package and peforming the same steps the issue doesn't occur only when you use ICA

For reference it was to do with the microsoft ui automation framework which gets enabled when you enable the citrix polices remote the combo box and automatic keyboard display. That was the cause of the crashes in the App-v apps

Depends on the sql cluster setup.  There are sql clusters that will have a cluster name to use .  Sql availabilty clusters will have a listener name while a sql cluster will just have a cluster/instance name

they don't have a listener just a sql cluster name

Probably not using a sql availabilty group then

Could be setup as a sql cluster.  Which would be cluster virtual name

got it...but will that will give them high availability ?

don't they really need SQL Always on or SQL Mirroring?

I've never created the databases using the cluster virtual name

always sql always on listener or the database server when a mirror is configured

No.  Sql clustering has been around for years and predates mirroring and always on.  Mirroring has been deprecated, and always on is the same as the availability group I mentioned above

I haven't setup a sql cluster since always on became a thing, but I know it still exists

Sql cluster is a MS cluster high availability solution as well

and specify the sql cluster name when creating the citrix dbs

and will give me high availability for the databases

Correct.  I would check with their dba to make sure it is a sql cluster and using standard port or instance name

Find out if sql agent and sql browser are also clustered

then why am i only doing sql always on for all of my clients...lol

Sql always on gives more flexibility, with the redundancy of a cluster and benefits of sql single instances

Sql cluster will use shared storage.  Sql always on will not

Both have benefits and their own complexity.  I prefer always on for many reasons, the main 2 listed above.  Cluster will only manage a name and not have to worry about the storage

i see...so thats why i don't see sql clusters being used a lot anymore

It takes a very strong sql admin to manage a sql cluster.  Easier/cheaper to manage the always on groups for most companies

One last point.  With clustering, you don't need to manage accounts across all nodes, don't need to replicate DBs, and have only one sql instance to manage across multiple nodes

Benefits to both.... But depends on what you know and can manage

all good points Mr Lange!   keep them coming.

I am using Citrix MCS 1912 LTSR to create Pooled-Random desktops. I have a master image that's 100 GB with 20 GB of Memory. When I check all the Server VDAs in vcenter, it's around 220 GB of "Provisioned Space", and the "Used Space" varies. Shouldn't "Provisioned Space" be around 120 GB? Shouldn't it be the potential size (100GB) of the differencing disk and VM swap file? When I look at the hard disk sizes for the VM, I see 100 GB Thick Provision Lazy Zeroed disk and 16 MB (thin provisioned). I browsed the VM files in the datastore and did not see any vmdks that are 100 GB, so where is VMware getting this info from? I thought only two disks should be created (differencing and Identity), so I don't understand why "Provisioned Space" is 220 GB...Can someone help me understand this?

I can tell you in the above sentence exactly what isn't supported :face_vomiting::the_horns::face_vomiting::woman-facepalming::face_palm::face_vomiting::the_horns: new farm, export import baby 

Yeah I know none of it is supported lol. Customer wants to decomm their SQL server and move all DBs to SQL2017 first and Citrix upgrade will come later. All on 2012 R2 as well... I'm just unsure if 7.11 will bitch about the newer SQL version.

That will be the last resort if we have to, but given change control procedures and having to do the work out of hours etc I'm hoping to only need one attempt at this if possible

The environment is too big to look at a side by side build? <@UK89L1Q02> is awesome with sql, he has likely seen this scenario before 

You can most likely restore the DB and change the sql compat level to an older version on the DB

And that should work to keeping it as the older version

Problem you may run into is TLS 1.2 on newer version of SQL

They are only going to SQL 2017... which is supported for 7.6 and 7.15 so I'm hoping that means it will also work for 7.11, just trying to confirm first. I think we will just have to give it a go...  and thanks for the tip Hal, I'll set the compat  level to SQL2016.  TLS if it's an issue I'm pretty sure can be disabled with a reg key.

Correct.  It can be disabled via registry

if you need to know where to set compat level

also our plan for the controller swap (correct me if i'm wrong on this) is to add the 2 new controllers into the environment so we'd temporarily have 4, then remove the old 2 after

thats exactly how i just did it.  Don't forget to adjust any scripts that reference the old ones, or add the AHV plugin if Nutanix, or Firewall rules for hosting connections, etc...

so if we do nothing else and JUST add those 2 new controllers in, will that autoupdate feature add them to that ListOfDDCs reg key (and at the same time remove the old once we clear out the old i assume?)