World of EUC

2nd deployment I had to set the regkey, I might stop enabling the auto enrollment GPO. Anyone else see "in correct username and password" in citrix fas and the app server was missing this setting? (Server 2019/Domain 2016) <http://www.cievo.sk/2012/07/02/problem-with-enterprise-certificate-distribution/>

Have you assigned the users to home zones?]

but the zone preference order is a bit unintuitive. <https://www.citrix.com/blogs/2017/04/17/zone-preference-internals/>

But how do you assign a desktop with the same name and use zones? Apps work great. 

You can put multiple machine catalogs into a single delivery group and have each catalog’s Hosting belong to a different zone

if you assign a home zone to user groups then the desktop launches out of the home zone

you don’t really assign the desktop to the zone, you assign the user

you can tell it to prefer the home zone, or only ever use the home zone

does it also happen when running the MS Remote App while NOT in the VDI?

Any reason you're not using a Citrix Published App, vs RemoteApp ?

3rd party app.. not our choice, something we're forced to use.

published apps, through Citrix and/or RemoteApp can be pretty weird sometimes

Hi maybe it is this? <https://support.citrix.com/article/CTX212905>

Thanks. I found that yesterday in my google searches. I dismissed it as it says 2012 R2, but it might be worth a shot after all

Yes but i see this on my 2019 Servers as well, there was also some issue with Firewall Rules

Ah, so the registry fix solved the issue?

App Readiness just creates your UWP apps, but has the unfortunate effect of stopping Windows updates being applied if you disable it

I know, but why would it sit there for 10-20 seconds on a black screen when that happens?

nevermind. NAT issue needed both private and public IPs in the policy. :smile:

If your published applications are hosted on a specific set of servers, you could enable the User Config | Admin Templates | Start Menu and Taskbar | Remove access to the context menu for the taskbar GPO. That might work

That was my first thought but does not help in this case. Context menu still opens locally. Tried to find something in the seamless flags document, but no luck so far.

image.png

Using CWA 2006 as well. Looking to see if there is anyissue with that verion as well

Yes. Get this all the time. Running IGEL with 2010 CWA. Or if it does load, it takes FOREVER.

Yea, I see that as well. It will take like 5 minutes to load, if it loads

For once, I wish something would work out of the box once settings are set.

Agreed. It did work well for a bit for us. We're on 1912 CU2. What VDA version are you on?

1912.2000. Yea I was reading that people get it work well with 7.15.

I seem the same behavior on 1912.1000 as well.

Seeing the same thing. That's disappointing. I wonder if it works better on &gt; 1912, or server 2019.

Not sure, Anyone else out there have any experince?

This part?
Due to the limitation of CEF(Chromium Embedded Framework), client endpoint GPU needs to be disabled if DPI scaling factor is set to a number other than 100% in order for BCR feature to work. Otherwise BCR displays the website in a corrupted fashion (like zoomed in and cropped). To disable, configure on the Client:
• HKLM\SOFTWARE\Citrix\HdxMediastream\
• For 64-bit:
• HKLM\SOFTWARE\Wow6432Node\Citrix\HdxMediastream\
• Key: GPU (DWORD)
• Value: 0


Not yet. Just looked it up when you said something. I can try this. Thanks for the quick tip

When the scaling is over 100%. I‘ll get blank sites. 

Ok, That is good to know. Didnt know that.

I'm running an IGEL TC, and still getting the white screen.

What scenario ? Client fetch , Client render?

One other thing that was a show stopper for me was that when using Chrome on my 3 monitor system, the <http://youtube.com|youtube.com> would be redirected, but it would appear to show a blank screen until I expanded the Chrome window across all 3 monitors, then the video would show on the middle monitor. IE did not have this behavior. Decided that BCR wasn't worth the headache at that point. May look at it again down the line.

LXOS 11.04.x. Dual monitors - so I wonder if I'm seeing the same thing as Dennis. Will have to try that.

Yea, If I span it accross all monitors like Dennis said it works haha.

I tried spanning mine across all monitors, and still no go for me.

So I have a chrome huge chrome window. Well just doesnt seem like it worth the effort anymore being its wonky

On IGEL there is a registry setting to use the CEF browser for BCR - with this enabled BCR wotks pretty stable for me. Give it a try!

It’s somewhere under ICA, AllRegions IIRC <@ULHA2PS2H> 

Found it! Configured it! BCR seems to be better. Still takes it more than a few seconds to connect "correctly." Here's the location:

Going to log a ticket I guess with Citrix support on the whole mulitple monitors and not being able to see it on one monitor. I know I am missing some silly, Going to do some google Fu and see what I can find first. It works well, but the whole multi mon solution will not fly with my users.

<@ULLH5JGFR> I hope you make progress on this with them. I really just figured it would get fixed in a future release and I'd look at it then. I didn't log a ticket because I despise working with Citrix support.

I would appreciate if you can keep us/me in the loop on the ticket.

Not a problem Dennis, I will keep everyone updated.

Well, I updated workspace from 2006 to 2012 and it appears to be working now. WTF?

That is for youtube as a example as well.

<@ULHA2PS2H> taking IGEL out of the situation, how does this work for you on Windows Devices with workspace 2012? I updated to workspace 2012 and it appears that its working now. Which I wanted someone else to confirm this. <@U012AAZRJ20> if you have time could you try it as well?

He's not even able to get BCR plugin to connect. If it's not one thing, it's everything else.

Took me while to find how to get it enabled again. But running workspace app 2012.1 it appears to be working properly now.

That good that it matches what I have now. Hopefully Tim can get some good results.

Added <https://accounts.google.com/*> to the authentication whitelist, and YouTube still takes upwards of 45 seconds to load. That's a LONG time. I will need to test BCR on a Windows device, I'm not sure my engineer understood what I was asking of him.

I get this for 6-7 seconds after visiting <http://youtube.com|youtube.com>. Once loaded, it seems to work fine. VDA is W10 1909/1912 LTSR, and I'm on CWA 2012.1.

I *WISH* mine was only 6-7 seconds! It's routinely 45.

I wonder what the performance is like when using Edge Chromium.

Well shit, looks like I win. What's my prize?

Seen this at customer, where some VMs are not powered on (PVS). I just created a script to loop through and start if not started

Anyone had experience with Nice CXone screen agent software?  It’s a screen recording tool we’ve been asked to add to a group of our Win10 VDI desktops.

This has been a popular topic in r/Citrix lately, too: <https://www.reddit.com/r/Citrix/comments/klqx0s/win10_1809_cache_disk_is_filling_up_even_when_we/>

So, running Procmon, it looks like it might be our AV solution, writing to a log file.

<@ULJM9DQF2> has a great Posh (what else) script for this. <https://github.com/guyrleech/Microsoft/blob/master/Get%20files%20modified%20since%20boot.ps1|https://github.com/guyrleech/Microsoft/blob/master/Get%20files%20modified%20since%20boot.ps1>

May help.  But Guy Leech scripts are the bomb
<https://docs.citrix.com/en-us/advanced-concepts/implementation-guides/digging-into-pvs-with-poolmon-and-wpa.html>

Thanks! Yeah, we've got a new version of our AV app and some new tools to play with. Daft question, but us using MCS shouldn't change using any of these tools, right?

To be honest, I do not now. I would think it would though being the concept is the same.

Yeah, that was my thinking too, that it's the same concept. I was thinking that writes are writes, and that it shouldn't really matter between PVS/MCS, just have to find the source of what was creating them.

But, I'm happy to be educated on that, if it's not a correct assumption

I assume that AV client also has a sealing procedure before you finalize your vDisk like most others? That would be the first thing I'd check.

(where it scans your image so that the client doesn't re-scan everything at first boot in standard mode)

It has a /VDI switch that was supplied to us. We're using that, other than that documentation has been a little sparse, but I will check on that also. If I recall, they hide their documentation and require and account to be tied to a contract. So, we're essentially flying blind.

Chiming in here with a followup question. Sure writes are writes, but most of the writes we’re seeing are to users appdata\ locations.  So… if using FSLogix profiles are those writes made direct to the VHD and not to the cache drive?

<https://worldofeuc.slack.com/archives/CKBVDG48H/p1607990384212700?thread_ts=1607986996.211500&amp;cid=CKBVDG48H>

Experimenting this morning. Should there be a mcsdiff.vhdx, regardless? Noticed on one test VM that there isn't one. Not sure if there's something up, or if the writes aren't overflowing to create one?

I don’t have one so I’m assuming it doesn’t create it unless it has to

turn on windows updates if you want to see it fill up :stuck_out_tongue:

Along these lines:

<https://old.reddit.com/r/Citrix/comments/kshkq4/cache_disk_filling_up_how_does_the_vdiskdiff_work/>

Out of ~213k writes, SentinelOne appears to be responsible for most of them (same is true for WriteBytes too):

You will need to edit the Hosting connection and add it. It will put the data on both Datastores but not in a HA fashion. It will put the data that is on 1 data store, and put the same data store on the 2nd datastore

Hi Ray - can you explain a bit more what you mean by "same data store on the 2nd datastore"?  From my reading it will take a copy of the template VM and also put it across the datastores, but maybe this is only during the creation of the image catalogue?

If I am understanding you correctl. If you add another datastore, the data that is one the first datastore, will all get copied to the second datastore. It will act as if the data is mirrored. Taking up the same space. are you using MCS?

As far as balancing. This is why the data is mirrored accorss all the datastore is you have more than one. The load balancing of the machiens is handle at the DDC level. The data will be copied to the 3 addition datastores you add.

This may help you <https://docs.citrix.com/en-us/tech-zone/design/reference-architectures/image-management.html>

MCS does not rebalance existing machines. Any new machines you create will be round robin across the datastores.

I recommend one datastore per hosting connection and separate Catalog per datastore so you have maximum control over datastore selection. Combine the multiple Catalogs into a single Delivery Group.

what about running poolmon.exe maybe you have a memory leak especially if the machines become slower over time during the day

Interesting, wasn't aware of that tool, thanks! It was moreso that I was curious about what folks do about page files.

WEM releases excess RAM that process consumes to pagefile if using memory optimization 

That's good to know also. We're looking at WEM in the very near future

set it to 20GB fixed and then ignore it, seems to be working well for us

we had it set to auto but it never grew from 4gb and was causing massive issues with hard paging

it doesn’t seem to write to the overflow disk at all

We were tempted to start a little smaller. But memory and storage doesn't seem to be an issue in our enviroment

memory is an issue for us as we’re using reservations because of GPU, but write latency is also an issue. Upping the RAM cache reduces the write latency

we were seeing huge spikes as we only have one big resource pool that everything shares

once we gave it enough ram that it didn’t need to write to disk then the spikes went away

also had to redirect the chrome cache to a location where it would end up in the RAM cache and not in the fslogix vhd

Cool. This is all great stuff. Looks like I'll be rebuilding Machine Catalogs this afternoon.

Hi all! Does anyone here have BOX and their applications running in a virtual desktop environment? Box Drive has been installed but we're having issues with performance when using the tool. Looking for suggestions in troubleshooting.

What if you use the UPM Container feature for the Teams folder?

The machines are azure ad Hybrid joined ?

image011.jpg

Have not investigated the UPM Container.  Did not really want to add another piece to an already frustrating puzzle. But i can certainly read up on it.  I assume I would find some good info on your site.

UPM is logging, but i’m not sure on the question, sorry. UPM would just log what it was doing, not what teams is doing… :confused:

Machines are not Hybrid joined.  We were initially also getting this stupid message on each session, but MS provided a REG fix to kill that.

I have had the error many times with upm in Windows Server 209. Azure ad joined was the only solution.

<@UR9T47CRM> MS gave us this, which killed that message box.

```HKLM\SOFTWARE\Policies\Microsoft\Windows\WorkplaceJoin: "BlockAADWorkplaceJoin"=dword:00000001```

I don’t mean the stay signed in window. I mean the sso behavior of teams. 

Ah. Sorry. Weird thing is on an identical set of machines that use FSL for profiles it works fine.  If i switch to using FSL for my account on the same group of machines it works fine. It dies with UPM only.

I may try the suggestion of putting teams folder in a UPM container this weekend.

It was all working fine since January when we started using win10 / teams. Then out the blue it stopped.

UPM logs will help you see if UPM is doing if something in UPM is part of the problem 

use to have only one licence server for a worlwide infrastructure, never had any issue with it.

Got a couple of customers with that setup works fine mostly, one big consideration is to ensure version creep of the sites doesn't occur as sometimes there's a reliance either up or down on a version of the licence server.

Bigger write cache probably best option. Using the web client is practically impossible as MS write so many entry points to the app.

I agree with <@UL5TCSW3D>. Either a bigger write cache, or accepting the storage increase that comes with Teams

I usually go with no less than 20GB write cache for a Session Host server

Our PVS infra (which is hosted with a CSP) uses a 60GB write cache :astonished:

Balls.  Makes sense guys thanks, but still - balls.  Has a reliable method of controlling the auto-launch 'feature' per-user on a session host been figured out?

Best dust off the cheque book then by the sounds of it.

<https://james-rankin.com/articles/microsoft-teams-on-citrix-virtual-apps-and-desktops-part-1-installing-the-damned-thing/> - part #2 should be along soon, assuming I can actually finish it

Perhaps I need to use it more frequently throughout the day, but I haven't seen Teams load up my PVC WC disk yet (W10), and I've got a number of redirections enabled for it.

I can confirm what Nick says. Never had an issue with the PVS write cache. 

I guess it's worth noting that our setup has 4GB pagefiles, and the vDisk is set to Cache on Ram w/ overflow, and a 10GB WC disk.

The obvious difference between W10 and W2016 is we have 15-20x the users per VM.  At 500MB of teams cachestorage each that's up to 10GB of write cache gone before we reach 10am.

So my paltry 12GB (i thought i was being generous!) needs to go up pretty sharpish!

Agree with <@ULL6F7DRA> 20GB WC is starting point. Hard set page file size as well.

is your write cache disk not thin provisioned? Why not just set the WC to the size of the C: drive and be done with it?

That’s what I do with MCS and then set RAM cache to 6GB.

Good point Mike! Our cache disk is about 30-70 GB. If you use Cloud Cache you also should use it for the FSLogix cache folder, otherwise the FSL cache gets into the PVS RAM cache. 

I am on 1809, I can give you my specs if you still want to compare?

I ask this because with just the OS, Skype and Teams open, that's eating 4.5 of the 5GB of RAM. JFC

We use 4vCPU and 8GB of Ram. about 3.5 is the OS and base things like office and normal operations ruaning

I don't even want to open Chrome at this point.

Edge is on the way, but we have to go through an official project, which means May at the earliest.

If I remember correctly I think there is something like a 10-15 performance decrease when going from Office 2016 to Office 365

Skype, Outlook, Teams, and Chrome with 2 tabs open.

side note if you do run chrome, set the ChromeCleanupReportingEnabled

No I wouldn't expect it to be very snappy

Ray: If I'm on the same page, I do have Chrome's software reporting or cleanup thing disabled.

I can probably use PS to bump up the memory on these VMs, but my capacity is going to take a hit

2vCPU should be ok though IMHO. What is eating up the CPU cycles?

Teams is the worst with RAM, followed by Outlook

Well at least here in DK, browsers are among the most used apps, and the resource consumption is sometimes through the roof

I have seen nasty CPU spikes when switching between different channels in Teams

My users stated they needed Chrome, Then I deployed Chromuim EDGE, and they said they are not using that. I said its the same thing. They said no. So I was like ok, I tricked them and pushed out Edge and changed the icon to Chrome and made it say Chrome. Then they were like Seen Chrome works good. I was like yea, it does LOL

55% CPU hit in Teams when switching between channels.

Damn, now I'm worried about addressing this before the vDisk goes live.

We're still a few years away from a hardware refresh, too. One that I'm not sure will even happen. We're on old UCS B200 blades.

Didn't someone tell me 3 vCPUs was now the sweet spot for Windows 10 VDI

We are doing that for a group of users to test.

I have read everything - 2, 3, 4 vCPU. But all of this was lab testing.

Kinda what prompted me to ask here, from real world users.

Users with Chrome, Teams, Outlook, OneNote and Slack plough through 8GB easily

I have 4 users with 128GB of RAM. But that is nVidia Cuda

Our highest spec users have 32GB RAM and 16 vCPUs for investment apps

If I bumped the CPU, it'd have to be to 3. I can't do 4 with this density right now.

Most of our apps are in XenApp, so that's helping

FWIW, I created a spreadsheet to figure out our density based on some Synergy sessions over the last few years, so everything is basically 'this should work'. So far it has, but now I'm kind of worried.

(as I type from my own 4 vCPU/10GB VM, haha)

3vCPU has been the sweetspot in our testing, 2 vCPU for higher density and 4 vCPUs just gave 1% extra app responsiveness vs 3 vCPU but density dropped another 7%

I’ve seen anything between 4-16GB: It’s whatever you feel comfortable with, just realise Windows/Electron apps will chew up as much RAM as possible

At the very least, I've got the PS code working, so I can at least script this change easily.

Hmm, 666 target devices per XS host in my environment. I am 100% sure that I did this intentionally. haha.

Looking at my estimations, if I went from 2/5GB to 3/8GB, I'd lose nearly 1k total VMs.

Think of the smiles on your users' faces though

The big wrench is that we're just about to go live with our shared device replacement where we swap a generic Windows logon box with IGEL thin clients that launch a Citrix desktop. I won't have capacity to replace all of them.

I think anything under 3vcpu/8GB RAM is going to be a very poor experience unless you lock down exactly what people can do

If you're on-prem don't forget NUMA when looking at CPU assignment, it does make a different.

I haven't even heard that phrase in forever. Is there an easy guideline?

Not really. Work out how many numa nodes you have in your physical kit and divide your number of cores down on a vm to fit within a node. Thats the simplest answer. XenServer can be a bit wafty with core assignment though. IMO its where all these odd core numbers come from.

We're running Intel Xeon E5-2697A v4s, 2x 16 cores. 512GB per host.

Its vmware but a pretty good explanation. <https://blogs.vmware.com/performance/2017/03/virtual-machine-vcpu-and-vnuma-rightsizing-rules-of-thumb.html|https://blogs.vmware.com/performance/2017/03/virtual-machine-vcpu-and-vnuma-rightsizing-rules-of-thumb.html>

At 3vCPUs NUMA impact is relatively low, its more impactful when doing RDSH/Server VDAs (I.e. larger VMs)

make sure to turn cpu hot add off if you want to see the benefit of vnuma

Deep dive <https://frankdenneman.nl/2016/07/08/numa-deep-dive-part-2-system-architecture/|https://frankdenneman.nl/2016/07/08/numa-deep-dive-part-2-system-architecture/>

Agree Kees. Larger VMs have a bigger impact due to spreading themselves across nodes.

Don’t assign an odd number of vCPUs when the size of your virtual machine, measured by vCPU count or configured memory, exceeds a physical NUMA node.

I'd probably leave the 4 vCPU VMs powered off when I make this change, so 99.9% of the VMs would then run 3 vCPU.

not an issue for 3 vcpu though, but I tend to avoid odd numbers of vcpu

COnversely, Nick Rintalan says not to be afraid to use an odd number in certain scenarios: <https://www.citrix.com/blogs/2013/10/24/xenapp-scalability-v2013-part-2/>

yeah, I should probably look into it more, it’s more of a superstition thing for me than anything else, it only really becomes an issue when you hit 11 vcpus or more

2013.. welcome in a world with vNUMA aware hypervisors ;-)

I'm less worried about CPU, if I'm being honest. Even the rises during certain tasks aren't that bad. It's the memory that's the bigger issue.

This change will drop my density down by over 20%.

I mean I guess it depends on how many cores your processor has

I know for VMware, but AHV I am not certain

Maybe I can use this conversation to drive the push for new hardware, since we're close to a refresh.

We’ve tested all of that, besides NUMA 8vCPU is a good spot for the Windows guest OS

10 or 12 vCPUs just add marginal perf so at that point it’s better to scale out instead of scaling up

I'll handle it like John Lithgow did on 3rd Rock from the Sun when it came to tipping. Everyone starts out with 3 vCPU. The more annoying the user, the more likely they get pegged back down to 2 vCPU. Problem solved?

We had to move all finance users to 3vcpu /12 gig Ram from 2vcpu 10 gig, even then users complained, our build was fully loaded with lots of agents and services, it was clear some software simply doesn't function as well as Physical, anyway I guess at one point we started placing users on 4vcpu and 16gig , from what I heard since I left in July they were looking to move users to surface pro's I guess that's for remote work not necessarily to do away with xendesktop. I used to tell them it's the extra security, appsense, crowdstrike, features such as systrack and so on which may add to slowing it down and of course 365 , but damn seeing office 2010 out perform W10 365 large spec vda's was some what annoying, Also going from a physical which most having a graphics card of some sort to having no vgpu - likely impacted our applications performances. I guess who ever designed the build years ago before our team started there didn't think far enough to take into consideration all the new security software and agents.

I take it NUMA is Numa regaurdless the HyperVisor? For me its always been Vmware. Now its AHV as well. Is there any differences around AHV <@ULJH9ED53>

Same on AHV, it’s not a hypervisor item, it’s RAM/Sockets, we all have to deal with it

yeah Microsoft have a fix scheduled for Q3 i believe to address this one, scheduled task for installation of the driver on boot is also an option (madness i know but hey....) - Windows 10 Multi Session has an issue with MCSIO also, remove the MCSIO component and things appear to get better (as in you can actually get MCS to do something)

I cant believe I spent so much time on this. Luckily MS support were good to work with. Its odd that Citrix does not have any info on this issue even though they hve several articles discussing how awesome this offer is.

yeah, i feel you, but it has some focus at the top at the moment for both of these issues, hopefully it's documented/resolved shortly

Also MCS does not allow premium disk on NVV4 from GUI. At least with CU1. I havent updated the farm to CU2 yet. maybe thats fixed.

ah i haven't touched on prem integration with Azure for a little while - only cloud :)

Release notes look like cu2 adds support for the premium disc machine 

Hybrid join sounds to me like the easiest way around that problem. But then again I have not yet found my self in a similar situation

Thanks, Kasper. That's what I'm thinking... my previous Azure deployments had either public IPs (for other app reqs) or a firewall/NAT for the VDAs, cheers