World of EUC

image.png

Sounds interesting, thinking about Kaspersky AV installed on VDA? <https://support.citrix.com/article/CTX241179>

Sentinel One and Defender, I will analyze the logs more in the detail if it happens again.

Seeing the same here with the latest Teams, VDA and CWA version.

It might be a limitation in the machine-wide install of Teams or a limitation with Teams HDX Optimization

Even though I know that I'm not alone, it's just annoying... We thought it is related to our Proxy settings but as long as you are having the same issue I don't think its related to our Proxy as we are having the corresponding Proxy.pac from MS.
What we are also experiencing are call delays, what means that the call is established but we cannot hear anything for roughly 4 seconds then RTP is working. Have you experienced the same?

I have no experience with with regular calls in Teams, I have only used Teams for collaboration and meetings both with and without video

When you say Teams calls, do you just mean when chatting with somebody, you click the phone icon to audio call them? I do this all the time in Citrix and have never seen that kind of delay.

Sometimes also for Teams Calls ( we call them internal calls ;-)), most problematic had been real phone calls to customers or vendors.

I'm running 1.4.00.22976 Public Preview, and I have to say, I agree, it's a lot more stable, resuming sessions etc. I used to have to close/quit Teams a lot more than I do now.

latest prod version is 1.4.00.26565, you should consider updating it pal :smile:

Damn. I'm usually good at running Evergreen on my machine to update it. Having said that, I rebuilt my machine with Windows 11, so that's probably why.

Hmm, mines newer than the one I had, but not as new as yours, which is weird.

Cool. Yeah, I grabbed the one from the <http://aka.ms|aka.ms> site.

maybe Evergreen reports an older one, try Nevergreen

Yeah, I'll give it a go. <@UR9T47CRM>'s script installs Nevergreen now too

yes. i must change the installer search from evergreen to nevergreen :slightly_smiling_face:

yes i must change the module for teams :wink: thx jonathan

team effort, most credits goes to Dan Gough

So, production is newer than General and Preview?

Ah, sorry. Yeah. And the difference between Evergreen and Nevergreen.

Different methods to get the installer. :sweat_smile:

yeah mine actually works :stuck_out_tongue: shots fired

Bildschirmfoto 2021-09-26 um 14.51.02.png

I’m confused… Does that mean that preview can sometimes be older than production??? So It’s better to always use production or am I wrong?

It's teams. So everything is possible and nothing is regular :sweat_smile:

<@UMB4XQE79> yes it's been the case for the last few months at least

For those who are testing out Windows 11, what are you doing with the profile versions? Using #SAMAccountName#\!CTX_OSNAME!!CTX_PROFILEVER! or something like that, or leaving them as is?

Yeah. Machine is poweredon but became unregistered when a user tried to reconnect after lunch break.
Same workaround, restart the VM.
Our guess is around VMWare tools…

In my case VM was registered however I saw too many sessions on such VM compared to other VM’s. Load balancing didn't work… and maintenance mode was helpful to eliminate server from load balaning.

Hi <@U01GWFT2BDJ> I worked for customers following Citrix article was not for a network change but for the vCenter certificate changed. It worked fine!

Image from iOS.jpg

I‘m sure the MCS part is pretty new as this worked only with PVS or manual deployed machine catalogs in the past (which worked without problems for me). I’m interested if MCS is working fine, too. 

I am wondering how that will work out since the provscheme needs to be updated as well to reflect possible new network assignments.

I deal with kind of mindset every day. I feel you pain. 

My greatest pet peeve is when they storage vmotion MCS provisioned machines (not full clones)

If you have access to the installation log file you get the arguments. It’s in the %temp% folder of the admin who installed the VDA. Good luck :wink:

VDA Installer should create a log file 'XenDesktop Installation.log' (by default)  - the log file should contain a line like this: 13:27:59.9211       : XenDesktopSetup:Commandline arguments: /components, VDA, /masterimage, /virtualmachine, /enable_remote_assistance, /disableexperiencemetrics, /enable_real_time_transport, /enable_hdx_ports, /exclude, Citrix Supportability Tools,Citrix Personalization for App-V - VDA,Personal vDisk, /noreboot, /quiet, /logpath, D:\Installs

if you dont want to go through the log file if you go into add /remove programs and click on change / add components and pre reqs you will be able to click through to see what was selected eg workspace app/ App v client / citrix files ect

you can check if DDCs were done as part of the install or pulled via policy by looking in the registry, HKEY_LOCAL_MACHINE\Software\Citrix\VirtualDesktopAgent\ListOfDDCs

so that's /controllers taken care of, I'm sure we could drill through all the options and registry locations eventually

I haven't yet. We still use it today with the limitations. But it depends on how dependent you are on teams I would say

I wouldn't dream of it without GPUs. Unoptimized Teams + video = good luck.

I do on many. But I am a community advisor with Rody on Nvidia side so a ton of mine involve GPU backed workloads. So by the nature of that I can do native unless latency/hairpin still is too bad. But from a density only standpoint yea, I can many times disable to do native

Executives especially don't like like having a "sort of native" experience. They want native

I preach, live and breath GPUs on prem and cloud so that's always part of my convo

I’ve done it a few times now (disabled optimisation) - with and without GPU - if teams is out there and you then introduce optimised teams - good luck with user acceptance - teams in virtual environments is quite frankly shithouse across the board 

exactly what i'm running into with introducing Optimized Teams after they were used to using it.

the give/take control during a sharing session is kind of a big deal

It’s a dogs breakfast. You aren’t alone. There is nothing more soul crushing and demotivating than the customer questions around teams in virtual environments (Citrix, AVD, Horizon) - it’s just crap and in 2021 I’m dumbfounded at the lack of progress - “optimised” cannot equal “kill off features that are core to use”

Microsoft: We're doing this for your own good...

What's your CU? 

As I remember from the past numbers were almost equal but not anymore… it seems to be some issue with internal load balancing 

It's not serious but something what should work properly not working fully properly. 

I’m paying every year full price for product - I’m expecting as customer products fully working properly. :yum:

Looks fine to me - most environments I see have an uneven split, don’t think this is a real problem based on those numbers

Define working properly. Just because it doesn’t register numbers evenly across controllers doesn’t mean it isn’t working properly

And the most important: once you restart a broker, VDA register to another at a glance. So, I don’t care either they are not properly “load balanced”.

I've never noticed that the controllers equally balance. The more important thing is if you take a controller down for patching or whatever, do any systems lose registration or don't connect to another controller. Is your first controller struggling under 42 registrations?

My question is, does this make sense? Since I'm using cloud connectors of course I can't look at any DDC logs to see what is going on

Does this happen when a sched restart is missed? Or does it happen whenever you restart the broker agent service? If the latter, then that makes no sense!

so, all hosted session on that machine are dropped and the host reboots when you restart the broker agent on that machine?

I did a check with engineering:

When the broker agent service is gracefully shut down by the OS, the agent signals this to the broker and the broker takes this as the OS being restarted as this is the common case, and there is no way for the agent to tell the difference. As the broker believes the OS is restarting it removes all the session records, and the machine appears idle; if the machine is set to shutdown after use, this then kicks in and sends a shutdown power action

yeah I may have a similar issue on Server 2019 with VDA 2106. Logoff get stuck, I suspect the nvidia driver.

so something specific to VDA 2106 then? Curious, did one of you try VDA 2109 yet to see?

I see the issue in RDP as well so I don't think it's Citrix related.

mhmh okay :thinking_face: no we have not tried vda 2109 yet - we will test it

yea, please do. Sounds like Jonathan is saying RDP shows it also, so 2109 probably won't help, but I'm curious what this ends up being. I always get strange GPU related shite working in it all the time, so just another one i want to keep an eye on

2109 has not brought any improvement. Maybe its related to the Windows Update CU from august. Now we will try to install the CU from september on a few VMs.

The September CU seems to have solved the problem for us.

<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.nvidia.com%2Fgrid%2Flatest%2Fgrid-vgpu-release-notes-vmware-vsphere%2Findex.html%23bug-3262923-disconnected-sessions-cannot-reconnect-with-nvwmi&amp;data=04%7C01%7CMaxime.Beaurivage%40groupemorneau.com%7C406a141423e94b11974d08d988ea8d99%7C332ffa18f17047fe9ce11b103d909683%7C0%7C0%7C637691363399002355%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=EM%2B%2F8bONgFcgzOhHbEzAb2vYeQSau29%2BpuYrAy6OlJQ%3D&amp;reserved=0|5.6. Disconnected sessions cannot be reconnected or might be reconnected very slowly with NVWMI installed>
Description
Disconnected sessions cannot be reconnected or might be reconnected very slowly when the NVIDIA Enterprise Management Toolkit (NVWMI) is installed. This issue affects Citrix Virtual Apps and Desktops and VMware Horizon sessions on Windows guest VMs.
Workaround
Uninstall NVWMI.
Status
Open
Ref. #
3262923
<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.nvidia.com%2Fgrid%2Flatest%2Fgrid-vgpu-release-notes-vmware-vsphere%2Findex.html&amp;data=04%7C01%7CMaxime.Beaurivage%40groupemorneau.com%7C406a141423e94b11974d08d988ea8d99%7C332ffa18f17047fe9ce11b103d909683%7C0%7C0%7C637691363399012350%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=d6lgvYRYzPomaMxsoQGWePQIaX1nFOXygrV5MQOkcDY%3D&amp;reserved=0|VMware vSphere :: NVIDIA Virtual GPU Software Documentation>

My client confirmed it fixed the issue and the GPU still seems to work fine

Perfect thank you. We will also try to uninstall it.

unfortunately uninstalling the wmi component didn't fixed the issue for us

Maybe I misunderstand, perhaps you are looking for this. client redirection is enabled with Citrix policies. not sure what the default state is default... (sorry, only had a cloud view, no vad1912)

Has anyone tried installing Citrix VDA on Windows 365 and access it via Citrix Cloud Management plane? Is it supported? 

That's not a new development. It's been known that 1912 is the last onprem version to support public cloud vdas.

Sad nevertheless. Cant believe large org's have not provided feedback.

I think I found something.

• `*Virtual Channel allow list security enhancements*: To improve security, we have changed the default setting from "enabled" to "disabled" for any custom/third-party virtual channels to be loaded in any session.`
<https://www.citrix.com/blogs/2021/09/30/whats-new-in-citrix-virtual-apps-and-desktops-september-2021/>

20$ bucks they blocked they're own RTOP channel.

Hmmm, Yea I bet. So I assume its not a Studio or GPO policy?

Nah it's part of the Citrix Policies, I told my client to add the exception and I'll report back once he test it.

ahh ok. Cool thanks for the details. I am still on 1912 LTSR. When I jump, I may have a rude awaking :disappointed:

i had a random deal a few weeks ago with customer hitting me up about "Green screen" around our CAD application on 2019. But wasn't able to dig into it at all and had not changed anything recently. They were 2106 though as well.  But when you said 'green' it made me think of it. It literally was a green background behind the app and was green everywhere.  Strange. I wasn't able to get into it with them though to troubleshoot and haven't heard anything back so i am assuming some hiccup of some sorts

<@U0158E1NUGH> could you DM me some screenshots and your setup ( hw / software / policies) so I can look into it.
( I work for the HDX team)

Ok, which vGPU version are you running? And which Workspace App version?

Could you try to change the Use Video Codec to: Do not use. That will disable For Actively Changing Regions

yes than the trouble ends, but then it will not use vGPU

Could you try to enable Hardware encoding?

Another thing to look at is the IGEL endpoint, did you enable H.264 Hardware decoding?

Have seen similar effects when h264 hardware decoding is enabled on the IGEL endpoints

If you have the _Legacy graphics mode_ enabled in the Policies. Try disabling it.

Seeing the same on IGEL with CWA 2109, not only for GPU workloads <@UNBRPVD4G> 

<@UL5TD2QM9> yes seems to be related to igel os. <@UNBRPVD4G> updates are all done. But i have some problems with actual releases from igel and citrix :disappointed:

<@UNBRPVD4G> same and reboot sheduler don't work over different customers.

workers sometimes turned off, instead of rebooted^^

Not exactly what you need but you can find relevant info here : <https://stevenlemonier.fr/how-to-create-an-mcs-catalog-in-powershell/|https://stevenlemonier.fr/how-to-create-an-mcs-catalog-in-powershell/>
And specially there by adapting the scripts to you need: <https://stevenlemonier.fr/export-and-import-on-prem-citrix-site-configuration/|https://stevenlemonier.fr/export-and-import-on-prem-citrix-site-configuration/>

I might have the commands to create the VMs as well if you need.

Are these persistent or non-persistent?  Are you using WEM memory optimization?

Any other system performance/memory optimization tools in play?  Is memory consumption high and are you paging a lot?  Is the page file on the OS disk? What profile management is in place? Where is the profile cache location configured to? C or temp/cache disk?

Have you looked at the ability to use Shared Image Gallery and Ephemeral OS disk on Azure VM sizes with a cache disk larger than their OS disk?

Does not look to be a correlation between RAM / CPU and Disk Latency. At least according to Director. The only optimization I have in place is Citrix Optimizer. As far as the profile I am using FSLogix pretty much out of the box.

I should add this is running on LTSR so not sure if this will work with ephemeral disk or shared image gallery

Cloudcache or standard VHDlocations for fslogix?

Have you tried putting profile cache and page file on temp disk to offload some iops?

Just to confirm this isn't PVS on Azure?

Ephemeral OS disk does work with 1912CU3 on Server 2019, just pay really close attention to the VM Series and cache disk size.  It's just sort of a pain to enable at the moment, not too bad if you're catalog is already configured to use the Shared Image Gallery.  If it's not it'd be easier to create a new catalog configured to use the shared image gallery and then enable the custom option on the provscheme in powershell and MCS/add some new vm's.  I've just started to work with this and am headed into user testing, so far it seems to perform very well for no cost other than some additional config work.

Standrd VHD Locations and have not tried to move the cache to temp disk. which sounds like good idea. No this is not PVS on Azure. Do you have any links on how to use ephemeral disks and shared image gallery and what mods I need to make to provscheme? I was going to test some other things in that regard (standard SSD)

<https://jkindon.com/2021/07/19/citrix-mcs-and-azure-ephemeral-disks/>

Yea I am just starting with Azure hence some ignorant questiosn :smile:

I was lazy, and didn't want to work through all the powershell to create the catalog, so I created the catalog in Studio, then after it was created took the existing CustomProperties and added the _UseEphemeralOsDisk value=true_ 

once the customproperties was updated, I provisioned a new VM and it was using Ephemeral Disk for OS.   I actually configured the catalog using standard hdd as it actually doesn't matter.  When you configure for Ephemeral disk, there actually isn't an OS disk attached to the VM.

Since I am not using shared image gallery I will prob need to create new catalog.

Before you do that, what size vm are you using? Vm size often caps throughput to disk… 

You can tap an existing catalog into a sig based one too, one line in your prov scheme 

these are NVv4. Not sure if they even support those disks. They do support premium tho. I also have some f4s

looks like they are supported <https://docs.microsoft.com/en-us/azure/virtual-machines/nvv4-series>

This is very cool. Can I just update provscheme of the existing catalog or do I have to build new prov scheme and new catalog?

Just tested. It works :slightly_smiling_face:

New machine created with "standard hdd" disk

Once both the SHared Image Gallery and Ephemeral OS disk custom parameters are in place you have to create new vm’s in the catalog

Do a Get-ProvScheme -PrivisioningSchemeName “catalog name” and look for the custome parameters in the article linked earlier… managed disk, shared image gallery and ephemeralosdisk will all need value set to true

Just to get back to this. It looks like these options are available for Citrix Cloud. In this case the environment uses 1912 connecting to azure.

On the cloud side an update too - looks like you can do this in the Citrix Cloud Studio GUI now as a checkbox during catalog creation.  After I did it manually a few times, typical.

Always use Citrix ADC to load balance Storefront. With load balancing it's easy to migrate SF servers. How many SF servers do you have? 

I didn't double check but I do not think upgrading the OS of Storefront is supported.

If you're doing In place OS updates remove Citrix components first, then upgrade, then reinstall. For Storefront: <https://support.citrix.com/article/CTX212452>

Citrix does NOT support in-place OS upgrades

Whoops... We previously updated our test site in this manner.   Thankfully all seems ok... Still, I'd rather do this properly for live as test had no users.

As a general rule, I just don't do in place OS upgrades. Build new VM, migrate services. I think the last OS upgrade I did was NT 4 PDC to 2000 domain controller, because that was the only migration path from NT 4 domain to AD.

Forgot to mention it does not affect all users.

I have the exact same issue since VDA 2012 on Server 2016. Got a private fix from Citrix which replaces a DLL. Support told me that I have to upgrade to 2109 to get it resolved, haven't done it for now. I can PM you the private fix tomorrow.

No, but I was asked the same thing. My first go to was nothing will beat the HDX side that Citrix offers.

I totally agree with you <@ULLH5JGFR> however what I am experiencing is that having the best transport protocol is sometimes not enough the people with the money are looking at other things

True, what I usually say is this. Citrix HDX protocol has been around for decades and many usb and network enhancements has been made to make it the best experience possible. Windows 365 is new and it may use the remote fx protocol or something newer. But will all usb functions work? With it handle low wan networks? Does it even have all this yet? If a company is just needed a desktop for a remote worker and no special functions. Then it may been good. Really hard to say. 

I have no idea the difference between this , azure vdi.

I have been working with on-prem based RDS based setups for more than 10 years. RDP/RemoteFX is still not better than ICA/HDX, but for some it's "good enough" and some also have focus on costs more than anything else

Is teams or usb devices stabile and work ok?

Teams, no :slightly_smiling_face: There is no redirection for RDS

USB devices mostly work using the generic USB redirection, if that doesn't work you are usually on your own :smile:

Really good to know. I was curious because I haven't used it very much before. 

I usually tell my RDS customers, that I don't really care if they chose RDS over Citrix. With RDS I will be able to bill them a lot of hours when I have to make up script based solutions to all the features they miss with Citrix or by selling them 3. party software trying to fill some managements gaps compared to Citrix

I asked MS about teleconferencing performance since it's still using RDP. They said it was all optimized and worked really well, but wouldn't bite when I asked if it was better than what we currently do with Citrix. I've also read about usb device redirection issues with Windows 365, but I'm not sure if that's valid or not.

My biggest issue appears to be my own manager - who does not own any aspect of Citrix or the virtualization space - continually bringing up MS solutions when we have a very large Citrix environment as-is, and can quickly add Citrix Cloud stuff in minutes if we needed to. It's frustrating, because I continually have to be the guy who shits on certain suggestions because they make no sense. We just waste a lot of time going through these cycles.

Citrix has been pestering us about moving some of our stuff to the cloud for years, and there's no doubt in my mind that they'd be significantly cheaper than MS. Money is a huge stumbling block in the not for profit sector, so it should be pretty easy to say ‘no, thanks’ to MS… provided that leadership is actually given the option.

We have that issue with managers as well. They don't understand the details and don't have the technical know how in cases like these. I deal with it weekly and it's frustrating for me as well.  It's like they sit in the toilet and read this stuff and come back with this and I have to educate them and then get on the phone with MS and break it all down. End the end they are like are you sure. I can go on and on about these types of managers but I would be insulting them on here. 

My company buys all cloud and I have to figure it all out. Such a pain In my butt. I know a lot but dang I so t know everything lmao. They really hate my smart butt personality at times. 

At the end of the day, you have to trust your SMEs. Otherwise, you're simply hurting the organization.

How are the servers provisioned? Via a VMware Template?

I don't know that solution, but it sounds like "something" has been changed in the image

I have done multiple deployments of 2019 via MDT, SCCM and VMware templates, I have never seen this issue

I was looking for a outside opinion because I can't believe that if Microsoft had a high-level issue that it would not have been solved. I'm sure one of our customizations is breaking the process.

If MS has an issue I am not aware of it, and I deploy 2019 on an almost daily basis in my test-lab

This is done via a "vanilla" Windows Server 2019 ISO where updates/hotfixes have been slipstreamed via WIMwitch/OSDSUS

I assume they've tried building one manually to confirm if it happens that way to rule vRealize in/out

<@UL5TCJ57D> you know the environment I work in. I'm sure it's something to do vRA or LanDesk but the team has buried their head on addressing it.

Just wanted to challenge the Microsoft is aware of the issue. You know me :slightly_smiling_face:

I'm fully aware which is why I asked :slightly_smiling_face:. Force them to see that it isn't a MSFT issue

Thank you for the feedback. I used it to squeeze the server team and magically they discovered an issue with the image...

Anyone know if it's possible somehow to add non-hybrid-joined Intune Clients to an OnPrem CVAD environment?

Greetings. ANyone using cloud w/azure authentication, MS MFA w/ FAS server?

Is FAS enabled in CItrix Cloud?

Did you configure the FAS GPO settings and assign it to your FAS server(s) and CItrix application host(s)?

yes. its turned on in cloud and GPo is set pointing to the FAS servers

Does your FAS rule restrictions include the XenApp application servers you want to access and the users you want to use FAS??

check the fas log to see if its generating a smart card cert

Depends on scale but Azure Files is fine and supported <https://docs.citrix.com/en-us/tech-zone/build/deployment-guides/citrix-azure-files.html|https://docs.citrix.com/en-us/tech-zone/build/deployment-guides/citrix-azure-files.html>

Seen it before a few times, in most instances I've come across its app's or processes that prevent a session from logging off. I'd start by looking at a long disconnected session looking at the running processes and kill them one at a time wait a bit between process kills and see if the session logs off. You might also have something in event viewer to add weight to that

But this shoud impact more users if it was hanging on a program ? In a normal week it happens 2 or 3 time and I have 42 Servers running.

which processes are remaining open? once I had to exclude kvg.exe (kaspersky AV) from LogoffCheckSysModules

but that should be needed only for Published Apps, correct me if I'm wrong

can you post what policies you have configured for auto disconnect/logoff? do you have one for idle session to disconnect? 
Also, on those hanged sessions, can you see whether the logoff process has been triggered in the first place (check event logs) 

But If it just from the Citrix Monitor use the logoff  then the user are actually logoff

if you can logoff the session from Director normally, that means the logoff policy is not being triggered on those servers.. check as above 

since it's a terminal server I would do the logoff in gpo try that out

Anyone use idp with sso for CItrix CVAD? I have an issue where my token  is not saving for web applications or m365 and every time I log off and back on I have to enter my ID all over again. I'm not sure what I need to include in upm.