Hey guys! happy friday! I'm currently helping a quite large Citrix customer and I have a few points that I would like to discuss. Their Citrix servers are quite enormous, 96GB RAM & 8 vCPUs(2sockets). They have around 200 servers and the user experience is not good. Basically, login is around 50 seconds to 3 minutes and the desktop/app experience is slow and laggy. I checked their load balancing settings and its set to 85% cpu, 80% mem. concurrent logon tolerance 2 and maximum number of sessions 12. Load index is around 8000-10000. They are running FSLogix and they run both profile and ODFC containers for some reason. I have been running Control Up to analyze their logon and found some stuff to optimize but still far away from stable. I have suggested a new windows server 2022 and remove all the components including FSLogix to try and isolate the problem. I would say 2025 would be even better. However, they are not too happy about starting a new project with a new environment and want to solve the current env. I'm trying to deepdive and trying to understand what causes the CPU spikes. I am checking to see if I can find some more control up scripts that might help me understand what causes the CPU spikes. Read somewhere that disabling windows search can improve the performance because files will not be indexed anymore but I don't know if that will break the profiles. Any suggestions? what would you do in this type of situation? https://community.spiceworks.com/t/high-cpu-usage-on-citrix-servers/768121/5

Update VDAs and Director to the latest versions that provide uberagent style data. Look into the full breakdown of the login flow to see what is taking the most time.

Because you are using ControlUp you already have the details Jeff mentions. You can rollout RemoteDX to get client metrics. BUT I would also include the hypervisors into ControlUp and first check "vCPU7pCPU" Ratio and "CPU Ready" or "CPU Wait" on the VMs...

They are limited to 12 users per box right now Jeff, thanks will check if I can test uberagent and gather some more data

They tell you if there are resources when resources are needed. Always start at the host, if the host can't deliver (and big hosts don't mean they can deliver, there is a reason there are only 12 users per box.. is box equal to host or is box equal to server?..

Thanks Rob, I am checking the hosts and they are around 30-50% cpu and RAM. They were set to 12 users by a consultant to even out the load so there is no real good reason for that. I am now trying to understand if the hosts are in eco mode. I am not good with xenserver but I'm trying to understand what to check and how

I am not good with XenServer either, I know my vSphere better. The eco mode would be the host power mode.. it should always be on full power.. can't imagine it being wrong but I've seen it in the distant past causing issues.

I second what Rob already mentioned. Do you have a Chance to get a (read-only) User for XenServer and then add it to ControlUp: https://support.controlup.com/docs/connect-to-your-hypervisors There you should See the „Power Management“ column Rob mentioned where you can See „High Performance“ or „Balanced“ (ECO-Mode)…

balanced... yeah that was what it was called...

Mostly its a BIOS/UEFI Setting for the CPU. CPU Cores going to sleep and being woke up within a Short time period, then you dont get the real power…

this is what its set to in idrac

That screams storage latency to me.. Profile load time at 47sec for FSLogix containers is pretty rough. I would focus on back hardware first (storage, even network) and move forward, highly doubt it's something specific on the VDA's. I typically run Profile containers and ODFC containers but that is generally from a transition perspective (move to ODFC first for O365, then to profiles) and haven't had any issues with load times.

Thanks Chad, can I somehow prove this? they run Xenserver. I mean some type of test on the file server where the profiles are etc. Can the latency also affect the user experience with slowness?

ControlUp should give you some data around storage as well (someone else may need to chime in on specifics with ControlUp though). Are your XenServer hosts using local storage or on a SAN? Assuming that large of an environment it's SAN, look at the SAN controllers and see if there are any performance metrics you can pull.

thanks Chad! will check it out