Apologies if this is the wrong channel to post thi...
# citrix-cloud
c
Apologies if this is the wrong channel to post this - I'm working on a deployment of an MCS catalog of Windows 11 multi-session OS using the Image Management functionality in Citrix Cloud. Utilizing ephemeral disks for the increased performance + no cost compared to Managed Disks. Experiencing problems with app hydration; first login suffers from poor login speeds and application launch times but subsequent logins are fine. I've tried an auto login account and an application hydration script but neither seems to have had a positive effect. Some stats for reference: • First login this morning; Explorer/task bar loaded at 1m 30 secs. Teams completed loading at 2m 25 secs, and Edge launched at 6 minutes. The homepage and new tab never actually loaded though – gave up waiting after 10 mins • Second login (for the same user); had Edge and intranet loaded at 26 seconds, and Teams completed by 35 secs • Third login (different user); had Edge and intranet loaded at 28 secs, and Teams completed by 34 secs Wondering if anyone else has encountered this problem and has any tips or suggestions to improve the first login experience? Thanks
j
@Yuhua Lu wasn't there something around this we saw way back with Hydration challenges on Ephemeral Disks?
1
j
Can confirm this is pretty much the standard. You can use Citrix Probe to connect to the resource and run apps, or setup a scheduled task for on startup to launch and then close applications to pre-load them.
r
If you use a script to touch all the files it will hydrate them. You can also use SttlementPeriodBeforeSUe to have newly started machines use last
👍 1
👍🏼 1
c
Thanks for the replies so far, I think what's throwing me is that I expect the auto login to perform hydration of the basic windows processes + UWP apps. The behaviour I'm seeing doesn't reflect that though. I wonder if I need to hydrate additional executables in support of FSLogix...
y
Yes, Ephemeral needs to fetch content from storage service to host cache/temp disk at each launch. @Chris Rodgers not sure if you noticed, v5 and above, Azure SKU with temp disk is lot more expensive than those without. From non-persistent VM perspective, Ephemeral disk does not save you storage cost because we delete disk when VM is shutdown, so ephemeral when running > non-ephemeral + disk when running, depending on total size of disk of course
Yeah, you can start VM ahead of time and hydrate files, currently we don't have handshake mechanism (we are building one with Broker) to prevent session from starting before hydration completes so you want to have good amount of time powered on ahead
If you have FSLogix attached, then I guess you need to have FSLogix content cached at network service side?
We have a feature call MCSIO, it comes with option to retain OS disk. If you boot VM one-time (MCS does not create OS disk until first boot), OS disk is cloned and its content is copied in background (MSFT guarantee copy completion in 24 hours). After that, OS disk is always hydrated and clean because all writes go into WBC disk. It is non-persistent catalog in a sense that at each reboot, VM goes back to clean state but you also have a persistent OS disk until next image update at which time you need to reboot again to let new OS disk to hydrate with update image.
c
in reply to the cost difference between temp disk and non-temp disk SKUs, in our example it's a cost difference of ~£35/month/VDA, whereas a premium SSD (P15) is £54/month/VDA so ephemeral is cheaper and also has higher specs (500MB/s throughput for ephemeral vs 125MB/s for managed disks). We've proved this through IOPS testing as well that we get better disk performance on ephemeral
r
Yuhua was suggesting looking at mcsio which sets up a ramcache so you can use standard hdd storage instead. But you can just use the script approach to hydrate the disks.
c
regarding time before boot we've tested and it doesn't seem to matter how long the VDA is booted for before use, we have to perform a first user logon (not an autologon process) and suffer the poor experience during that first session before subsequent sessions perform normally. FSLogix content is indeed cached via cloudcache from azure netapp share, but we tested with non-fsl user and had same first login experience to try and rule FSL out as a main contender for the issue.
appreciate the comments about MCSIO but that's incompatible with ephemeral disks, so it sounds like the support for ephemeral disk is not quite there yet? I understand if that's the case and happy to go back to my customer with that explanation
r
Yes you need to actually use the files so you need a script that will touch them all and that will pull them all into the ephemeral disk. This si the one i have used:
Copy code
#Version 1.2 | 2019-04-23 | Citrix updated the revised script to dynamically recurse four directories then read all files with .dll and .exe extensions
#Version 1.1 | 2019-04-10 | Microsoft revised script to dynamically recurse the "Program Files" directory and read all files with .dll and .exe extensions
#Version 1.0 | 2019-02-07 | Microsoft created script to parse a text document with files to read.

$sw = [Diagnostics.Stopwatch]::StartNew()

function CacheFile([string]$filename)
{
    $bytes = [System.IO.File]::ReadAllBytes($filename) 
}

$filelist =  @(Get-ChildItem "C:\Windows\System32\*.*" -Recurse)
$filelist +=  @(Get-ChildItem "C:\Windows\SysWOW64\*.*" -Recurse)
$filelist +=  @(Get-ChildItem "C:\Program Files\*.*" -Recurse)
$filelist +=  @(Get-ChildItem "C:\Program Files (x86)\*.*" -Recurse)

foreach ($f in $filelist)
{
    $ext = [IO.Path]::GetExtension($f)
    Switch ($ext)
    {
     {($_ -eq ".dll") -or ($_ -eq ".exe")} {CacheFile($f)}
    }
} 

$sw.Stop()
$sw.Elapsed | Out-File -FilePath "C:\VDICache\CacheRunTime.log" -Force
🙌 1
🙌🏼 1
c
thanks Rob let me give that a try and see what effect it has
r
I would also use SettlementPeriodBeforeUSe and set to something like 3-5 minutes
c
we've set SettlementPeriodBeforeUse up to 10 minutes to try and give it plenty of time to hydrate, test results in OP were with that value configured
r
Yes its will only be useful if you also run the script
quick directions on that
Copy code
#Version 1.2 | 2019-04-23 | Citrix updated the revised script to dynamically recurse four directories then read all files with .dll and .exe extensions
#Version 1.1 | 2019-04-10 | Microsoft revised script to dynamically recurse the "Program Files" directory and read all files with .dll and .exe extensions
#Version 1.0 | 2019-02-07 | Microsoft created script to parse a text document with files to read.

This script is intended to be used for MCS provisioned machines in Microsoft Azure Cloud.  The script should be placed onto the master image virtual machine and is used to resolve slowness issues with the basedisk deployed from disk snapshots.

Instructions for use:
The below steps should be completed on the master image virtual machine.

1. Move this folder "VDICache" to C:\ such that accessing this file is C:\VDICache\README.txt
2. Create the scheduled task:

schtasks /create /sc onstart /ru "SYSTEM" /tn VDICache /TR "PowerShell.exe -ExecutionPolicy Bypass C:\VDICache\Cachem.ps1"

3. Test run the scheduled task through the Task Scheduler. Once completed CacheRunTime.log will be created under C:\VDICache\.
4. Power off the virtual machine and make sure it is deallocated in the Azure portal
5. Wait about 10 minutes for disk to be hydrated
6. At this point, you can create/update the machine catalog to begin the deployment process
7. Once machines are provisioned and powered on, log into the machine or remotely navigate and check the C:\VDICache\CacheRunTime.log modification date to ensure the script has finished running (Timestamp will be close to time machine was powered on)
8. Inside CacheRunTime.log the execution time of the script is recorded
9. Run applications to check the launch performance
🍻 1
c
appreciate this Rob 🙏
r
🙏 it helps
y
We have not extended MCSIO with Ephemeral Disk yet. If we do, it won't help your hydration problem. MCSIO will only help you with OS disk retained option when OS disk is hydrated after first boot.
Maybe you have typo? P20 is at cost you mentioned. Are you using Standard_D8ads_v5?
c
apologies yes typo from me, P15 is the alternative. Customer has used a few different VM specs through this project but yes Standard_D8ads_v5 was used at one point, with the difference between D8ads_v5 and D8as_v5 being ~£46/month so you're correct, in that case managed disk is the cheaper option
👍 1
We have not extended MCSIO with Ephemeral Disk yet. If we do, it won't help your hydration problem. MCSIO will only help you with OS disk retained option when OS disk is hydrated after first boot. Yeah I understand this as it's useless building a cache on a disk that's wiped at every reboot. Regardless of MCSIO support, my previous statement stands that currently, out of the box, ephemeral disk support for non-persistent VDAs is not quite there (unless you're willing to suffer the slow first session experience)
y
Yes, you are correct, cache (WBC) which has writes is wiped so that VM is clean after each boot. However, with MCSIO feature, OS disk can be retained across boot because no writes going to OS disk (all writes go to WBC cache disk), meaning at each boot, OS disk already has its content and will not suffer from hydration. Having said that, we have not done comprehensive test and recommendation yet because our feature sets are not completed for MCSIO yet, things like changing MCSIO setting on existing VM because you will likely change WBC disk size from time to time to accommodate usage change (more app installed for example). There are different parameters that could alter recommendation regarding performance, money & operation complexity which I hope we can provide comprehensive tech document later. In short, I think you want to go ahead with hydration script. If you are still not happy with hydration script, then maybe we can explore MCSIO option
🙌 1
n
Really useful script - is this on a github somewhere? @Rob Zylowski
r
No just a ccs internal script
c
some feedback following our implementation of the hydration script, here's our initial test results: First login is definitely improved, however subsequent logins have slowed slightly (extra 10-20 seconds): • First user; 59 seconds • Second user (same user); 42 seconds • Third user (different user); 55 seconds Same test results on a VDA which had just finished 10 minute settlement period and another which was left for an hour post boot/registration. I am surprised to see worse performance on subsequent logins, no other changes made to the image/VDA spec from previous test results. To be continued....
r
Doesn't really make sense that its related.