A Citrix DaaS Azure customer on Windows 11 24H2 Mu...
# citrix-cloud
m
A Citrix DaaS Azure customer on Windows 11 24H2 Multi User requests a lot more HDD/SSD performance for his workload. Currently they use Azure ephemeral OS disks and I already tried to set Premium SSDs instead. But the performance of both is the same (screenshot) and far from what they request. Ideally they would like to see local NVMe like performance, something like 2000 MB/s RW +. But I obviously miss the point in DaaS to set something else besides Azure ephemeral OS disk or Premium SSD. Where in DaaS could I get way faster SSDs? Do I solve that through a machine profile?
n
If you look in Azure, you can select different performance tiers for your disks under Size + performance. Cost quickly skyrockets, though.
m
Yes, I know how to set this on Azure, but not with Citrix DaaS MCS hosted on Azure.
n
Gotcha. I thought maybe a Template Spec (used for the machine profile) would show that info, but I'm not seeing it in mine.
m
Found a post from @James Kindon here, that maybe the Template Spec should be the solution. https://community.citrix.com/forums/topic/249890-citrix-daas-mcs-azure-vms-premium-ssd-storage-with-performance-level-p50/
n
I just don't see it in mine. Maybe @Rob Zylowski knows?
j
I rarely did this in azure, but MCSIO could be a good option here for performance - use a big RAM cache
m
I have no idea what the translation is, but let Google do its magic: This is for an office of CAD engineeres. Besides AutoCAD they need to process "point clouds" from 3D Laser scans and this is ultra I/O intensive. AFAIK MCSIO only has limited space and it gets slow as soon as it becomes full (?!). Therefore I thought a faster SSD by itself might be better, than a "limited" MCSIO. Just thinking out loud.
j
You can customise the cache size for MCSIO and then overflow it to high speed disk - there shouldn’t be a speed delay - you are trickling out to disk and operating primarily on memory (as much memory as you assign)… just a theory worth maybe trying But back to your original question - I can’t recall exactly how you do this - I feel like it was maybe a custom prov scheme option but it’s been a while… @Yuhua Lu will know
r
This is why we moved out of Azure. Ass disk performance.
"Ideally they would like to see local NVMe like performance, something like 2000 MB/s RW +. " <---- for $5000/month
n
Yeah, we haven't been a fan of that aspect. The v6 stuff is coming Q4 2024ish, and that brings NVMe disks.
Or so I was told.
r
brings NVMe disks.... affordably?
n
I think you and I both know the answer to that, haha
r
😂
m
(Im OOF for the rest of the day, sorry)
r
I thought Ephemeral disk were the fastest you coudl get since they are on server and even the ultra disks are network based
👍 1
When siting in on Microsoft meeting where disk performance is needed they talk about running software raid to get better performance
You cant with mcs. You would have to create your own vdas
j
With MCS you could provision them, and then up the OS disk performance tier to whatever you want post provision? You can do this without a reboot I believe because you are just playing with the performance Tier vs the Disk Sku. (I can make a P10 run with a P20 or P50 performance profile) I used to have a runbook in Azure to change the Identity Disk SKU's back in the day - could do the same thing. Or you could persist the OS disk, change the entire Sku and get the bigger disk, but then you would have to change it each provision and it would require a reboot so the above path would be better I wonder if your Ephemeral Performance issues are because you have a Sku where the Ephemeral Disk is stored on the Temp disk rather than Host Cache (your instance doesn't have host cache)
Oldie but a goodie - you would need to adapt the logic for performance tier changes - but that's the framework https://jkindon.com/enhancing-citrix-mcs-and-microsoft-azure-part-1-identity-disk-cost-optimization/
Or just make the gold image use a bigger disk to start with, MCS calls from the snapshot or OS disk base size
m
up the OS disk performance tier
So I would have to create a MCS Catalog with Premium SSD instead of ephemeral first. And because the OS Disk gets deleted each shutdown, I could maybe up that to a higher P level, through a Runbook that runs regularly. Interesting idea. Would be interesting, if this is possible. Sad it's no native MCS feature.
(I can make a P10 run with a P20 or P50 performance profile)
Yes I know, that's part of the reason I asking these questions.
Or you could persist the OS disk
Seems sad to up the costs through unnecessary persistent disks?
a Sku where the Ephemeral Disk is stored on the Temp disk rather than Host Cache (your instance doesn't have host cache)
When I create a MCS Catalog in DaaS, there is a nice helpful tool-tip, which summarizes the SKU, and it states the SKU has a cache and temp disk, and 127GB OS disk. Should fit the cache? Additionally, there is a PS script to see what ephemeral disk placement is possible: https://learn.microsoft.com/en-us/azure/virtual-machines/ephemeral-os-disks-faq And for our SKU it also states, that cache and temp are supported. This brings me to two questions: How do I see, where my ephemeral disk resides now and if I'm able with MCS to determine where it should be. I have the feeling, that this could be a solution. But no idea where to start.
Okay first question was easy, that's just visible in the Azure portal:
And this is our MCS catalog for the Office workers SKU Standard_NV12ads_A10_v5:
j
So a few points here 1. Yes, persisting an OS disk will add cost - you would want to RI it 2. Yes, the framework works, I used it for many different things, you can set it to just run every x hours and make the changes - good as gold 3. sounds like a decent MCS feature request if it’s not doable today - let me ping and ask at the source 4. I noticed MS were cracking down on ephemeral disk glory - some of the latter instances removed local temp disk options altogether - nothing is free forever :-) 5. Weird on the host cache miss - the docs have changed, it used to specifically call out host cache availability
m
Just FYI I just benchmarked the ephemeral disk from the NC16as_T4_v3 Temp Disk VS. the NV12ads_A10_v5 Cache Disk: EDIT: > I have the feeling, that this could be a solution. But no idea where to start. Obviously this is not the solution, if ephemeral is both slow on Cache and Temp disk placement..
Packer is ready. I now create a Machine Catalog with MCSIO and 36GB RAM Cache und 300GB Write Cache Disk on Premium SSD. Let's see what happens.
So the numbers are nice, I would say:
👀 1
j
Nothing beats memory
m
I'll now have to wait for the customer to test, that will take a few days, until I get feedback.
j
Depends on your overflow, but you might not even need the premium disk - idea is that you can offload to less performant disk (cheaper) but gotta understand what’s coming out from ram
m
I think I create a thread in #CK6TCV3JN what's the best way to monitor MCSIO RAM usage, so I can properly interpret and size, according to the customers needs.
👍 1
j
Pvs has counters for this in perfmon. MCSIO is built on the same tech, wonder if it has the same counters
n
Those IOPS figures are making me reconsider Ephemeral discs for MCSIO
m
@Neil Spellings If you are interested in IOPS instead of MB/s I just did another test pattern with the same software today: Maybe there are any other tests you are interested in? If they are easy to perform, I can maybe grab the numbers.
r
IMHO better to use IOmeter with a VDI profile @JimMoyle created a while ago than using Crystal Disk Mark
💯 1
m
Can take a look on Monday. I guess you mean this one? http://www.iometer.org/ Do you know where to get the profile?
j
Profile spec for iometer is in my ephemeral post from way back, wasn’t a download but easy to create
m
Sorry for the late reply, but I now have the values as promised @Ruben Spruijt @James Kindon @Neil Spellings
Ephemeral on SKU NC16as_T4_v3
MCSIO with 36 GB RAM on SKU NC16as_T4_v3
j
Those are some very cool improvements…cpu gets hit about double but everything else is ridiculously better on the MCSIO test
m
In theory, yes! Sadly the customer hasn't tested anything, yet.
n
Are there any counters to help determine how much write cache I'd likely need to add in terms of RAM prior to switching to MCSIO?
m
Prior? Absolutely no idea. There are MCSIO Perfmon counters, that measure the actual use of the RAM cache, but well, it has to be there, to be measured.
Here is the Graph from during the Crystal Disk Mark tests, I monitor that with Zabbix:
This is from the IOMeter Tests: (I don't know, why this one didn't went down after the tests)
This are all the MCSIO values I monitor:
n
Prior would be useful so I can mop up anything doing horrible things prior to switching. I already know Tanium constantly writes to its local DB so I'd need to move that out of the write cache somehow
j
Only if you are worried about it filling up and then overflowing… might not be an issue
Could get cheeky with a redirection rule via appmasking if you needed to move things around I guess
m
I would be so bold as to say the following, at least that is how I understand MCSIO: There is no scenario in which activating it makes something worse. It can only get better. As long as it writes to the RAM, it is faster than before. If the RAM is full, it writes to the VHDX instead and if this is on the same storage where the VDA is normally located, it is as fast as if MCSIO were inactive. So it can only be better? That means that what can happen is that the effect is smaller than expected, and you can keep an eye on that with the PerfMon values.
n
I do see your point...but there's also a cost element. Unless you can do MCSIO+Ephemeral (I don't think you can mix these two), you need to move your main discs to SSD (which should be slower) and those cost money. So performance hasn't suffered, but I'm paying more for larger RAM and my storage. MCSIO used to allow cheap-as-chips storage because you'd be reliant on the RAM cache for most of the IO.
Tanium is literally recording every file and registry access all the users on my VM are making to its local file-based DB. it's the top I/O consumer by far and I know would blow my write cache away very quickly
n
You had to say the T word
n
Sorry, I know it's a trigger for many
j
It still does allow cheap storage (well, it’s azure so it’s relative when I say cheap), that’s one of the biggest drivers for it in public cloud - use rubbish disk but hide it with RAM. If it’s not cost beneficial to do it, then it is what it is, this scenario is performance driven though so cost is going to be less a factor. Won’t make sense for every scenario
👍 1
y
Sorry for coming late to discussion. After my laptop got reset, I was kicked out of EUC slack instance, didn't notice till yesterday.
I am glad that you folks figured it out with moving forward with MCSIO. Thank you @James Kindon, as usual you get to right solution quickly 🙂
🍺 1
Couple of things 1. you can define OS and WBC disk tier at catalog creation and later edit 2. you can also change memory size and WBC disk size after creation so you can choose optimal size based on your test 3. currently all these change only affect new VM. So probably best you have testing catalogs and utilize new image management feature (decoupling), that way creating new catalog and one new VM will only take 2 minutes
👍 1