World of EUC

Interesting... one of my team are looking at a very similar issue. We suspected WEM at first but that been disabled and the issue persists, the VDA is Win10 multisession running Azure. Tried most things you have but not running via RDP. I'll get them to check to see if that changes anything... will keep this updated with anything of interest, cheers

Is hardware graphics acceleration disabled in Excel?

In our case we have disabled hardware acceleration. This customer doesn't use WEM either

And my deployment is M365/Proplus (MCS in Azure)

Disable Ica hooking for excel? If it works in rdp then chances are it’s a hook issue (maybe)

<@UKJTJD4KA> - i tried doing that by adding it to the list of processes at

HKLM\SYSTEM\CurrentControlSet\services\CtxUvi\UviprocessExcludes

Is there anywhere else i should add it as well or should that be sufficient?

<https://support.citrix.com/article/CTX107825|https://support.citrix.com/article/CTX107825> check that one out 

Unlike ExcludedImageNames, UviProcessExcludes requires a restart of the VDA, check loaded DLLs with Process Explorer

I saw that article. So as long as i'm understanding everything right i should only have to configure the UviProcessExcludes since i am running newer version than 7.9. I'll try to check loaded DLL's to see if i can see anything

I checked with Process Explorer and i can't find any loaded Citrix DLL's under Excel

what profile solution are you using and does logging in via RDP bypass that?

Good old excel performance issues!! what's the performance look like on your ESXI hosts any balooning/swapping ? cpu ready times ok ? datastore R/W latency's ok ?  i been looking at excel performance issues as well in a new environment i tried enabling multi threaded calculations / disabled hardware graphics acceleration in Excel but didn't make a difference.  In the end i found it  was because the ESXI hosts hadn't been set to high performance power mode (They was set to mixed mode)  and C states hadn't been disabled. Once the hosts was set to high performance mode and C states disabled the excel performance improved by about 30%

Not sure if it is still an issue, but I recall Excel running like a dog in a Xenapp environment a few years ago that was caused by clipboard mapping being enabled. Once I turned off clipboard mapping it ran fine.

<@ULLAD12P2> - Customer is running UPM for profile solution.  So RDP should not bypass it

<@ULM6WD9LY> - Performance looks good. The sluggishness is always in place even if we're only one test user logged onto the terminal server. We're running Nutanix with ballooning

<@U01NDQN63GX> - thanks for the information, i'll try disabling clipboard mapping to see if that has any effect

image.png

I can only see 2 procs under advanced &gt; formulas. The VM has 8 vCPU so might be some weird abstraction stuff on Azure... we are mostly seeing it on chunky Excel worksbook with lots of formulas etc... the argument is it runs better locally on low spec machines....

Did you try to see if RDP makes any difference in terms of performance?

Sorry.. meant to feedback... going via RDP we have the same issue so not related to app hooks etc

By the way, we tried disabling client clipboard redirection but that did not have any effect on the issue

So, from what I can see, Excel running on virtual environments gets capped at 2 processors  for multithreaded calculations... Ive checked a few Azure and vSphere builds and 2 is all thats detected when there are more available, this screengrab should show 4:

This is the cause of the issue I have... the folks complaining are finance people new to Citrix and I can tell their smaller laptops are working better due to more available threads with large spreadsheets... looking at the CPU usage between the endpoint and Citrix server confirms this... when grafting in Citrix its maxing at 15% CPU... I'll ask MS about this and feedback, cheers

If the spreadsheet has a large number of formulae have you tried turning off automatic recalculation? I have seen this make a huge difference entering data on complex spreadsheets.

Hi, guys, small update. Found this article and tried deactivating the three policies as mentioned in this article. So far it looks like it actually has a positive effect from the testing i've done - it reduces the sluggishness of excel quite a bit but user's havent had enough time to test properly yet - <https://support.citrix.com/article/CTX310699>
Consider trying it out in your environment <@UR99U4W0Y> and see if it has a positive effect

PVS I don’t think would work since not even supported on Azure today. 

I thought perhaps with Azure stack being on LAN pxe would not be an issue. How about MCS? Currently there is no connector into Azure Stack, would it be possible to use connection ARM and through that use Azure Stack as an Azure zone?

You might want to try ARM connection. I don’t see anything support wise for Citrix on Azure Stack. I don’t think anything is supported at this time.

Thats the same thing I am seeing. Want to give ppl heads up before they commit to this kind setup

<@UK6S90JDR> Azure PVS is currently in beta, so it's coming, but not sure how it all works.  I imagine that you get a big saving on storage IOPs with cache to RAM purely inside Azure

Still Azure not entirely same as AzureStack HCI. Again I don’t see anywhere it being supported by Citrix at this time. 

Thanks friends. The rep confirmed today it is not supported for CVAD.

CPU spikes protection enabled, #CPUs set to number of cores on the host. RAM disabled (we have enough RAM on the box, and didn't want to page).

<@ULHA2PS2H> how much RAM? We're using WEM for memory management, but I don't think we need to. We seem to have a lot of RAM assigned to our XenApp Servers

CPU optimization has that newer auto setting as well

<@ULL7R5KT6> / <@U014GAQNQJK> - we did use it for RAM management, and had a couple odd problems. We used ControlUp to determine that we did not need the RAM management. Our hosts were 10 x vCPU and 60 GB RAM for ~25 users. When we investigated our actual RAM usage, we found that we were not hitting even 80% utilization. WEM RAM management pages the memory to disk, which we found was of little benefit to us, and turned it off.

Back to the odd problems for a moment - we noticed that Outlook specifically does *not* like to have it's RAM paged, and even when we had memory management enabled we did have to exclude Outlook. I believe there may have been one or two other apps that were also excluded. By odd behavior I mean random crashing that started after we enabled RAM management.

I've been rethinking our setup which we carried over as legacy settings. We have 8vcpu and 32GB RAM. We typically get 16-18 users. We rarely get more than 5 machines that exceed 50% RAM utilization. I think I want to scrap local page files (still using those). Remove WEM memory management and increase the PVS cache. Current set to 4GB with overflow to disk. The only thing is whether to increase our PVS cache drive. We're currently at 30GB and as long as users don't have huge profiles it seems enough. Dumb question I didn't think about until just now, if we migrated to FSLogix for profiles, then writing the user profile to the local server wouldn't happen anymore and not grow the vdiskdif.vhdx as mich, right?

I like the way you think :smiley: We went through that same process a couple years ago, and ended up drastically reducing our page file size (I think it's like 4 GB, just for *something*). It was ControlUp that really showed us how much paging we were doing. We increased the PVS write cache size to ~12 GB or so, with overflow to disk, and also migrated from UPM to FSLogix. The FSLogix profile exclusions *ARE* written to local disk, so keep that in mind. Everything else is redirected into the profile. The recommendation is to redirect as little as possible, so that should not be a problem for your local storage / write cache. YMMV, and test test test!

<@UK6S90JDR> I'm using the Auto spike protection in QA and UAT. I've had it on for a while. I need to move the settings to production after we get through our QA test for the current patch cycle

<@ULHA2PS2H> We also have had to exclude Outlook from memory management. Along with a host of agents (AV, auditing, etc) that created lots of event log noise due to WEM trying to do what it does.

CPU optimization to auto, RAM enabled, 8vCPU 32GB RAM for ~15 users (16GB page file), nothing set regarding Outlook right now :slightly_smiling_face:. We tried without and it was a nightmare !

<@U014GAQNQJK> have done that with agents. What was Outlook doing?

<@UK6S90JDR> I don't recall now. It may have been related to some plug-in we were using. We can test again. We've upgraded office a couple of times since then

If you have enough memory don’t enable RAM optimization! It will fill up the pagefile, keep that in mind. Better buy some more RAM. I use the Auto config for the CPU optimization.

To keep things simple, I‘d recommend not to use SAML for internal authentication. We did a separate store with passthrough authentication enabled for internal connections. Also keep in mind if for any reasons AzureAD is not available for SAML nobody can log in.

and we've seen enough of those outages lately.....

using the new vda command line helper tool for the arguments

<@ULL6F7DRA> did you ever get this working?

If I remember correctly the command line parameter changed

it did, it’s now under /includeadditional

That's right, this bug should be already fixed

I’m not sure why it doesn’t install the driver then

XenDesktopSetup:Component ‘Citrix MCS IODriver’ and dependencies instantiated.

Are you using the standalone VDA installer, or the one from the ISO?

Following this, as I'm pretty sure I'm seeing this too.

I opened a case, I have to change all my automation code to move to the iso installer so it’s a bit of a pain

What do you mean with "Cortana search"?

As far as I can see Cortana is Microsofts digital assistant, like Siri and Google Assistant.

What is different with a Cortana search compared to a non-cortana search?

You're right :wink: The difference I meant are the search results using the start menu. If you're using the search bar, files are not found. If you're using the Windows Explorer, files are found immediately. Same behaviour in different environments for Multi OS. Desktop OS is returning the search. Maybe some AppX Packages that are missing

OK, you are probably running into the known issue with Windows Search in 2019, which requires a workaround to make the search feature work.

We're using FsLogix, but same behaviour also with my Admin Account using a local profile and being the first and only one connected to the server. Restarting the WSearch Service has no effect and I did not found any errors related to the service

Yeah the issue with windows search is not related to FSLogix, it's a Windows issue.

If I remember correctly there are some limitations with the Windows Search feature in a server OS in terms of what is indexed.

I think it only indexes the users profile information (start menu, appdata, documents etc) and an OST file if present

I am looking for some official MS statement on the matter

That sounds reasonable. It's kind of perplex finding the files using windows explorer but not when you're using the search box

I have only been able to find this official documentation of Windows Search:
<https://docs.microsoft.com/en-us/windows/win32/search/-search-3x-wds-included-in-index>

However it hasn't been updated for almost 3 years, so it's probably no longer relevant

Probably not for Windows Server 2019. Thank you!

did you disable cortana? there’s a ton of GPO that controls search results int he start menu

but it also depends on what you are indexing, as Kasper said.

I think by default it will index all user profiles though, it was doing it for me and I had to disable it because a lot of our profile content is in onedrive and it kills perfornace to index that

how fast is explorer finding them? Because if it’s instant then it sounds like they are being indexed and there is another setting limiting your search results in the start menu

Sorry for the late response. Cortana is already disabled by GPO. Explorer is finding the files immediately.

Trend micro is the devil for grey screens - behavioural monitoring is typically the cause (Sophos does it too) - those keys are also narky 

I don't have Trend Micro installed, just Defender

I will have a case open for this one, I'll report my findings here.

These reg keys are still recommended for GPU so in theory they should work no problem. If they are no longer supported someone needs to update the documentation!
<https://docs.citrix.com/en-us/citrix-virtual-apps-desktops/graphics/hdx-3d-pro/gpu-acceleration-server.html>

cc <@ULGKFDUVD>

Nvidia Licenser Server - Config Page Empty // Hi Guys, the latest nvidia license server only shows an blank page on config site. any idea how to fix it?

you’d be better off with basic availability groups

mirroring isn’t a thing in newer sql versions

This is to be set up in a production environment, which is running 2014 SQL,  so i don't think it's available?

SQL Mirroring is pretty simple to setup. The  process i did when i last did a migration from single sql instance to a mirror was take a full backup of the Citrix databases and restore them on the mirror with the option restore with no recovery, run the sql mirrroring security wizard on each DB, connect to the principal/mirror/witness instances , add sql service account, use high safety with automatic failover (syncronous commit), create SQL logins for controllers on mirror and make sure they are mapped to Citrix DBs and assigned correct DB roles thats it from sql point of view. Last change would be to add the failoverpartner to the FMA services DB connection strings you will have to null them out first

That article you posted is a waste of time and wont work as its missing loads of the new FMA services your best off using <https://www.citrix.com/blogs/2014/02/28/whats-the-right-order-to-update-xendesktop-7-1-services-for-sql-mirroring/|https://www.citrix.com/blogs/2014/02/28/whats-the-right-order-to-update-xendesktop-7-1-services-for-sql-mirroring/> but even that looks like it is missing some of the new stuff tbh i always use the scripts or the manual methods to null out/add new db strings from here <https://docs.citrix.com/en-us/advanced-concepts/implementation-guides/database-connection-strings.html|https://docs.citrix.com/en-us/advanced-concepts/implementation-guides/database-connection-strings.html> and never had issues

Thanks! Might be a silly question but when following the "Update strings manually" method in the article. Can i setup the mirroring in the sql mirroring security wizard before removing citrix db connections, or should i do it after i've removed the connections? or does it not matter? :slightly_smiling_face:

setup the mirroring in the security wizard before nulling out the strings and adding the new db strings with the failover partner:+1:

This grey screen I see all the time on Slack. I had it once, but it was because of CTXhooks missing

I have seen this many times. In most cases it was the antivirus software

Trend is implicated in our problems (they started about December last year)

Any hints in eventlog? I had EventID 1005 from Source CtxUvi

Beware of Trend. It makes things very difficult in my experience

Yep that's the same event ID I get. Behaviour Monitoring in Trend apparently a big cause

No AV issues on my side, had to reinstall VDA after using the Cleanup utility. 2103 VDA installer is botched anyway, gives error messages for App-V personalization even if not installed. See Known Issues

Yes indeed <@U01GWFT2BDJ> ping <@ULLH59VQF>

<https://discussions.citrix.com/topic/412451-just-grey-screen-instead-of-published-desktop/>

we don't have the registry keys, so it has to be related to Defender and Symantec. Triple checking AV exclusions. Applies to many CVAD builds, so not related to 2103 as first expected

We have the same issue in our own environment since today! Windows Defender with standard exclusions. CVAD 2103 on Server 2019.

Citrix need to get hold of this. I've had a case open for months and I'm seeing no progress from them

VDA Cleanup and reinstall of VDA fixed it…

WOW so you had the exclusion in place, and still grey screen? Then VDA cleanup and re-install fixed it? Almost like defender is hijacking something with graphics.

Cleanup and removal didn’t work for me. 

Unlikely to work here either, as our images are built fresh every 30 days, and we have been having this issue since December

It seems to be related to WEM Agent Auto Update which fails, and need reboot for roll back (NP).

Sounds like there's a million different reasons then.....we don't use WEM

Wem is one of these things I keep trying to use but it keeps disappointing. 

Do you guys usually deploy WEM when spinning up a completely new environment? The minority of our customers use it and i usually avoid it. Sometimes I wonder if I miss something. 

Only use it very rarely and then generally just for the performance optimizations

try using the VDAWorkstationCoreSetup.exe installer instead? Then you wont need the /remotePC switch which auto excludes those UPM components? You've now made me realise i have a problem on an existing engagement that i need to go check - bugger <https://docs.citrix.com/en-us/citrix-virtual-apps-desktops-service/install-configure/install-command.html>

I’ve seen issues with the standalone as well, using full ISO instead.

<@UKJTJD4KA> the /includeadditional parameter is not supported with the Core setup.

We deploy without /remotepc switch on physical machines.

and are you able to use it via RemotePC with logon metrics ? I'm assuming you exclude everything else except CPM and CPM WMI?

Yes works via RemotePC. I have not checked logon metrics tbh. And no I dont really exclude much. Maybe I should :D

interesting, I will need to test if I can still trim the unnecessary fat

ok so when you say logon metrics, are we talking session info in Director that shows logon speed?

the / remotepc switch caused some issues around disconnected sessions. Had to stop using it and put in a registry change. Otherwise sessions did not go into disconnected state but rather into console and machines became unusable to other ppl. The only issue I found with this deployment has been around some GPU's.

the command line install for the standalone installer is really bad, it won’t do mcsio either

I'm having that issue also <@U0125S0736V>, Citrix session stays active forever even if the user disconnect them or the timeout are reached. Are you seeing the same bug?

Can you share that registry change with us please ?

<@ULDHJB8BB> HKLM\Software\Citrix\Portica\ForceDisableRemotePC, DWORD, set to 1 
HKLM\Software\WOW6432\Citrix\Portica\ForceDisableRemotePC, DWORD, set to 1

the issue I saw is when HDX user disconnects, the session does not go into disconnected state but rather switches into the "console" if you look at session in CVAD.

I guess the reason for it, is the remotepc is meant for one user using the same machine. not random ppl using pools of devices.

yeah well I use it with students in a class so I'm using single session os catalog with random user

so now I'm wondering if the reg value has other adverse impact ?

and what if I modify the install parameters not to include /remotepc, will the key still be required ?

I need to test some more.

Same issue reported here
<https://discussions.citrix.com/topic/410389-sessions-to-physical-pcs-do-not-go-into-disconnected-state/#comment-2070410>

Yes I believe the key is still needed. I haven’t seen any issues besides some related to gpus but for those there is a reg fix too :joy:

Are you having those gpu issues tho? Because the fix involves switching to Citrix gpu driver. Would not be my first preference 

The issues I’m talking about are grey or black screen on logon or being unable to launch desktop completely 

well not yet but I have not tested the ForceDisableRemotePC enough yet

for you're GPU issue, are you using the following GPOs?

I actually have not set any GPU policies at all for those machines

*There may be cases where using the Citrix IDD adapter (generic GPU support) is preferred or required, over the GPU vendor’s.* Mainly as a troubleshooting step or a workaround to issues with GPU vendor adapters. For these cases the registry key may be configured with the single value as follows:
HKLM\Software\Citrix\Graphics\AdapterMerits
DWORD CitrixIDD = 3

We have a customer using nomacs, and they seem to be happy with it. I (think) it supports HEIC files.<https://nomacs.org/| > <https://nomacs.org/>

Thank you, looks good. I'll check it out on a test server

I have used Irfanview a couple of times. Had issues with some .jpg files, that photo viewer couldn't open - <https://www.irfanview.com/>

Irfanview is nice but not free for commercial use

USAGE AGREEMENT
IrfanView is provided as freeware, but only for private, non-commercial use (that means at home).
IrfanView is free for educational use (schools, universities, museums and libraries) and for use in charity or humanitarian organisations.
If you intend to use IrfanView at your place of business or for commercial purposes, please register and purchase it.

<https://www.irfanview.com/main_what_is_engl.htm#:~:text=IrfanView%20is%20provided%20as%20freeware,in%20charity%20or%20humanitarian%20organisations.|What is IrfanView>

We used Image Glass, very lightweight, open source image viewer

DesktopStudio_ErrorId : ProvisioningTaskError
ErrorCategory : NotSpecified
ErrorID : FailedToCreateImagePreparationVm
TaskErrorInformation : Terminated
InternalErrorMessage : The object has already been deleted or has not been completely created; The object has already been deleted or has not been completely created

It failed to create the temp VM mandatory to prep the image.
Check on ESX there is sufficient resource to run the VM.

Another bug: Filtering Citrix Cloud Policies with Tag does not work as expected. I've seen one case where no policies applies at all and another one where even non-tagged delivery group gets the policy applied.

I don't know german, but often the plug-in will stop working if user has been signed out of Teams or never has been signed into.

Also make sure you are roaming the proper folders, if you don't then they can cause these issues

The Plugin always work at the first time, when i plan the second meeting it stops

Anyone ever done deployment of Office365 with the E1 (web only) license in Citrix at all? Any gotchas if they have (besides the obvious functionality gap) [cross-posted]

What I would do:
1. Add the new site/DDCs to your existing store. This will allow you to access resources from both sites
2. Build new SF servers on Win2019. Do not create a new SF deployment
3. Add the new SF servers to your existing SF deployment. This will sync your already existing store to those new servers
4. Update your load balancing config to include the new servers and remove the old ones
5. Remove the 2012R2 SF servers from the SF deployment

great thanks, looks clean and painless. :slightly_smiling_face:

i forgot to mention but i don't know if it's important in this case: old site is based on xenapp server 2008r2 manual provisioning, new one will have mcs with 2019.
The only server that the two sites will share will be the license server that I will only update to 1912.

You will need to upgrade the license server to versio 1912 before the 1912 site can use it, though

btw, don't forget to create appropriate backups/snapshots before making any changes, and do the proper testing before decommissioning anything :wink:

I also always do it as described above. Per site aggregation you can even use the same names and migrate the users "unnoticed".

Hi Manuel, users need to see the icons of the new desktop with a different name than the current one. If I understand what you meant.

it is not supported/recommended to mix storefront OS versions in the same group.  Instead create a new group and subscription sync between them.  if you use Workspace app self service (account configured on device for shortcuts or for the self service portion), you'll need to remove the old and re-add the new store as the store ID is different.

Bild von iOS.jpg

The only way I know to fix this is to log in and back out, even exiting Teams doesn't seem to work. Once updated to latest VDA, latest CWA and latest Teams though, it seems to have stopped happening so much

Hi James. For some cases even a reboot of the VDA is necessary. I didn’t had the time yet to analyze this on my own but hoped to get some answers before jumping into it...  When you are saying stopped happening so much. How many % of the users are reporting this issue?

I used to see this personally very regularly, as I use several virtual desktops at once. We also had about 5% of our user base reporting at least one instance that we know of. Since I upgraded to the latest releases this month, I must say I haven't noticed it as much personally. I will check with the ops guys tomorrow to see if the incident reporting has changed at all

I do, however, notice a black square or even a video "ghost" welding itself across the Teams screen very regularly now, when I've been in a video call

Sounds promising. I Just checked the 2103 release notes and the only thing mentioned is: „Teams redirection doesn’t support more than 31 concurrent user sessions. [HDX-27083]“

Wow, didn't know that was a limitation :slightly_smiling_face: the problems seem to come and go in "waves" which makes me think that things are breaking and then being fixed. We rebuild fresh with the newest of everything every month, although updating unmanaged devices is tricky. This makes me think that updating is fixing some of the issues (but then, as I said, new ones pop along)

Guess the black square wouldn’t be a big problem. They just want to get audio/video working for the remote users. Have two other customers where Teams optimization is running without any hassle but I don’t have any control over the image life cycle in this environment. Guess it’s not the best but can not say for sure... After all speaking and chatting with the op-guys it seems they can rule out the endpoints. 

Hey <@UL5TCSW3D> any update from the ops guys?

Incidence reporting is down slightly, so it seemingly makes *some* difference......

Seen the same : most of the time linked with websocketagent.exe (or websocketservice.exe) crash/hang. Don't have events about these ? If so, a fix is included in CU3

As far as I have been told no events about crashing websocket stuff

You can create a new catalog based on a new VM snap with GPU but you can also add the GPU to the VM (manually or scripted) as long as the NVIDIA drivers are installed in the guest OS.

VM configuration is only cloned during initial roll-out. So as Kees said there are two options:
• Change the current VMs to the new GPU config
• Delete all VMs and create a new Machine Catalog

That's what I was thinking, I check the ProvScheme but I cannot find any GPU ref (like it's possible for CPU or RAM).
Thanks guys for yours answers.

To change the GPU profile, you need to edith VM Hardware on the cloned VMs

Any news about WorkspaceApp for Win 2104? 

Same 1912 Cu1 version of DC as existing DCs. Nothing else installed on the new DC except Studio and Delivery Controller.

Only found that googling: <https://support.citrix.com/article/CTX215625>

Anyone else running TeamViewer on non-persistant (MCS on WS2016) ? Randomly GUI closes unexpectedly (Exception Code: 0xC0000005) when accessing options or starting a remote session with customer. Restarting TeamViewer service fix the issue. Already tried to disable DEP/SEHOP without success. The only reply I got from support is to uninstall/resintall…

If I would have the MAX values, I could then express the RAM usage in percent for example, and alert myself, if RAM cache usage is &gt;95% for 15 minutes.

Why don't you use Citrix PowerShell cmdlets from the VDA itself? This way, you could talk to the broker and get the values directly from your VDA.

Interesting. But that way I would need to know, which Machine Catalog I (the VDA) belong to, before asking the broker, wouldn't I?

You can find it easily: (Get-Brokermachine -MachineName YourDomain\$env:computername).CatalogName

So another question that the VDA doesn't know about, that I would have to ask from the VDA against the broker

Currently I'm searching for the RAM and VHDX max size, haven't found them on the Broker, yet.

I think you have the info in Get-ProvScheme

Find the CatalogName from $catalogname = (Get-Brokermachine -MachineName YourDomain\$env:computername).CatalogName
Then the ProvSchemeID with $ProvSchemeID = (Get-BrokerCatalog -Name $CatalogName).ProvisioningSchemeUID
And Finally Get-ProvScheme -ID $ProvSchemeID

Get-ProvScheme -ProvisioningSchemeName "name"

Well we know it with `(Get-Brokermachine -MachineName YourDomain\$env:computername).CatalogName` don't we?

If it was created through the GUI, yes it should be the same as your Catalog

I have, They will need a reboot in my experince

excellent. But can restore be done while they running? As in can I do to it without taking down entire site? Something did not work when adding a DDC and now prod DDC's give errors in Studio

Well, I am not 100% to be honest. If you think about it. The DDC are what is talking to the DB. The VDA are getting the brokering infromation from the DDC. Depending on if you have LHC or connection leasing setup. I dont see any outages occuring for users.

1. Restore DB
2. Reboot once DDC at a time
3. Verify DB connectivlity from DDC.

This is a high level summary without much thought going into the setup

My thoughts as well. Then again, I was not expecting problems with adding a DC either :smile:

What is the issue you are face with? I add DDC about once ever 4-5 years. I do it on OS upgrades.

Tried to add DC, ran a script to add it on SQL and after that the DC and exsting DC's are erroring on Zones and Controllers in Studio. with     DesktopStudio_ErrorId : DatabaseError
    Exception : System.NullReferenceException: Object reference not set to an instance of an object.

working with DBA to see what tables changed.

The DDC was added on permissions for the DB correct? Servername$

the problem now is not even so much the new DC but fixing existing ones

i think when the script ran several tables did not get updated properly

but I dont know how easy it will be to track those changes as opposed to just restore the site db

I would restore the DB. Then get it back on track. Like you are doing.

But there are some bad asses on world of EUC. Maybe they have more experince? I know of a couple that can add to this. <@ULL6F7DRA> <@UMB4XQE79> <@UL5TCSW3D> <@UK6S90JDR> <@UKJTJD4KA> <@UR9T47CRM>, then DJ which is awesome dude <@ULKDZDN86>. This is just a couple that may have a better opinion than me. I am still a Jr compared to them.

I have tested restoring DB's / moving cnames in DR tests in the past. But this is live environment. want to make sure I am not missing something

Are things working with the errors now? Do you have 24/7 users using it?

Yea things are working. The issues are around Delegated Admins functions - Zones and controllers configs. Nothing huge. I will probably schedule an outage just in case and have DBAs restore. Trying to understand what expectations to set for the steakholders.

Personally I would roll back as <@ULLH5JGFR> suggests. Which CVAD versions are we talking here?

will update eventually but would like to fix this first. I think the DC addition added partially populated fields to the DB

But aren't you updating from one version to another?

OK. I haven't read the entire thread, but what is the reason for the DB restore then?