World of EUC

I would use get-brokerdesktop with most of the same parameters as you are using on the group. e.g. get-brokerdesktop -desktopgroupname "2016 PVS DG"  -desktopkind "shared", etc.  Then you will get DNSname or machinename to chose from.

If you run get-brokerdesktop -desktopgroupname you do not have the option for machine name or DNSname

I'm not sure what you mean by option... They are  member properties of the results. Note, I am talking about get-brokerdesktop and not get-brokerdesktopgroup.

PS&gt;get-brokermachine -DesktopKind Shared | get-member  | select-string Name

string AssignedClientName {get;set;}
string[] AssociatedUserFullNames {get;set;}
string[] AssociatedUserNames {get;set;}
string BrowserName {get;set;}
string CatalogName {get;set;}
string ControllerDNSName {get;set;}
string DesktopGroupName {get;set;}
*string DNSName {get;set;}*
string HostedMachineName {get;set;}
string HostingServerName {get;set;}
string HypervisorConnectionName {get;set;}
*string MachineName {get;set;}*
string PublishedName {get;set;}
string SessionClientName {get;set;}
string SessionConnectedViaHostName {get;set;}
string SessionLaunchedViaHostName {get;set;}
string SessionUserName {get;set;}
string ZoneName {get;set;}

PS&gt;get-brokerdesktop -desktopkind Shared | get-member *Name*


   TypeName: Citrix.Broker.Admin.SDK.Desktop

Name                     MemberType Definition
----                     ---------- ----------
...
*DNSName                  Property   string DNSName {get;set;}*
...
*MachineName              Property   string MachineName {get;set;}*
...

That is a good one, and this is another one. <https://github.com/JamesKindon/Citrix/tree/master/Migration%20Scripts>

No problem, Its alot of information I know. But once you work through it. It will all make sense

Yes I'm sure it will! :slightly_smiling_face:

Is Citrix stance regarding Citrix policies to not export/import them to new Site?  Supposedly that can carrry over site specific information and they might not apply right?  I've always 'heard' this, but never saw it not work.

I read that too, but yes I guess it is specific site related info that can't be migrated. I used this tool <http://citrixtips.com/tools/citrix-policy-migration-tool/> to migrate my Citrix policies and it worked perfectly.

I'm struggling to get Windows media pack (required for VDA) in Windows 10 2009 (h2) N. Does anyone have something working please?

Let me see if I can find and let you know. Hadn't tried that yet.

VDA Registration - All check in this group passed.

One application path in Session launch missing. Looks like FileZilla didn't get installed, but nothing really related to the issue.

I use AD for DDC push. But I also ensured all DDCs are in the ListOfDDC registry key as I found that to be a common cause for this.

Are you using Citrix Policy or Register to push ListofDDC from AD

Citrix Policy in AD Group Policy assigned to a Site Policy.
____

Error from the VDA side in event viewer:
The Citrix Desktop Service's connection to the Citrix Desktop Delivery Controller Service was terminated. The Citrix Desktop Delivery Controller Service is running on server '<http://server003.domain.com|server003.domain.com>'.

Check that there is no active firewall blocking the controller communicating with this machine.

Please refer to Citrix Knowledge Base article CTX117248 for further information.

Error details:
Keep-alive request was rejected.

They register with the DDCs fine, until a user attempts to launch a session and they get unregistered.

Register machine again with domain controller

The VDAs? remove from domain an re-add to domain?

K. will work on that. users are logged in, so will need to get them out to do the restarts.

Looks promising...Got one of the servers done and a successful login without a deregistration.

Edit:  now 3 successful sessions with no deregistration on the one server.

Edit 2: Working on getting users out of the other server now and will do the same process. This looks like the fix. 8 successes with no more deregistrations.

Well, it worked for a while...then it came back.

I have indicators pointing to the Mac clients causing issues all the sudden. I have other clients starting to have the same issue now. All of them have a Mac client attempting to connect about the same time, but that still could be a red herring.

I did upgrade the infrastructure to CU2 over the weekend, but I had seen one of these issues on Thursday before the upgrade as well.

Anybody having issues with Big Sur Mac clients? Just throwing that out there as a potential...I don't have confirmation of the user Mac version.

Many different MAC versions.

This almost certainly is related to 1912 CU2 rollout. Don't know what's up with it though for sure.

Waiting on the phone for a Citrix Escalation engineer.

Will probably have to rollback tonight. :disappointed:

Yeah it does. What's worse...I don't think that rolling back is going to help much either since I saw the issue for the first time on Thursday, before I did the upgrade. Just happen to have a new client onboarded on Monday that has the issue very regularly now.

did you tried to clean installation of VDA 1912 CU2?

Yes. Script does a VDA Cleanup utility. But also happened on CU1 before the VDA was upgrade to CU2.

Going to be rolling back to CU1 in a couple hours. Have not been able to get Citrix support to look at the logs the requested. Rolling back will very likely be the fix for the problems, because it started with these two main servers right at the time the CU2 upgrade was completed on the DDCs.

I also am pretty certain that the issue has something to do with Mac Big Sur, as the other servers with the sporadic issues also have confirmed a user with Big Sur connected.

Hopeful for a smooth rollback and calm tomorrow.

So your seeing MAC client connections causing the vda to unregister?

I'm curious, when the session starts and then it gets unregister. If you cycle the desktop service on the vda that just unregistered. Will it go back to a registered state. If so then launch the session again. After that will it continue to happen?

To add to that what can u find on the DDC showing it as unregistered? This is a basic check as I'm sure you have already checked. Just make sure your Discs show here correct ListofDDCs on the VDAs. Also check and make sure you dont have  the GPO applied that states Access this computer from the network.  If by chance you do add the 2 DDC names in the GPO. I had  both of these issue before and it was the same thing you described. Chances are you went through all this already.

Do you have a event I'd? I'll help you research it man.

These servers were working normally, with Mac clients until I installed 1912 CU2, so I'm pretty certain at this point that'll be the root cause. I was hoping for a fix before having to rollback. Brand new client for us had a rough couple of days.

It's not 100% consistent even with the two worst offending servers. Clients get connected, several of them in fact, without any issue. Then something will cause the VDA to become Unregistered. Once that happens, even restarting the Citrix Desktop Service doesn't allow new sessions to connect. The server will successfully register, but the Session Count in Studio will remain at 0. No Windows or Mac clients are able to connect. It seems that the Mac clients are the ones that seem to kick the issue off, but I'm not sure if it is when they are logging in or sometime after.

I can get the servers to start working again eventually by logging of users one at a time and restarting the Citrix Desktop Service. I know when users will be able to log back in again because the Session Count in Studio will start to show the actual number again. It hasn't been consistent about what session will free the server up. It could be a Mac or a Windows client...it could be a disconnected or connected session. I didn't have the forethought to see if it was a reconnected session. There's nothing I can see that would cause these servers to be worse than the rest of our clients, other than this being a new client and didn't get our recommendation to hold off on Big Sur and they have a lot of Mac clients, where most run Windows most of the time.

I have been through the ListOfDDCs and with Citrix Support added ListOfSIDs as well as a test. I use a GPO to push DDC names, not SIDs normally and this is all pretty stable until 1912 CU2.

The event on the VDA side is below but everything points at fixes we already attempted:
The Citrix Desktop Service's connection to the Citrix Desktop Delivery Controller Service was terminated. The Citrix Desktop Delivery Controller Service is running on server '<http://server003.domain.com|server003.domain.com>'.

Check that there is no active firewall blocking the controller communicating with this machine.

Please refer to Citrix Knowledge Base article CTX117248 for further information.

Error details:
Keep-alive request was rejected.

Dangan. Ok. You have done your research for sure. Well, let us know if reverting fixes it.

To be clear, I installed 1912 CU2 on the DDCs first due to some information about a bug with Director in the release notes recommending that this time rather than doing the VDAs first. Or I totally misread the release notes...which is possible.

The VDA on CU1 started having the issue within minutes of the DDC CU2 update being complete. I restarted the VDA and didn't think too much more of it because it didn't happen again until the next morning.

This has been my sole focus for two days. And I have other co-workers putting in time on it as well, everybody looking for research ideas.

Ok, well I hope you get it fixed. I need to read the release note in more detail now. Being that you mentioned bug and the install order.

I did upgrade the VDAs to CU2 as well Monday morning, but the issue persists.

Well I appreciate the information as it will help me out as well when time comes. Citrix upgrades make me nervous. Iv been doing them for 12 years now and it's always the same feeling.

Yeah, normally I prefer to hold off a little bit, but there are some bug fixes in this release that I really wanted in place (BSODs that we have been having occasionally). I figured since it was a bug fix only release it would be pretty safe, and the last few have been pretty safe too with no real issues.

I've been doing upgrades on Citrix since MetaFrame 1.8 .... I know the feeling about being nervous. :slightly_smiling_face:

Known issues in 1912 CU2
Citrix Director might fail to enumerate policy information. The issue occurs when you use an older version of Director to view session details associated with 1912 LTSR CU2 VDA. As a workaround, follow the steps mentioned in the upgrade sequence. [LCM-8201]

Looks like I initially skipped over Step 1:  Prepare where it says upgrade VDAs to 1912 or later, and went to Step 11: Upgrade VDAs being the last step in the process.

I also didn't upgrade the machine catalogs, which I remember seeing not needing to be done in newer versions. It says 7.9 or newer. Not sure if that could have anything to do with this, and there isn't much I can do to test at this point. Only when "several" users are logging in or already logged in do I see the issue.

Rolling back to CU1 on the DDCs and the two VDAs has allowed the servers to work normally. I have multiple sites and only one of the sites has been consistently having the issue. Another site had 1 instance of it that resolved itself with out any manual intervention. Therefore we made the decision to only roll back the one site.

The client with the issue has several Mac clients running Big Sur. I may have mentioned this already here, but we sent out a communication to all of our clients to please hold off upgrading to Big Sur, but this client is brand new and wasn't sent the request. The other incidents we have seen where intervention was required have also had Mac Big Sur clients connected at the time. I'm pointing the finger with only circumstantial evidence at CU2 DDCs with Mac Big Sur clients at this point. We have somewhere around 400 VDAs across the sites, and a grand total of 8 VDAs that have had the issue since Sunday.

If it weren't for the fact that one client had long periods where new users weren't able to connect, I would have much preferred to not do the rollback and continue working with Citrix support more on the issue.

I do appreciate all of the those who have taken the time to respond to both of my threads on this. I hope someone else may find some value from my pain in the future. :slightly_smiling_face:

For anyone following this still....

I only downgraded one of my sites, since the other sites were not having the issue regularly. An instance here and there, but nothing like the two servers in the specific site with the new client.

Progress on finding a root cause has been found though. One of our admins, who is the primary person responsible for the specific onboarding, appears to be causing the issue when he is logged in to a server via RDP. I am working now to narrow down what's special about his account or workstation (leaning towards workstation) to determine what's breaking the registration process when he's logged into the VDA via RDP and 1912 CU2 is installed on the DDCs.

I'll post another update here as I find out more.

It's not the workstation. There is something with his user account that causes the VDA to reset registration when he makes and RDP connection to any VDA in a site (2008R2 with 7.15, 2012R2 with 1912 CU1, 2016 with 1912 CU1, 2019 with 1912 CU1 or CU2)  with DDCs on 1912 CU2 from any workstation. He can log in with an ICA connection though and that does not cause the registration issue. I have provided CDF traces to Citrix yesterday, but haven't heard anything back  (my Citrix ticket # 80137286). I am attempting to raise the severity back to high, but they have been reluctant to do so, so far this morning.

Basics of what I have been able to see:
• This admin account logs in to a VDA.
• During or immediately following his login,  Event from Citrix Desktop Service 1048:  The Citrix Desktop Service is re-registering with the DDC: 'NotificationManager:NotificationServiceThread: WCF failure or rejection by broker (DDC: <http://ddcname.domain.com|ddcname.domain.com>)'
• If monitoring Studio closely can catch the VDA going to Initializing and back to Registered quickly. No additonal useful events I can find on the DDCs or VDAs. At this point there is nothing that appears to be wrong with the VDA.
• Next attempt for a user to login, the user gets an error that the app can't launch because the VDA is the only available machine in the Delivery Group and goes unregistered in Studio. On a DDC, Event 1101 from the Citrix Broker Service can be observed...`The Citrix Broker Service failed to broker a connection for user 'domain\useraccount' to resource 'Microsoft Outlook'. The Citrix Broker Service cannot find any available virtual machines.` 
 
• `Please add more virtual machines to the site. If the problem is due to existing virtual machines not becoming available, see Citrix Knowledge Base article CTX126992.`
• Log the Admin user out and restart the Citrix Desktop Service on the VDA and everything is back to normal.
• Does not happen with *any* of the other administrative accounts. Just this one. I don't know how to troubleshoot the WCF failure message. 

VDA 2009 and 1912 CU2 - FIXED THIS Issue

Event from Citrix Desktop Service 1048:  The Citrix Desktop Service is re-registering with the DDC: 'NotificationManager:NotificationServiceThread: WCF failure or rejection by broker (DDC: <http://ddcname.domain.com|ddcname.domain.com>)'

Well, no...at least not here...1912 CU2 started having this issue where never really had it before. :slightly_smiling_face:

Yes:  VDA in a site (2008R2 with 7.15, 2012R2 with 1912 CU1, 2016 with 1912 CU1, 2019 with 1912 CU1 or CU2)

DDCs are all Server 2019 with 1912 CU2 that have the issue. The downgraded site back to 1912 CU1 does not have the issue.

For me on WS2016, same error i have seen it fixed after installing 1912 cu2

<@U012AAZRJ20> Did you make any progress on this issue?  I just updated my test environment from 7.15 LTSR CU5 to 1912 LTSR CU2 and I'm seeing a similar de-registration/session launch issue.  I'm testing with a single Win10 client.  DDCs and VDAs are both Server 2016.  VDAs are fixed VMs so no PVS/MCS involved.  It seems like the VDAs will allow sessions sometimes but then stop.  I'm not seeing any relationship to RDP logins though like you are encountering.  I can see in Studio at the time of session launch the VDA will go to unregistered state and session launch will fail.  I have the same 1015 event at the VDA with "Keep-alive request was rejected."  Eventually the VDA will reregister with the DDC, but future launch attempts may fail.  If I can't find anything else, I might just trying to roll the DDCs and DB back to 7.15 CU5 to see if it functions normally.

<@U012AAZRJ20> Update: I spoke too soon.  I had used the VDA Cleanup utility on one of my test VDAs to remove the 1912 CU2 agent and then reinstalled it.  I was able to launch sessions from this VDA after the system rebooted to complete the agent installation.  However after I logged on to the VDA with my admin account, I see the 1048 event "The Citrix Desktop Service is re-registering with the DDC: 'NotificationManager:NotificationServiceThread: WCF failure or rejection by broker"  Attempting to launch new sessions on the VDA fails after I logged in with my admin account through RDP.  Rebooting the VDA restores connectivity until I log in with my admin account again.  This looks to be the same issue as yours.

I do not have any update on this yet. Citrix says we will connect tomorrow (Monday) for an update on the progress. The plan was to involve development with the logs we gathered on Tuesday and reconnect on Monday. Most likely will be installing some debug code in a test lab, assuming the issue happens in the test lab as well, which I have very little doubt that it will occur there. I'm putting some final touches on the lab build out to ensure it matches production as close as possible.

Sorry to hear that you are having the issue, but glad at the same time to not be the only one with the issue.  It sure sounds like the same issue, and if you open a ticket, you can reference my ticket 80137286 and I would appreciate being able to reference yours as not being the only person having this issue now. Perhaps we can get this solved pretty quickly.

Update from Citrix this morning:
"I‘ll need to open a case with the development team to address the issue being experienced. From the data collected we are seeing the Logon notification being sent to the controller however, the controller rejects it because it is not able to translate the Identity in the logon notification. This causes the VDA to deregister."

<@U012AAZRJ20> Thanks for the update.  I opened my own case yesterday 80165705 and referenced yours. It's currently being "reviewed" by the engineer.  I'll let you know if there are any new developments on the issue.

<@U01BLNEMDFU> Thanks for the ticket number. I have passed it along in my ticket notes as well. Just waiting for a developer now.

Received a private fix for the issue this morning and installed in test and it seems to resolve the issue. I will probably be putting into production tonight.

ahh that is good to know. is that private fix enviromental? or be public I wonder

It's a single DLL swap. I'm sure it will be included in a future release. I would bet that they won't make it public because how specific the issue seems to present, so they would likely force traces to ensure the issue is the same. At least that's my assumption based on past experience.

I see that 2012 Current Release was pushed out to the wild this morning. I'd be interested if the issue pops up for anyone or if it never existed there or if it was patched with the findings from the 3 cases I'm aware of. Most likely, I'll never know...

So far based on Slack, I seen this twice now. Which is on this thread as well. My luck it will happen.

<@U012AAZRJ20> Are you still running 1912 CU2 in prod?  Any issues to report? I did end up working with support and the private hotfix resolved the issue for me.  I'm not sure what we have in common that may have contributed to the problem.  The only thing I can think of is we do have some GPO hardening on these systems so I do wonder if this may contribute to the problem.  At any rate thanks for posting your case number here as it made the process of getting the private hotfix much smoother.

Glad you got the hotfix. I am still running CU2 in prod and having no issues. I am actually still working the case with Citrix as it appears they don't fully understand what the cause of the issue is yet. I owe them traces from my test environment with and without the hotfix and with a successful account login and with the account that caused the issue. Since I haven't been having issues, and I have been out sick most of this week I am a bit behind on things there.

Thanks for your reply.  I only made it to a level 1 tech so they didn't do anything more than have me repro the issue and then test the hotfix.  I asked him if this hotfix would be issued as a separate public hotfix, and he said that it wouldn't.  He said it would be incorporated into CU3.  I'm a little leery of upgrading my prod environment to 1912 CU2 and having to apply this fix.  I may end up waiting for CU3.  Please keep this thread up to date with any new information from Citrix.

I would expect CU3 to be about 5 months away at this point.

I wouldn't let this issue be a reason not to go with CU2 though, with the fix in place anyway. The issue is a simple name lookup/translation issue that they fixed with the patch. They are trying to claim that it is an issue with looking up UPNs from RDP that are longer than 20 characters. But I explained that if that were the case all of my admin accounts would cause the issue. There's something else in the lookup process they used with CU2 that is broken. Which is why I owe them traces.

its frustrating once someone gets something in there head, nothing else matters it seems.

<https://worldofeuc.slack.com/archives/CK6TCV3JN/p1606843222411800?thread_ts=1606150044.279600&amp;cid=CK6TCV3JN>

having the very same issue in our environment but with CVAD 2209 Broker and 2212 VDA.
Anyone with more insides to this issue?

image.png

Not on those versions myself at all, so can't speak to why it would crop up.

Is it exactly the same that if an RDP user is logged in when an ICA connection is attempted it causes the VDA to deregister?

I know Citrix handles the lookups for RDP vs ICA sessions differently for how they display in Studio and this difference caused the issue for us, but not for every RDP user, just one specific RDP user in our environment caused it, which made for a difficult troubleshooting experience.

hm, not RDP, but there might be an Autologon account using a local session. but that user gets logged off before the VDA is being registered

Top of my head without testing it :)  get-brokerapplication | where visible -eq $false | set-brokerapplication -enabled $false

What are you taking to remember things like this? I want some

Linking to here, help appreciated!  Might be some weird Chrome thing.

I think that’s not possible. You can only register your license server in the cloud for better monitoring. 

<https://docs.citrix.com/en-us/citrix-cloud/citrix-cloud-management/citrix-cloud-on-premises-registration.html|https://docs.citrix.com/en-us/citrix-cloud/citrix-cloud-management/citrix-cloud-on-premises-registration.html>

ok, than it should be easier to move to cloud connectors :wink:

Tried it (Zoom) as a package and it didn't work. Then CISO banned it so I didn't have to worry any more

Ok, well not something I wanted to do anyways.

Some of our customers use it but they have it in full desktops not as a published app

Which goes back to James putting it in explorer, as I think it loads the peices need for explorer to make it function like a published desktop if your using seamleas app approach

Yeah launching the shell processes some stuff that isn't normally done with a seamless app (sorry I can't be any clearer than "some stuff" - think processing Registry shell startup items and Active Setup, etc.)

Your are exactly correct. I use this alot for that <https://docs.microsoft.com/en-us/archive/blogs/askie/things-to-do-when-troubleshooting-internet-explorer-terminal-server-and-profiles-issues>

Just foun it. Every hour one json file :shushing_face:

Only in my lab so I can update the CVAD V2 doc script. No issues in my lab.

I didn't have any issues in my lab either, just production with about 6 VDAs occasionally (2 almost always) going deregistered.

Thanks for the reply! :slightly_smiling_face:

Update 3 customers with &gt; 1000 users. No problems. 

I've started my side-by-side upgrade using it, too early to say! but good to hear nobody have issue with it!

More info on my issue after working with Citrix. Something on some of our servers is hanging up the Desktop Broker service. It is very strange. Getting the proper user account to log out has released whatever is locking up the Broker and then a reregistration allows new connections again.

Main issue is on 2019 Server. But had limited instances on 2016, and 2012R2

Carbon Black. Happens with CB in ByPass mode (not doing anything)

Watch out if you are using HTML5 Workspace. After updating Citrix StoreFront from older Version to the 1912 LTSR CU2, HTML5 Workspace stopped working. Checked this twice in three different environments. After a few minutes there is a connection timeout with "SESSION:|:ICA:|:TRANSPORT:|:DRIVER:|:close with code=1006"

Workaround:
- Fallback to Citrix StoreFront 1912 LTSR CU1
- Immediate Rollout 1912 LTSR CU2 VDA to all Citrix Worker (not always possible as supporting older OS with 7.15 LTSR VDA for example)

it's not related to the ADC Firmware, tested with several versions. It's a problem with CU2 StoreFront and the communication with "older" VDA Agents, it's only working with CU2 VDAs. Created a citrix case and now waiting for any confirmation.

I didn't have any issues with HTML5 with 1912 CU1 VDA connections fortunately. Also, just tested with a Server 2008R2 OS with 7.15 CU4 VDA (I know, we're working to get rid of them.)  and was able to connect with HTML5 on and CU2 StoreFront.

Citrix Workspace app for HTML5 - v: 20.11.0.4148 (2011) in case the version is different due to upgrades I have done with the HTML5 clients. Haven't looked to see what version is shipping with CU2.

Julian: did you get any answer from support?

No confirmation atm. Citrix is working and recreating the problem. Will keep you updated. 

I’ve deployed 1912 CU2 to about 30 win 10 1909 machines so far , no issues noted so far. Sunday , will be a bigger push of approx 750. I’ll report back if I note anything major 

Hey all, as per my above post, I completed a roll out of 1912 CU2 on Sunday, and there were no major issues noted. Win 10 1909 was the OS build

DOes it go away when you point directly to DDC?

Yeah i did try that during my testing and it does work and enumerate resources when pointing directly at the DDC rather than the DNS A record that points to the DDC XML vip thats why i believed  it was Netscaler specific with the DDC XML Vserver/VIP so i started changing the ssl settings on the vserver and found it only works with the secure default front end ssl profile

Yeh info is in the doc i posted above. I will do a packet capture to see what is going on with the ssl handshake but surprised it doesnt work when using ssl parameters with just ssl v3 off and default cipher group. Even with ssl profiles and the default front end profile it doesnt work it will only work with the secure default front end profile

Interesting. wonder whats in that profile

i thought it would list ciphers. i had an issue with Win 10 machines and default profile. had to use secure back end profile.

so i looked at the cipher group the default front end ssl profile is using and it uses the default cipher group but the secure front end ssl profile is using the secure cipher group. I copied the default front end ssl profile and swapped out the cipher group to secure and guess what eneumeration works fine when SF is pointing at the XML VIP these are the ciphers the secure cipher group is using

interesting article  seems to suggest that TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, or TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 must preceed any TLS_DHE_ cipher suites which is not the case in the default cipher group on the Netscaler

created this new cipher group and bound it to a copy of the default frontend ssl profile and its all working :slightly_smiling_face: so its something to do with the cipher priority in the default cipher group which is used in the default frontend ssl profile and even when using ssl parameters. It seems like the ECDHE ciphers must be prioritized over the TLS_DHE cipher suites for the DDC/XML VIP to work

If I could give you an advice is do NOT LB xml service.

SF is better at detecting an XML issue than NS. I had a P1 because a bad XML controller and NS didn't detect it, throwing an error when starting a desktop. If going SF to XML controllers directly with the broken controller, it was working.

I opened a case with Citrix, fight hard on this one, asking how NS is not as good as SF to LB XML, etc. never got anything back. I just stop LB my XML.

I am not the only one preaching this.

Which NS version do you have? I had a customer with a similar issue where Palo Alto update was pushed which blocks the use of CBC ciphers... looking at the (not working) network capture you shared, it appears CBC was in use by the default NS profile. It’s addressed in here <https://support.citrix.com/article/CTX240139|https://support.citrix.com/article/CTX240139> 

ran into this issue on another environment today it had 2012R2 storefront servers but 2016 delivery controllers that are added on the storefront store. The storefront servers couldn't enumerate anything using secure  XML from this XD site that has the 2016 DDC's. There was alot of XML service failures and SSL handshake and cipher negotiation errors in the SF event logs. I had to add the ECDHE Ciphers to the 2016 DDC's and make sure they was above DHE Ciphers then enumeration worked so its not just XML/DDC Virtual servers on the Netscaler it affects :)

Yes it does, it replaces your sysprep, it takes care of the identity (uuid, etc.). The shutdown probably fixed your issue. In MCS, shut down is different than reboot.

I wonder why it keeps running without 16mb disk

The VM can run but you're losing the identity part.

Maybe your logon is cached, can you logon using a different logon? Access network resources, etc.? After I don't know in details how/when the identity disk is used. Maybe it is a one time only like a sysprep, I've never really done much tests.

ok i re-attached the identity disk and rebooted. seems to work. interesting how it  worked without it

yes I tried logon from another user now with identity disk attached

It should work without identity disk, because you‘re using full clones, so identity disk is only used for creating, as after creating, every clone runs as standalone. I can say it will always run without the identity disk, migrated MCS full clones from one xenpool to another, it’s not officially supported as it’s bound to the hosting / storage connection. I deleted all identity disks and did an export / import for all machines. Worked without problems. Thought about new machine catalog, but users installed their personal software, so redeployment wasn’t an option. 

For MCS linked clones, the clone will loose the binding to the master image if you delete the identity disk, so it’s Never getting an updated snapshot if you’re doing a rollout

Alright that's good to know, thanks <@ULCBD9AJD>! so for a full clone, the identity disk is used a one off like a Sysprep's mini-setup.

<@ULN862X8E> was this you that had a similar problem?

<@ULLH5JGFR> HDX Audio fails to import devices when there are >5 audio devices.  You'll have to disable devices on your local machine to get under that threshold and then reboot the VDA

eg.

Ahhh nice man. That is great. Thank you so much

Keep in mind, if you have an NVIDIA GPU... the NVIDIA HD audio reenables anytime you do driver updates

I had the issue when Steam added new playback and recording devices to my machine, someone else had it with an Oculus (sorry I can't remember who).  It seems to happen when exceeding a max number of recording/playback devices on your endpoint, my workaround was to remove an old wireless headset, looks like It might be fixed in 1912 CU2 and 2009 [CVADHELP-12760]

As I recall it's in the virtual machine folder on the datastore.  I am sad that I even know that

I have no idea what I’m looking for, it’s not named pvdisk.vmdk or anything…

is there a powershell command that shows you what it’s using? God I hate PvD

Actually in the hosting connection it'll let you define the datastore they go to.  When I used it, it was just part of the machine.  It was a long 3 months trying to get it to work and it never did

so the machine I’m looking at has 2 disks, multiple vmdk files, but the pvd is supposed to be a vhd? Where is that stored?

why is it not referenced in the log as being mounted?

am I looking at the wrong log? This is so confusing, I can see why everyone dumped it

there’s nothing at all in the vhdmp log…

oh it’s not even mounting the vmdk where the vhd should be because vmware sees it as corrupt, great

PvD is end life, move away from it when you get a chance.

that’s whjat I’m trying to do but need to be able to get it mounted to get the user data out

I managed to clone the corrupt drive to a new drive and mount that but pvd still complains it doesn’t have a disk to expand to

Hm ok. PvD userdatav2.vhd is contain within another vhd. You need to expand your "another" vhd to grow your userdatav2.vhd. That's how I was doing. I've seen the userdatav2.vhd failing to grow. PvD can be a pain, Citrix support is useless on it.

that’s about what I expected. I can’t find the VHD, where is it normally?

P:\ProgramData\Citrix\personal vDisk\UserData.VDESK.V2.TEMPLATE

It depends on how is your Resource configured, usually on a your shared storage, you should see this info on the Resource used. In XenServer, it is on my storage repository.

No it is not what you need.

On your VM, you have a PVD disk attached no?

I have no idea… they aren’t labelled as pvdisk

I think I found the disk it should be and attached it

with PvD, you should have 3 VHDs, 1 diff, 1 identity, 1 PvD. Yes?

Ok so 1 of them is your PVD, usually around 10-20GB

Mount this VMDK to a normal VM, then open it, and mount the userdatav2.vhd file, it contains the user data.

so I mounted that as P:, I can’t find where the VHD on that drive is

what do you have at the root? Are you sure it is not the other vmdk? This one could be the .diff vmdk

This P:\ProgramData\Citrix\personal vDisk\UserData.VDESK.V2.TEMPLATE is usually found on the .diff disk.

I cn mount that, this shouyld be enough to copy out for them to a new machine

Indeed, you're welcome! :slightly_smiling_face:

Does anyone know how to force Windows 10 to use specific graphics adapter? I have VM in Azure with AMD GPU but for some reason VM keeps defaulting to Citrix IDD. I tried setting the reg key for AdapterMerits but its still not working.

Isn’t that in the release notes for cu2?

Indeed, but setting this key to the recommended value may also have downsides

I just got confirmation that SE was able to reproduce the issue

Good morning, Has anyone seen this on their citrix servers when people try to login to office 365?

You talking about micro apps or the whole Citrix vision of workspace? I am trying to use those th my B2B guest users but a lot of stuff doesn’t seem to work when I use Azure AAD as the authentication method for workspace

Either/or. The demo from Citrix at Synergy 2019 had an envisionment of the company home page hosted on Citrix workspace, it was pretty slick, but , it was a demo :slightly_smiling_face:

I am trying to embrace it, we own it. I am hoping when SAML authentication is available more things work. 

Check the Registry and see if HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer - NoViewOnDrives and NoDrives are actually being set

They need to be set *before* Explorer starts to generally take effect properly

Thanks, James. I'll check that... WEM failed to create them, I recall checking when in the weeds at that point but at that stage, I was sure WEM was just being shite :slightly_smiling_face:

Steady, you'll get the WEMbles around here upset

Kindon already slapped me for putting those settings in WEM in the first place :stuck_out_tongue:

Citrix ADM is great for the network analysis (HDX Insight). It is a great way to prove the issue is network related or not, my helpdesk has access to it. Director is also a good start (CPU, memory, load index, ICA RTT).

Unfortunately it still can’t do almost any check on endpoint especially on BYoD. Are you able to check CPU /Memory on endpoint?

I was using HDX Insight but it’s still missing a lot.

No the endpoint is a black hole, not much we can do to monitor those. But we have a process in place. If we don't find issues with other monitoring tools, we ask to try with a different endpoint.

You are right. However I’m going to raise feature request with Citrix for such feature “diagnostic mode”. From my experience: few features requested by me already implement by Citrix and version 2012 will bring one more :shushing_face:

Client montitoing is a real pain. We need a api in workspace the gather stats on the home network around more in depth. There are times ADM shows client RTO is high. Then we ask them to try another device. The end users are ALWAYS pissed. They blame Citrix so much and it's aggregates me bad.

Workspace App already on client device. Just add more features (diagnostic features)

What would the diag feature do? meaning what would it collect for us?

They are all here: <https://docs.citrix.com/en-us/tech-zone/build/tech-papers/citrix-communication-ports.html>, I guess most of what you need will be under the CVAD section... cheers

<@UR99U4W0Y> unfortunately there's no answer to my question...

Where are you hosting the DDCs? Presumably Azure is an extension to your network hosting the DDCs and there connectivity in the way or a site2site or Express route?

DDCs are on local premises. It's quite clear how to enable communication between DDCs and VDAs, but I'm asking about the MCS part. Machine creation, power management, etc.

You need a hosting connection in Studio for <https://management.azure.com> ... are you using MCS for provision your VDAs?

<@UR99U4W0Y> Even on the main connection window I can see at least two more URLs, seems just <http://management.azure.com|management.azure.com> is not enough...
Yes, I suppose to use MCS, because PVS is not supported in Azure yet.

Did you create a service principal for the connection via Citrix Studio? If so, it should have the required permission to create/delete resources... check if you have an app registration for Citrix in Azure AD ent apps, cheers

My question is not how to make a connection, but what to put into the request for network team. The most important: this list should be comprehensive, I have to be sure that all the features will work as they should.

I 'think' the auth URL is only used for having Studio create the required app registration... once thats in place I think it juts needs <http://management.azure.com|management.azure.com> with the app id/secret... I'd confirm with Citrix if you want to be sure you get the request right... cheers

*Im guessing its 443 and 80 to azure (which IP's maybe different story). The MCS part is basically talking to azure management to create and delete machines.* 

I'm guessing too :grin: And I'm surprised it's never been described anywhere.

outbound ports :smile: not as critical. I think your biggest issue is going to be determining the block of MS ip's.

i just did stat on my DDC. all connections to ms are https.

Check this out <https://www.kreyman.de/index.php/citrix/citrix-xendesktop-xenapp-server/180-citrix-cvad-architecture-diagram-and-communication-ports>

Look on the diagram. Cloud connections are all 443.

Yeah... server vda... so it works... cheers

<#CQ1AGQJ85|> might be a more appropriate venue?

Not sure why I didn't see that one.  Thank you.

Seeing the same issue with CU1 Hotfix and CU2. Just building a new vDisk with 1912 VDA without the security hotfix