https://worldofeuc.com logo
Join Slack
Powered by
Hi all, nut driving issue here. Got two servers. B...
# citrix-vad
m
Hi all, nut driving issue here. Got two servers. Both server with 98% same software. Server A initially installed by an automated task sequence + "SOME THINGS" the customer did manually after that. Server B fully automated deployment without these "THINGS". Software set is identical. Server A does a logon in 15 secs Sserver B takes 30-40 secs. Been searching for a day now and cant get the cause for this. Diskmark is identical on both servers. Same host, same storage, same VLAN, same WS 2022 Image. It seems there is an issue while appliying policy. Removed AV for test, still slow. Removed WEM, still slow. Any idea? Also an (AD) Admin logon via RDP is about 10 Secs slower when appliying GPOs at logon..
k
Breakdown the logon phase with UberAgent or ControlUp.
Could be as simple as a wrong OU, Sites and Services, WMI filters on GPOs
m
Same OU, same DC..
k
Or mappings to non-existing shares (yes, I’ve seen this in the wild)
Something’s different so start with breaking down the process so you can narrow it down
s
did ngen run in the fully automated image? or AV scan? Thats something that could have kicked off and finished in the manual one in the background before sealing.
m
i removed AV from the "slow" server and its still much slower than the other one. NGEN did run, however it also should only run when the Server is idle? Both servers seald by bisf
s
any idea on "THINGS" that were done?
m
image.png
Group Policy 9.3 vs 1.8, servers in the same OU..
s
this is non persistent? is it just the first logon this happens?
m
yes, non-persistent., but every logon. It seems every communication against DC is laggy. For example adding a user to local direct access users group.. Takes 1-2s longer for searching the ad user
s
MCS?
m
yes
k
My first hunch: DNS, sites and services, WMI filters
s
the machine catalogues are identical? Same NIC? (e1000 vs vmxnet3)
is this vmware?
b
RSS Optimization?
@Ray Davis has great input here: 
When opening a UNC path to the Golden image or the VDA to \W11\C$ for example.
When copying a file from a source machine to the destination machine. Destination being \W11\C$ for example (Golden or VDA that is W1124H2). it would stall and state calculating for a long time. After about 12 minutes, it would start copying, but it was 355kbps.
The seemed to impact anything that relied on anything be called via SMB
Did some digging and relized it was due to DisableTaskOffload' -Value '1' and rss=disable
Once I put the DisableTaskOffload' -Value '0" the speeds went from 355KB to over 165MB
Then once I reset this rss=enable, the speeds were over 300MB and things were back to normal.
These two settings are legacy and not needed for modern Operating systems from what I am seeing now.
Quick workaround to update [Preparation/97_PrepBISF_PRE_BaseImage.ps1]
$BISFDir = "C:\Program Files (x86)\Base Image Script Framework (BIS-F)\Framework\SubCall"
((Get-Content "$BISFDir\Preparation\97_PrepBISF_PRE_BaseImage.ps1" -Raw) -replace "DisableTaskOffload' -Value '1'","DisableTaskOffload' -Value '0'") | Set-Content -Path "$BISFDir\Preparation\97_PrepBISF_PRE_BaseImage.ps1"
((Get-Content "$BISFDir\Preparation\97_PrepBISF_PRE_BaseImage.ps1" -Raw) -replace 'rss=disable','rss=enable') | Set-Content -Path "$BISFDir\Preparation\97_PrepBISF_PRE_BaseImage.ps1"
m
@Steve Noel yes, vmware. Happens also on both "fat" masters, so no MCS. This is the logon of a domain admin via RDP:
s
can you see if switching the NIC to e1000 fixes the issue on the master?
m
no change with e1000e (or should i also try e1000 non-e ?)
s
Prob not needed and not related to what i was thinkign was https://www.jeffriechers.com/wiki/24h2-windows-11-and-server-2025-vmxnet3-issues/ I had similar symptoms, but dont think it's related if issue still persists with e1000
m
I installed dsa.msc on both servers. When "changing DC" the slow server cant enumerate the DC correctly:
when i test-netconnection to the DCs all seems open
s
can you do another automated build and stop it in the beginning to see if it's good/bad out of the gate?
m
jepp, right after domain join without any customization:
c
check network interface settings between both systems? netsh interface tcp show global
m
@Corey Tracey no difference between fast and slow server
c
Same network settings? DNS, gateway, etc?
m
same
k
Same ESXi host? Wondering if it’s a network/vlan mapping issue
m
Just installed a 2022 ISO manually, domain join, dsa.msc, same. It is VLAN related. Moved this naked server to another VLAN and the DCs went available. The strange thing is, that the "fast" servers are in the same VLAN. A watchguard is routing the traffic to DCs so a watchguard guy needs to have a look on it. Puh, what a journey - and thank you all for contributing. Nice side effect for @Kees Baggerman: The customer moved from esx to ntx and reported performance issues. It seems theyre network related and have notghing to do with ntx 😉 To clarify, all tests were made on ESX.
🙏 1
s
there we go. way to isolate.
k
@Marco Zimmermann you can see in vCenter which vlans are offered on the vswitch/nic. This might just be an issue with the vlan mapping or vswitch setup (assuming it’s not a dVS)
m
#solved 😄 There is a FW between DCs and Citrix Servers and a DHCP Scope from .100 - .150 Coincidentally every new VM did get a IP below .110, because there was no lease. Changing this IP manually to an IP .110-.150 and the VM is fast. For me it seems like a watchguard bug, as the rule watchguard rule applies to .100 - .150. Also the traffic was not blocked, it was just slowed down. Puh, now i´m delighted :D
6 Views
Open in Slack
PreviousNext
Powered by