Hey! I'm looking for advice on a strange PKI issue...
# _general
j
Hey! I'm looking for advice on a strange PKI issue. We're testing the move of a stand-alone Enterprise Root CA from one server to another. On the day of the move, things work fine. On day two, "smartcard" (SAML) users get "The user name or password is incorrect", and the RDS host logs "unable to check revocation because the revocation server was offline." Day three and onward, things work again. (more details in thread)
In the lab: -Move CA service following Microsoft and PeteNetLive guides -Republish enrollment agent and smartcard logon templates -Reissue DC certificates -Delete exising smartcard logon user certs -Reissue Enrollment Agent certificates We've tried deleting the cert cache on the RDS host per this Parallels KB: https://kb.parallels.com/en/128635. And checked for duplicate certs per https://support.citrix.com/article/CTX238881. The CA is online and reachable. In our latest test we noticed in the Enterprise PKI snap-in that the DeltaCRL Location #1 was expired and still pointing to the previous CA server, but we're still broken even after correcting that with certutil -crl and reissuing new certs to DCs, enrollment agents, and users.
c
Have you checked your CRL/AIA locations with pkiview ?
i would also export a users Citrix_smartcardlogon cert from your FAS servers and run Certutil -verify -urlfetch "name of the certificate.cer" and check it for any errors related to the CRL/AIA
j
Thanks, John! I'm only just getting familiar with the Enterprise PKI snap-in. We did see an expired DeltaCRL ldap location pointing to the previous server. Running certutil -crl on the CA fixed that, but the RDS "unable to check revocation" errors continued. I believe the certutil -verify came back clean on the user cert. I'll double-check that during our next test. It's tough when it only breaks for 24-hours and then we have to wait another 24-hours after moving the CA for it to break again. It seems to line-up with the DeltaCRL expiring/not renewing, but fixing that CRL doesn't seem to fix the issue. Is it suggested to delete all issued certs and re-issue after a PKI change? DCs, Enrollment Agents, users, etc...
o
Hi JD, i recently had a FAS issue for a client, I ended up checking event logs in 3 places: StoreFront, FAS , CA, ultimately, it was a permissions issue. I had to add back a permission i'd removed on the related Citrix FAS smart card certs that I'd removed 2 weejs prior. The error that helped me was logged on the CA. Where was your error logged?
j
On the RD session host I see an Audit Failure in the Windows Security logs: Failure Information: Failure Reason: An Error occured during Logon. Status: 0xC000006D Sub Status: 0xC000038B and a Security-Kerberos Error in the Windows System logs: The domain controller rejected the client certificate of user <username>@testdomain.local, used for smart card logon. The following error was returned from the certificate validation process: The revocation function was unable to check revocation because the revocation server was offline. I'll try rebuilding the enrollment agent and smartcard user cert templates and permissions next test. Thanks!
a
I'd guess at CRL/CDP timing issues based on my experience when switching to a new CA at a customer. If using LDAP by default for your CDL, it includes the CN of the server the CA was on, and that full path with that CN will be embedded in existing certs. For FAS issued certs from before the CA move, they will try to lookup the old server for the CRL as that's what's in the certs. If that is offline then they cannot be verified and may fail. Certs issued by FAS/moved CA will have the new servers CN in the LDAP CRL and as its live it will work. If you are only in test phase look at changing the LDAP CDL location on the CA to a location that doesn't include the server name (think I recall you can do that), or to publish to an http location to save pain if the CA ever moves again.
thankyou 1
j
If I correct and republish the CRL, which certs need to be regenerated? Domain Controllers, EnrollmentAgents, or just the user smartcard certs? I won't be able to test again for a few more days. I appreciate all the pointers!
It appears that even though the correct CA was showing in the mmc snap-in on the DCs, it was missing on one DC when we ran 'certutil -store CA'. Manually adding the new root CA cert via certuil CLI on that DC seems to have fixed things.