Mine just crashed, with the remediation applied
# citrix-netscaler
r
Mine just crashed, with the remediation applied
i'm going to ask Claude to look at it, and alter/fix the expression to match better
r
Investigate and possibly apply the gotham mitigation script from the ShareFile script. I am hearing that some copy and pastes are not working properly. The shell script creates the policy correctly for certain.
šŸ‘ 1
r
The same forged assertion came back. It reached nsaaad at 12528 and 12550 PM on ns-01, then at 12604 and 12625 PM on ns-02 after the failover: the identical 16-entry PrefixList="P P P …", idp.test.local / The policy didn't match it. It showed 0 hits and 0 undefined hits.
i think it's on me... i applied to Gateway servers, and not SAML vservers..
āœ… 1
j
You mean you forgot the AAA vServer Bindings?
r
yes
bind authentication vserver
d
doh'
r
I disabled our Authentication Virtual Server. Users are still able to log in, probably if they've logged in somewhat recently.
e
@Ryan : Any policy Hit now that is bound on AAA ?? As you seem you were a tgt
j
@Ryan Gallier are you working now? Im still crashing
r
i just applied the Auth vServer fix 20 minutes ago... but still up..
j
and you are talking about these right?
r
bind authentication vserver
j
image.png
this?
r
yes
j
how do you have them bound?
image.png
im dying here
just need anything from anyone
r
image.png
yes, that
if this doesn't work, i've re-written the responder policy to be more strict... but you say you've still seen a crash with yours bound like that?
j
yep
r
guess i should try now... sec
j
im changing all my bindings to 10 from 100
r
Here's what I just did
j
so this is a new responder policy? more strict
d
yes
c
i dont see any hits on my responder policy šŸ˜ž
j
@c4rm0 are yours still rebooting?
r
i didn't see any hits on the responder policy, then it crashed again, hence why i tightened it
j
ok we are doing PA blocking now
j
Anyone see any hits on the resp pol? šŸ˜‚ Citrix itself maybe?
r
by my estimates the next attack is in 7 minutes
j
you'll only see the hit if you are being attacked right?
not on every auth
since its looking at the saml payload and dropping it when it meets the criteria?
d
that is my expectation
r
Yes, the responder should only be hit on an attack
good logins pass through
k
would the policy drop it at the send and it sitting in log, or after the fact? We added the policies, and tested the exploit again:
j
how did you test it?
k
Sent a malformed payload similar to the WT blog.
s
ya'll get another reboot about 5 mins ago?
r
if you did, and I didn't, that would prove my new policy works
d
need to rename that policy RGPolicy.. hahaha
c
that pastebin link isnt working ryan
r
sorry, my home network geoblocks. pasting here
Untitled
So, did others crash at 2:41 ?
c
i crashed at 7.39pm GMT
r
so, 2 crashes at that time, but we didn't crash...
but, i still have no responder hits
we've been up for 1h 45m
still, sadly, 0 hits on the responder
l
supposedly there is a v2 of the responder policy - has anyone seen that yet? I didnt see it on reddit
r
i mean, i wrote the one above...
and i called it v2.. so it's probably mine. šŸ˜„
We also blocked 213.209.159.55 on the firewall, which could also be helping
l
ah i see the screenshot for the one you called v2 -did i miss it in text?
n
There's v2 in the naming convention, so I think you read past it. :p
d
anyone still seeing reboots in the near recent?
l
@Ryan Gallier thank you. Curious, is this a script you created or got from citrix?
d
been up for 2h20 here
r
i created it, from the data from my traces on the crashes
šŸ’Ŗ 1
l
Citrix insinuated they have a V2 of the responder policy but we have to 'wait for support.' so im interested to if they are getting their solutions from the community
r
using Citrix's basic logic
it would not be the first time they used one of my fixes... wrote a KB article on it... and didn't credit me. LOL
🤬 2
l
the v1 we got from citrix matched everything here
r
Thats... not helpful?
2. If you are currently experiencing the impact from this issue, please contact Citrix support.
d
call support
šŸ‘€ 1
and wait
well contact.. cuz you can't call
c
that article is a joke
n
"Wait until the AI is updated to tell you what to do cause we don't know yet, currently sourcing scripts from the community."
😁 1
šŸ˜‚ 1
d
you could have atleast pointed to some in the article...
m
looks like another patch coming:
r
It will have to be as the responder policy is only going to block the current payload. Threat actor changes the payload and you are broken again. Two items good to know 1. IP Reputation had the IP address in the database already so enabling IP Reputation should stop it. • You can send the management logs (httpaccess.log etc.) to syslog now. • switches • managementlevel ā—¦ ALL ā—¦ SHELL ā—¦ NSMGMT ā—¦ ACCESS • mgmtloglevel ā—¦ ALL - All events. ā—¦ EMERGENCY - Events that indicate an immediate crisis on the server. ā—¦ ALERT - Events that might require action. ā—¦ CRITICAL - Events that indicate an imminent server crisis. ā—¦ ERROR - Events that indicate some type of error. ā—¦ WARNING - Events that require action in the near future. ā—¦ NOTICE - Events that the administrator should know about. ā—¦ INFORMATIONAL - All but low-level events. ā—¦ DEBUG - All events, in extreme detail. a. NONE - No events.
m
So we think the current policy from Citrix support is not blocking it?
And IP blocking is the only thing working now?
r
that's accurate
d
pretty sure that for us blocking the IP's is what's actually doing the work.. don't see any hits on the ns
so the good news is that atleast for now.. that means that the low tech is effective
m
Yeah I haven't seen anyone get a hit on the policy
d
have a test environment that hasn't been hit at all yet.. with only the v2 that Ryan wrote
issue is.. it's not in the attack pool.. so may not help
but it's out there
not willing to remove the ip block from "production"
s
is 213.209.159.55 the only IP people are seeing for this issue?
šŸ‘€ 1
m
I don't think so but it's the main one or the first one
d
38.134.148.238 - Payload host on :8080 (x.sh, client.py) and reverse-shell listener on :8888 167.148.88.236 - Exfil target on :5000 (uploads a /flash/nsconfig tarball) 213.209.159.55
as i understand it
j
The 213.209.159.55 is marked as BAD by Opentext, which is the IP-DB for NetScaler's IP-Reputation. Sadly, the two others not at the moment. Checked here support.threatintel.opentext.com/tools/url-ip-lookup.php
d
full credit.. those are the ip's that Ryan saw pretty early in the payload... we blocked them on our firewall for extra comfort should the payload actually find a way to run
j
I've reported the IP's now to opentext
ā¤ļø 2
They updated the DB, it's now marked as scanners and Web attacks šŸ‘Œ
d
Fantastic work