https://metaplay.io logo
Unordered Actions
# questions-and-feedback
a
Hi again. I had trouble sorting an out-of-order action processing error, which was causing clients to be kicked. I thought that was when the client fires two actions, and their processing on the server may get interleaved as they have async calls, but I fixed that by making the server listeners quick and sync, and the problem still occurred. I've been unable to repeat it locally and with test code, but it happens on the cloud hosted server and actual use sometimes. So after not being able to fix it, I did this terrible hack experimentally, to just tolerate unordered processing of actions, in the case where the order does not matter, and everything seems to work great now 😮 I wonder what all that breaks, though. I haven't found any problems so far. in PlayerActorBase HandleFlush
Copy code
csharp
    else if (nextUnsynchronizedServerAction.TrackingId != unsynchronizedServerActionMarker.TrackingId)
    {
        //_log.Warning("Unsynchronized server action rejected. Out of order. Expected id {ExpectedId}, got: {ActionId}", nextUnsynchronizedServerAction.TrackingId, unsynchronizedServerActionMarker.TrackingId);
        _log.Warning("Unsynchronized server action detected as Out of order. Expected id {ExpectedId}, got: {ActionId}. HACK FIX: am tolerating this! --antont", nextUnsynchronizedServerAction.TrackingId, unsynchronizedServerActionMarker.TrackingId);
        //throw new InvalidOperationException($"PlayerFlushActions, trying to re-order Unsynchronized Server Action");
    }
hm. it has not occurred in 27h on the server anymore, though, so maybe my fix did help, and i just checked it wrongly earlier i'll check again later and remove the hack if it's actually not needed
j
The hack doesn't break anything, except allows clients to "run" unsynchronized actions in a different order than server. But that doesn't really matter as server is the authority. With async interleaving yea I can see this happening.
ExecuteServerActionImmediately
tries to prevent callers from non-actor-current thread exactly to avoid races. Not sure why that didn't work here. But with sync calls, I don't really see how the order could get mixed up in the first place. A tip: if you have async operations that may complete in various orders, one easy way to order/linearize them is to use
ExecuteOnActorContextAsync
and push the result handling to the actor thread. So in this case, we could compute stuff in the async continuation, but finally to apply the result, we could do
ExecuteOnActorContextAsync(() => ExecuteServerActionImmediately(action))
a
yep that's what i do now, and the error got really rare i've still gotten it a couple of times in a day, when i checked it late last week, will check soon again
no occurrences since Fri 20th.. so in 4 days i think i'll check again if i have some async server player listeners still, but i don't think so.. i'll try to bomb it with my test ui and perhaps with botclient too, maybe that's how i got those on Fri thanks a lot for the info that the back is mostly harmless, a great relief for now! i might also look into improving the logging at some point, if it could tell which actions the ones in queue are, i might be barking at a wrong tree
oh now a bunch has appeared. gotta check who that player is 🙂 one of our devs i think
got 13 occurrences within 1h
Copy code
2024-09-24 14:42:10.992    
[11:42:10.992 WRN Player:P7YuheTk4z] Unsynchronized server action detected as Out of order. Expected id 16, got: 24. HACK FIX: am tolerating this! --antont
2024-09-24 14:39:02.129    
[11:39:02.129 WRN Player:P7YuheTk4z] Unsynchronized server action detected as Out of order. Expected id 15, got: 22. HACK FIX: am tolerating this! --antont
2024-09-24 14:35:03.764    
[11:35:03.764 WRN Player:P7YuheTk4z] Unsynchronized server action detected as Out of order. Expected id 11, got: 20. HACK FIX: am tolerating this! --antont
2024-09-24 14:31:07.502    
[11:31:07.501 WRN Player:P7YuheTk4z] Unsynchronized server action detected as Out of order. Expected id 10, got: 18. HACK FIX: am tolerating this! --antont
usually the id has been only 1 off but in that bunch it seems to be 7-8
oh FYI, i did realize that i had some ServerPlayerListener s as async still, and was getting this out-of-order thing about once in 24h with our test users i've now done the conversion to the ExecuteOnActorContextAsync for all of them so they are probably gone, that's in our develop env and i'll check at some point if all has been good and will update prod and then this is probably gone