https://linen.dev logo
tailnet coordinator
# general
s
I don't believe DERP is over UDP
The problems I've seen involve just being unable to connect and it eventually timing out, since you miss the update that tells you the other side's preferred DERP server.
I think we are separately investigating connection stalls.
Do you mean that the client and workspace are reachable directly over UDP (no relays required)?
j
sorry, saw the thread too late
I've posted two screencasts with the behaviour I experience
The coder-HA-Setup is in the same network as the workspace target - there is no networking issue between them on the layer 2 path
s
what about the client that calls
coder ssh
? Is it also on the same network?
j
no
but it goes the very same path as the scp without coder
I've just verified, there are no stalls between the client machine and the coder-loadbalancer or the coder-service-vm either
s
you said "it says UDP is working" --- can you clarify what you mean by that?
j
If I do a coder ping -v I see the following netcheck:
Copy code
netcheck: [v1] report: udp=true v6=false v6os=true mapvarydest= hair= portmap= v4a=172.18.159.2:40307 derp=999 derpdist=999v4:27ms
I meant the "udp=true"
I do not see this stalls from another machine where the connection to the coder server is tunnled through ssh (and there netcheck reports udp=false - probably because teh STUN server cannot be reached)
s
thanks! also, what is the workspace and workspace host? --- e.g. a Pod on Kubernetes, or a VM on bare metal?
j
Its a sysbox container on docker on a debian stable bare-metal-machine
If you have any ideas how I can debug this further, please let me know
s
what would really help is
Copy code
coder -v speedtest your-workspace -t 60s
then create a GitHub issue and attach the output
that will give us detailed logging from the networking layer to see what might be correlated with the slowdowns
tag me on the issue, I'm @spikecurtis on Github
j
Is there anything sensitive in these logs?
If you need anything else or have any suggestions, just give me a ping
I've just checked from a machine on the same L2 network as the workspace and the coder components - so they should be able to use a p2p connection (at least if I use
coder ping
it switches to p2p after 2 sec) and I still see stalls - event thougl much less often:
Copy code
INTERVAL           THROUGHPUT          
0.00-1.08 sec      341.9350 Mbits/sec  
1.08-9.33 sec      34.5482 Mbits/sec   
9.33-27.50 sec     0.9234 Mbits/sec    
27.50-28.50 sec    436.1056 Mbits/sec  
28.50-29.54 sec    500.1755 Mbits/sec  
29.54-30.56 sec    478.0351 Mbits/sec  
30.56-31.59 sec    473.8567 Mbits/sec  
31.59-32.61 sec    376.7655 Mbits/sec  
32.61-59.47 sec    8.7444 Mbits/sec    
59.47-60.49 sec    479.2057 Mbits/sec  
60.49-61.59 sec    396.0868 Mbits/sec  
61.59-62.62 sec    405.9099 Mbits/sec  
62.62-63.63 sec    417.3954 Mbits/sec  
63.63-64.63 sec    450.1503 Mbits/sec  
64.63-65.64 sec    349.9535 Mbits/sec  
65.64-66.64 sec    434.3882 Mbits/sec  
66.64-67.66 sec    427.8146 Mbits/sec  
67.66-68.68 sec    414.3816 Mbits/sec  
68.68-69.69 sec    445.3199 Mbits/sec  
69.69-70.83 sec    398.2959 Mbits/sec  
70.83-71.85 sec    495.4804 Mbits/sec  
71.85-100.95 sec   4.6113 Mbits/sec    
100.95-129.92 sec  9.2683 Mbits/sec    
129.92-129.99 sec  479.7206 Mbits/sec  
---------------------------------------
0.00-129.99 sec    68.6650 Mbits/sec
c
are you able to share the logs from
coder ping -v
? it may have public ips so feel free to redact or dm
j
@colin (rust enthusiast) I attached them to the issue.
I would update to 23.1 tomorrow. I can postpone if you want more debug information.