This message was deleted.

cc @fancy-wire-61616 when he’s awake

@alert-apartment-84559 just trying to understand: are you trying to access a the raw RTP packets from the TURN servers? As far as I know, TURN would be passing through DTLS-encrypted packets. It doesn't seem possible to be able to decrypt the DTLS payload without the negotiated key. Are you decrypting it somehow first?

I’m trying to intercept the srtp packet, perform some modifications to the underlying video stream and then continue forwarding to the livekit sfu. I was able to unmarshall some of the srtp packet header. Essentially I’m trying to determine if it’s possible to expose the dtls keys to the turn server, via redis or say a ws endpoint and if it is, how inadvisable it is to do so.

Ah I see. that's an interesting path. it seems possible to expose DTLS keys via redis (so it remains private to your backend).

Another approach I've considered is using the peer connection interceptors, defined here using the interceptor api. Where I could define a relatively general plugin model. Are you aware whether these interceptors are post dtls termination?

yes of course! I should've thought of that. Interceptors should work for it. they work on the RTP stream (after DTLS termination).

SRTP does not encrypt RTP header and header extension. So you should be able to unmarshal without decryption. But a few things to keep in mind • DTLS should not be in the picture once SRTP keys are set up. RTP streams will use the SRTP context. But decryption should not be necessary to look at just header. • TURN will have STUN header. You will have to parse past that in TURN server to get to the packet. • The parsed values do not look correct. RTP version should be 2. • RTP header extension id is not fixed. Different browsers could use different id for the same extension. You will have to use SDP to get the right id. • Modifying the packet (something in the RTP header or extension) will break authentication check. RTP payload is encrypted. But the whole packet (including header) is authenticated. So, modifying RTP header/extension before SRTP auth check/decryption will break that.

Thanks for your responses @fancy-wire-61616 and @dry-elephant-14928. I'm getting more sane rtp header values with the packetdump interceptor (see example log below). Interceptors seem to be the most straightforward way to modify the payload without having to worry about SRTP auth. Initially I would plan to forward the rtp packet on a new track to an existing peer connection, following the pion rtp-to-webrtc example. Then set the interceptor output payload to the modified rtp payload, updating necessary headers. Enforcing, in livekit config and/or interceptor PeerConnection sdp appropriate codec/parameters.

livekit_server_1  | RTP PACKET:                               
livekit_server_1  |     Version: 2                            
livekit_server_1  |     Marker: false                         
livekit_server_1  |     Payload Type: 63                      
livekit_server_1  |     Sequence Number: 46521                
livekit_server_1  |     Timestamp: 3330104184                 
livekit_server_1  |     SSRC: 3676080904 (db1c8b08)           
livekit_server_1  |     Payload Length: 41

Participant info does not have all the information from SDP, for example things like RTP header extension ids are not available in participant info. The formatting part is an implementation you can add. The default one is the one you have shown above which just prints the RTP packet.

I think I should rephrase that between the server_sdk and the WebRtcConfig. I think I can get the header extensions in particular from the DirectionConfig but it does look like there could be separate configuration that might not be inferable. If not (and probably even if so), I imagine the peer connection is the better abstraction to be working with, rather than the raw rtp streams. And just implement the track routing as needed, based on some provided configuration.