Hey team, I have identified an issue with Livekit SIP where audio breaks occur during interruptions. This problem is specific to SIP calls and doesn't happen with other connection types. After investigating, I have traced the root cause is the mixer component in LiveKit SIP. Any help would be greatly appreciated.

I have traced the root cause is the mixer component in LiveKit SIP. Any help would be greatly appreciated.

Can you tell me more about this, how you traced it and what is wrong with the mixer?

When the user interrupts the call, we stop sending audio to LiveKit and clear the audio source queue, but the audio breaks at the end during the interruption. I only faced this issue with SIP telephony calls, so I set up LiveKit and the SIP service locally and debugged it. I found that the audio works properly before sending it to the mixer, but once it goes through the mixer, it starts cracking/breaking. In my use case, there is only one standard participant and one SIP participant, so I temporarily removed the mixer from the code, and the issue disappeared, and the audio started working fine. I am still digging deeper to find the root cause in the mixer, but it is definitely related to that.

@bored-accountant-30277 ^

Thanks @helpful-raincoat-87696. I'm going to bring it up with the team.

Thanks @bored-accountant-30277, I will also try to debug the issue in the mean time

Any update here? @bored-accountant-30277

Tagging @rough-pilot-96201 who might have some thoughts on this.

@helpful-raincoat-87696, this is great input! Generally speaking the mixer is there for a purpose, but you're right in the sense that if we only have 1-on-1 connections, it serves no function other than being there in case another track gets published from whichever participant. More to the point, would you be able to share exactly where you snipped the mixer out of the loop here?

We are working on voice AI agents and using LiveKit + LiveKit SIP as a transport. When the agent is speaking and the user interrupts the agent (during TTS playback), we stop sending audio chunks to the audio source and clear the audio source queue. Now, during the final second of the interruption, the audio breaks/cracks on telephony. While debugging, I recorded the input audio to the mixer and the output audio from the mixer. The input audio was perfect, but the output audio had that crackling sound during the last second.

@helpful-raincoat-87696 could you share the diff of removing the mixer code that caused the issues to go away ?

In the

ParticipantCallback

inside

room.go

, when

OnTrackSubscribed

is triggered, we currently create a new track in a goroutine using

mTrack := r.NewTrack()

. I’ve updated the

NewTrack

method so that instead of sending audio to the mixer, it now sends the audio directly to the room’s output.

bumping this @bored-accountant-30277

Thanks for the clarification on where the shortcut has happened. Would you also be able to clarify what the following means, exactly?

we stop sending audio to LiveKit and clear the audio source queue.

Specifically, if you are generating audio and then subsequently dropping that audio, that might be something to look at.

When the user interrupts the agent, the voice agent orchestrator stops sending audio chunks to the audio source by halting

audio_source.capture_frame(audio_frame)

, and then clears any pending audio by calling

audio_source.clear_queue()

.

Would

audio_source.clear_queue()

not risk dropping audio that is generated by the agent, or does that happen well after agent stops generating voice?

Yes it happens after the agent stops generating voice