https://livekit.io logo
Join Slack
Powered by
is there any way I can integrate custom noise canc...
# ask-ai
n
is there any way I can integrate custom noise cancellation in self hosted livekit?
t
Thanks for reaching out, Tareq. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
If you’re looking to add custom noise cancellation in your self-hosted LiveKit setup, you can totally do that using the 
AudioCustomProcessingDelegate
to build your own noise cancellation logic. While LiveKit Cloud comes with built-in noise cancellation powered by Krisp, this feature isn’t available out-of-the-box for self-hosted setups. But don’t worry—you can create a custom solution like this: 1. Build a Custom Audio Processor: ◦ Use the 
AudioCustomProcessingDelegate
protocol to craft a processor that applies noise cancellation to your audio buffers. ◦ For example, check out the 
LiveKitKrispNoiseFilter
class in the Swift SDK—it’s a great starting point for integrating a noise filter. 1. Set Up the Audio Processing Delegate: ◦ Register your custom processor with your audio track through the 
AudioManager
or 
LocalAudioTrack
class. ◦ In Swift, you can set the filter as the 
capturePostProcessingDelegate
and add it as a room delegate. 1. Keep These in Mind: ◦ Make sure your noise cancellation logic lives inside the 
audioProcessingProcess
method. ◦ Handle sample rate changes and audio buffer processing smoothly for the best results. Here’s a quick example in Swift to get you started:
Copy code
import LiveKit
import LiveKitKrispNoiseFilter

let filter = LiveKitKrispNoiseFilter()
let room = Room()

struct ContentView: View {
    var body: some View {
        VStack {
            Image(systemName: "globe")
                .imageScale(.large)
                .foregroundStyle(.tint)
            Text("Hello, world!")
        }
        .padding()
        .onAppear(perform: {
            Task {
                AudioManager.shared.capturePostProcessingDelegate = filter
                room.add(delegate: filter)
                try await room.connect(url: "", token: "")
                try await room.localParticipant.setMicrophone(enabled: true)
            }
        })
        .onDisappear(perform: {
            Task {
                await room.disconnect()
            }
        })
    }
}
Of course, you’re free to use other noise cancellation libraries or algorithms that suit your needs and integrate them in a similar way. If you want a hand with a specific algorithm or programming language, just give me a shout—I’m here to help! Sources: Knowledge Management System | livekit/livekit_composite | DeepWiki | Sources/LiveKitKrispNoiseFilter/LiveKitKrispNoiseFilter.swift | krisp-minimal/krisp-minimal/ContentView.swift
n
how to do it in python agents?
t
Hey there! If you’re looking to add custom noise cancellation in LiveKit using Python agents, here’s a simple step-by-step guide to help you out: 1. First, install the necessary package: 
pip install "livekit-plugins-noise-cancellation~=0.2"
2. Next, import the noise cancellation module: 
from livekit.plugins import noise_cancellation
3. Then, enable noise cancellation in your agent session: Just include the noise cancellation options in the 
RoomInputOptions
when you start your agent session, like this:
Copy code
await session.start(
    room=ctx.room,
    agent=MyAgent(),
    room_input_options=RoomInputOptions(
        noise_cancellation=noise_cancellation.BVC(),
    ),
)
Here are the models you can choose from: • `noise_cancellation.NC()`: This is the standard noise cancellation. • `noise_cancellation.BVC()`: Background voice cancellation, which is great if you’re working on voice AI apps. • `noise_cancellation.BVCTelephony()`: Optimized background voice cancellation for telephony applications. Here’s a quick example to get you started:
Copy code
from livekit.plugins import noise_cancellation

class MyAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="You are a voice assistant created by LiveKit.",
            stt=deepgram.STT(),
            llm=openai.LLM(),
            tts=cartesia.TTS(),
        )

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    session = AgentSession(
        vad=silero.VAD.load(),
    )

    await session.start(
        agent=MyAgent(),
        room=ctx.room,
        room_input_options=RoomInputOptions(
            noise_cancellation=noise_cancellation.BVC(),
        ),
    )
A couple of important tips: • Make sure to disable any client-side noise cancellation when you’re using server-side noise cancellation to avoid any conflicts. • The noise cancellation models work best with raw audio input, so try to keep client-side processing to a minimum. If anything’s unclear or you want to dive deeper, just let me know—I’m here to help!
2 Views
Open in Slack
PreviousNext
Powered by