https://livekit.io logo
Join Slack
Powered by
I want to stop background noise once the user has ...
# ask-ai
q
I want to stop background noise once the user has started talking
t
Thanks for reaching out, Harsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I understand you want background noise to be suppressed automatically once the user starts talking. You can accomplish that by using Background Voice Cancellation (BVC) together with Voice Activity Detection (VAD) in LiveKit Agents. BVC isolates the active speaker and filters background sounds and other voices; VAD tells the system when the user has started speaking so the filter can apply appropriately. Quick summary of what to do • Use BVC to filter background noise and non-primary speakers. • Use a VAD model (for example Silero VAD) to detect when the user starts speaking. • Configure the agent session so audio input runs through BVC (or BVCTelephony for SIP) and turn detection uses VAD. Example (Python) — shows how to enable BVC and VAD in an AgentSession:
Copy code
from livekit.agents import AgentSession, room_io
from livekit.plugins import noise_cancellation, silero

session = AgentSession(
    vad=silero.VAD.load(),
    turn_detection="vad",
    stt=...,  # your STT provider
    llm=...,
    tts=...,
)

await session.start(
    room=ctx.room,
    agent=my_agent,
    room_options=room_io.RoomOptions(
        audio_input=room_io.AudioInputOptions(
            noise_cancellation=noise_cancellation.BVC(),  # enable BVC
        ),
    ),
)
If your participant is a SIP/telephony endpoint, use 
BVCTelephony()
instead:
Copy code
filter = (
    noise_cancellation.BVCTelephony()
    if participant.kind == rtc.ParticipantKind.PARTICIPANT_KIND_SIP
    else noise_cancellation.BVC()
)
Notes and best practices • BVC runs on the agent side in real time and adds minimal latency. • Don’t chain multiple noise-cancellation solutions (e.g., BVC plus another NC service) — that can degrade audio quality. • BVC will suppress background noise and other non-primary voices while the primary user speaks. To tailor this for your setup • Which platform/type are you integrating with? (WebRTC/browser, SIP/telephony trunk, iOS/Android client, or a server-side participant?) • Are you already using a specific VAD or STT provider, or would you like recommendations? • Do you need code for a different language or framework (JS, iOS Swift, etc.)? Tell me the platform and any existing components (VAD/STT/agent code) and I’ll provide a concrete integration snippet and any platform-specific notes. Sources: Choosing the right noise cancellation for your agent | LiveKit
q
no the background audio, I want to turn if off once the user starts speaking
t
Thanks — I understand you want the background audio track stopped (muted) as soon as the user starts speaking. You can do that by listening for the AgentSession user state change (detect when the user transitions to "speaking") and then stopping or muting the BackgroundAudioPlayer. This ensures the ambient/background audio is turned off while the user speaks and can optionally resume afterward. Before I give an exact code snippet for your setup, which platform / SDK are you using? • JavaScript / TypeScript (Node or browser) • Python (server-side agent) • iOS (Swift) • Android (Kotlin/Java) • Or a SIP/telephony integration Also tell me: • Are you using the built-in BackgroundAudioPlayer, or are you playing background audio another way (e.g., a custom audio track, media element in the browser, or SIP music-on-hold)? • Do you want the background audio to automatically resume when the user stops speaking? Once you tell me the platform and those details, I’ll provide a concise code example and any platform-specific notes. Sources: Turns overview | LiveKit Documentation | Agent speech and audio | LiveKit Documentation
q
I am using python sdk
t
Thanks — great, since you're using the Python SDK I’ll show a focused example that listens for the user turning to the "speaking" state and stops the BackgroundAudioPlayer. This will mute/stop the ambient audio immediately when the user starts talking and optionally restart it when they stop. Example (Python) 1. Start an AgentSession with a VAD (optional but recommended for accurate turn detection). 2. Create and start a BackgroundAudioPlayer. 3. Listen for 
user_state_changed
and call 
background_audio.stop()
when the user starts speaking; call 
background_audio.play()
or 
background_audio.start()
when they stop if you want it to resume.
Copy code
from livekit.agents import AgentSession, BackgroundAudioPlayer
from livekit.plugins import silero
import asyncio

# Replace with your actual agent and room context usage
async def entrypoint(ctx, my_agent):
    # Load a VAD for reliable turn detection (optional but recommended)
    vad = silero.VAD.load()

    session = AgentSession(
        vad=vad,
        turn_detection="vad",  # use VAD for turn detection
        # stt=..., llm=..., tts=...  # your other agent components
    )

    # Create the background audio player with an ambient file or stream
    background_audio = BackgroundAudioPlayer(ambient_sound="path/to/ambient.mp3")

    # Start the background audio and attach it to the room/agent session
    await background_audio.start(room=ctx.room, agent_session=session)

    # React to user state changes
    @session.on("user_state_changed")
    def _on_user_state_changed(ev):
        # ev.new_state values include "speaking", "listening", etc.
        if ev.new_state == "speaking":
            # Stop background audio immediately when user starts talking
            try:
                background_audio.stop()
            except Exception:
                # handle or log stop error if needed
                pass
        elif ev.new_state == "listening":
            # Optionally resume background audio when user stops speaking
            try:
                background_audio.play()
            except Exception:
                # handle or log play error if needed
                pass

    # Start the agent session (blocks until session ends)
    await session.start(agent=my_agent, room=ctx.room)
Key notes and recommendations • Use VAD-based turn detection for the fastest, most reliable "user started speaking" signal. • 
background_audio.stop()
removes the background audio track so it doesn't interfere with the user's speech; 
background_audio.play()
(or 
start()
depending on your use) resumes it. • Wrap the stop/play calls with error handling if your environment may return async errors. • If you prefer muting instead of stopping (so state is preserved), you can implement a mute flag in your BackgroundAudioPlayer wrapper and call 
background_audio.set_volume(0)
when speaking and restore volume after; the SDK methods above stop/play the track directly. Would you like: • A version that uses muting (volume control) instead of stop/play? • A full runnable example showing agent setup (stt/llm/tts) and room context?
4 Views
Open in Slack
PreviousNext
Powered by