Is it posible to do VAD or noise cancellation with inference LiveKit Community #inference

Is it posible to do VAD or noise cancellation with...

quaint-waitress-91864

10/02/2025, 8:06 PM

Is it posible to do VAD or noise cancellation with inference. I used to use the silero plugin but it seems like it's not working when I switched to inference.

rough-gpu-50664

10/03/2025, 5:20 AM

Hmm, vad and turn-taking are handled at the agent, not at the inference service. I wouldn't expect to see an influence on your vad by switching from plugin to livekit inference. would you mind sharing your

AgentSession

code? I'd like to see what models and params you're using. If you're not comfortable sharing it publicly, would you please DM me the code?

quaint-waitress-91864

10/03/2025, 5:19 PM

Copy code

session = AgentSession(
            stt="deepgram/nova-3:en",
            llm="google/gemini-2.5-flash",
            tts="elevenlabs/eleven_flash_v2_5",
            turn_detection=MultilingualModel(),
            vad=ctx.proc.userdata["vad"]
        )

quaint-waitress-91864

10/03/2025, 5:20 PM

Copy code

def prewarm(proc: agents.JobProcess):
    proc.userdata["vad"] = silero.VAD.load(activation_threshold=0.6)

quaint-waitress-91864

10/03/2025, 5:22 PM

@rough-gpu-50664 is it counterproductive for me to use the silero VAD then? I initially added the silero VAD with Groq STT (no inference, just plugin) and it was super useful. But upon switching to deepgram nova-3 (and inference), I noticed that background noise isn't as reliably blocked off. Then, when I did the same model combination but without inference, the background noise cancellation seemed to be fine. Of course there's no concrete numbers I can point to which benchmark this, it's just what I noticed after a few tests. Wondering if you have any thoughts.

rough-gpu-50664

10/05/2025, 3:26 AM

Copy code

is it counterproductive for me to use the silero VAD then? I initially added the silero VAD with Groq STT (no inference, just plugin) and it was super useful.

Your AgentSession looks good. You should have the VAD and in the AgentSession. Our turn detection model works with the VAD. Were you previously using our turn detection model (i.e.,

MultilingualModels()

) in your Groq STT implementation? I'm wondering if you're seeing the difference between using just VAD for turn taking and using VAD+LK-turn-detection for turn taking.

Copy code

But upon switching to deepgram nova-3 (and inference), I noticed that background noise isn't as reliably blocked off. Then, when I did the same model combination but without inference, the background noise cancellation seemed to be fine.

This is unexpected. Background noise cancellation is applied before the audio reaches the STT. When you listen to the audio recording when using the plugins (i.e.,

stt=deepgram.STT(model="nova-3")

), you hear less background noise compared to the recording when using inference (

stt="deepgram/nova-3"

Open in Slack

Previous Next