https://livekit.io logo
Join Slack
Powered by
getting this error please help [21:48:16.721] ERR...
# ask-ai
e
getting this error please help [214816.721] ERROR (43913): Error running STT task: Error: STT does not support streaming, add a VAD to the AgentTask/VoiceAgent to enable streaming
t
Thanks for reaching out, abhinav. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
It looks like the error is happening because the STT provider you're using doesn’t support streaming input directly. No worries, though! We can fix this by adding a Voice Activity Detection (VAD) mechanism, which helps detect when someone is speaking and allows the audio to be processed in chunks. Here’s a simple way to get this working: 1. Try using a VAD like Silero to catch speech boundaries. For example:
Copy code
from livekit.plugins import openai, silero

whisper_stt = openai.STT()
vad = silero.VAD.load(min_speech_duration=0.1, min_silence_duration=0.5)
stt = agents.stt.StreamAdapter(whisper_stt, vad)
1. Make sure you pass both the STT and VAD when you initialize your agent:
Copy code
agent = Agent(
    stt=whisper_stt,
    vad=vad,
    # ... other options
)
This setup will hold onto the audio until the VAD detects the end of speech, making it easier for the STT engine to process everything smoothly. If you have any questions or want me to walk you through it, just let me know—I’m here to help! Sources: agents/src/stt/stt.ts | Speech-to-text (STT) integrations | LiveKit Docs | Working with plugins | LiveKit Docs
e
Copy code
const vad = ctx.proc.userData.vad! as silero.VAD;
      
      const assistant = new voice.Agent({
          instructions: 'You are a helpful voice AI assistant.',
      });
  
      const session = new voice.AgentSession({
        vad: await silero.VAD.load(),
        stt: new openai.STT({ model: 'gpt-4o-transcribe'}),
        llm: new openai.LLM({ model: 'gpt-4o-mini' }),
        tts: new cartesia.TTS({ 
          model: 'sonic-2', 
          voice: 'f786b574-daa5-4673-aa0c-cbe3e8534c02' 
        }),
        turnDetection: new livekit.turnDetector.MultilingualModel(),
      });
  
      await session.start({
        agent: assistant,
        room: ctx.room,
        inputOptions: {
          // For telephony applications, use `TelephonyBackgroundVoiceCancellation` for best results
          noiseCancellation: BackgroundVoiceCancellation(),
        },
      });
this is the code
t
Thank you for sharing your code! It looks like you’re on the right track. To help you get the Silero VAD and OpenAI STT working smoothly within your 
voice.AgentSession
, here are a few friendly pointers: 1. First, make sure to load the Silero VAD before creating your session. Here’s a quick example:
Copy code
const vad = await silero.VAD.load({
  minSpeechDuration: 0.1,
  minSilenceDuration: 0.5
});
1. Next, pass the loaded VAD and STT instances into your 
AgentSession
like this:
Copy code
const session = new voice.AgentSession({
  vad: vad,
  stt: new openai.STT({ model: 'gpt-4o-transcribe' }),
  llm: new openai.LLM({ model: 'gpt-4o-mini' }),
  tts: new cartesia.TTS({ 
    model: 'sonic-2', 
    voice: 'f786b574-daa5-4673-aa0c-cbe3e8534c02' 
  }),
  turnDetection: new livekit.turnDetector.MultilingualModel(),
});
1. Finally, start the session and connect it to your room like this:
Copy code
await session.start({
  agent: assistant,
  room: ctx.room,
  inputOptions: {
    noiseCancellation: BackgroundVoiceCancellation(),
  },
});
If you run into any bumps or have questions along the way, please don’t hesitate to reach out. I’m here to help! Sources: plugins/silero/src/vad.ts | Silero VAD plugin | LiveKit Docs | Turn detection and interruptions | LiveKit Docs
Open in Slack
PreviousNext
Powered by