https://livekit.io logo
Join Slack
Powered by
5-07-27 15:15:47,485 - ERROR livekit.agents - Erro...
# ask-ai
b
2025-07-27 151547,485 - ERROR livekit.agents - Error in _stt_task Traceback (most recent call last): File "/root/miniconda3/envs/mm/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs return await fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/mm/lib/python3.12/site-packages/livekit/agents/voice/audio_recognition.py", line 408, in _stt_task async for ev in node: File "/root/miniconda3/envs/mm/lib/python3.12/site-packages/livekit/agents/voice/agent.py", line 313, in stt_node raise RuntimeError( RuntimeError: The STT (livekit.plugins.openai.stt.STT) does not support streaming, add a VAD to the AgentTask/VoiceAgent to enable streamingOr manually wrap your STT in a stt.StreamAdapter If I use this agent definition: session = AgentSession( stt=agent_config.get_stt_service(), llm=agent_config.get_llm_service(), tts=agent_config.get_tts_service(), turn_detection="manual", ) I get the error above, I need to add vad=silero.VAD.load() to make it work. How can I make it so the STT processes the spoken words only when the @ctx.room.local_participant.register_rpc_method("end_turn") is called? Or in general when the user turn ends? What are the pros and cons of doing it like this?
t
Thanks for reaching out, aile. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To configure your STT to process spoken words only when the 
end_turn
method is called, you can use manual turn detection. This approach gives you full control over when audio is captured and processed. Below are the steps to achieve this: 1. Set Manual Turn Detection Configure your 
AgentSession
with manual turn detection:
Copy code
session = AgentSession(
    turn_detection="manual",
    stt=agent_config.get_stt_service(),
    llm=agent_config.get_llm_service(),
    tts=agent_config.get_tts_service(),
)
2. Disable Audio Input Initially Start with audio input disabled to prevent STT from processing audio automatically: 
session.input.set_audio_enabled(False)
3. Implement RPC Methods for Turn Control Set up RPC methods to control when audio capture starts and when STT processing occurs:
Copy code
@ctx.room.local_participant.register_rpc_method("start_turn")
async def start_turn(data: rtc.RpcInvocationData):
    session.interrupt()  # Stop any current agent speech
    session.clear_user_turn()  # Clear any previous input
    session.input.set_audio_enabled(True)  # Start listening

@ctx.room.local_participant.register_rpc_method("end_turn")
async def end_turn(data: rtc.RpcInvocationData):
    session.input.set_audio_enabled(False)  # Stop listening
    session.commit_user_turn(transcript_timeout=10.0)  # Process the input
The 
transcript_timeout
parameter ensures the system waits for the final transcript before processing. 4. Optional: Add a Cancel Method You can also implement a method to discard audio without processing:
Copy code
@ctx.room.local_participant.register_rpc_method("cancel_turn")
async def cancel_turn(data: rtc.RpcInvocationData):
    session.input.set_audio_enabled(False)  # Stop listening
    session.clear_user_turn()  # Discard the input
Pros and Cons of This Approach Pros: • Full control over when STT processes audio. • Avoids unnecessary processing during user pauses or background noise. • Allows integration with custom logic for turn detection. Cons: • Requires additional implementation effort. • May introduce slight delays if not managed efficiently. This setup ensures that STT only processes spoken words when you explicitly call the 
end_turn
RPC method, providing precise control over speech-to-text conversion. Sources: Turn detection and interruptions | LiveKit Docs | Turn detection and interruptions | LiveKit Docs | examples/voice_agents/push_to_talk.py
b
Ok but like this if I do not provide a vad and I put turn_detection="manual" I get: 2025-07-27 151547,485 - ERROR livekit.agents - Error in _stt_task Traceback (most recent call last): File "/root/miniconda3/envs/mm/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs return await fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/mm/lib/python3.12/site-packages/livekit/agents/voice/audio_recognition.py", line 408, in _stt_task async for ev in node: File "/root/miniconda3/envs/mm/lib/python3.12/site-packages/livekit/agents/voice/agent.py", line 313, in stt_node raise RuntimeError( RuntimeError: The STT (livekit.plugins.openai.stt.STT) does not support streaming, add a VAD to the AgentTask/VoiceAgent to enable streamingOr manually wrap your STT in a stt.StreamAdapter
t
The error you're experiencing is totally normal, and I'm here to help you solve it! 👋 It looks like your speech-to-text (STT) provider isn't quite ready for streaming, and we need to add a little magic to make everything work smoothly. ### Solution 1: Add VAD to Your Agent Configuration No worries! The easiest fix is to add a Voice Activity Detection (VAD) to your agent setup. Think of it like giving your agent a helpful ear that knows exactly when to listen. Here's how:
Copy code
python from livekit.plugins import openai, silero # Create VAD vad = silero.VAD.load( min_speech_duration=0.1, min_silence_duration=0.5, ) # Create agent with VAD agent = Agent( instructions="Your agent instructions", stt=openai.STT(), # Non-streaming STT vad=vad, # VAD for handling non-streaming STT # ... other configurations )
### Solution 2: Use a StreamAdapter If VAD isn't your thing, no problem! We can use a 
StreamAdapter
to handle audio buffering. It's like giving your STT a smart assistant that manages the audio flow:
Copy code
python from livekit import agents from livekit.plugins import openai, silero # Create STT and VAD whisper_stt = openai.STT() vad = silero.VAD.load( min_speech_duration=0.1, min_silence_duration=0.5, ) # Wrap STT in StreamAdapter stt = agents.stt.StreamAdapter(whisper_stt, vad.stream())
### Solution 3: Use a Streaming-Capable STT Want an even simpler solution? Just switch to an STT provider that's already streaming-friendly, like Deepgram. It's like upgrading to a more flexible communication tool! ### Why This Happens Here's the scoop: your system noticed that the current STT provider doesn't support streaming, and there's no VAD to help manage the audio. It's basically saying, "Hey, I need a bit more help to process this audio smoothly!" Need more help? I'm all ears! Just reach out, and we'll get this sorted together. 😊 Sources: Speech-to-text (STT) integrations | LiveKit Docs | livekit-plugins/livekit-plugins-spitch/livekit/plugins/spitch/stt.py
2 Views
Open in Slack
PreviousNext
Powered by