https://livekit.io logo
Join Slack
Powered by
``` const session = new voice.AgentSession({ ...
# ask-ai
e
Copy code
const session = new voice.AgentSession({
        vad: vad,
        stt: new openai.STT({ model: 'whisper-1'}),
        llm: new openai.LLM({ model: 'gpt-4o-mini' }),
        tts: new cartesia.TTS({ 
          model: 'sonic-2', 
          voice: 'f786b574-daa5-4673-aa0c-cbe3e8534c02' 
        }),
        turnDetection: new livekit.turnDetector.MultilingualModel(),
      });
t
Thanks for reaching out, abhinav. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
e
i checked the logs, it does not show any transcript 
audioTranscript: ""
t
Could you please share a bit more about what you’re looking for help with regarding this code snippet? Are you running into an error, or would you like some guidance on understanding or tweaking it? I’m here to help!
e
i checked the logs, it does not show any transcript 
audioTranscript: ""
even when i am sending the audioe
t
I understand it can be a bit confusing when the 
audioTranscript
field remains empty even though you're sending audio. Let’s explore a few common reasons why this might happen: • Final Transcript Not Received: The 
audioTranscript
updates only when a 
FINAL_TRANSCRIPT
event comes through from the Speech-to-Text (STT) system. Interim transcripts are saved separately and don’t update this field. • STT Node Misconfiguration: It’s worth double-checking that the STT node is properly set up and enabled. If 
input_audio_transcription
isn’t enabled in your model settings, transcription won’t happen. • Empty or Invalid STT Response: Sometimes, if the STT engine returns an empty or invalid transcript, the field won’t update. • Audio Input Issues: Make sure the audio you’re sending matches the expected format, like the correct sample rate and channel count. Here’s a simple way to troubleshoot this: 1. Confirm that 
input_audio_transcription
is enabled in your model settings. 2. Check your logs to see if 
FINAL_TRANSCRIPT
events are coming through. 3. Ensure the STT node is properly initialized and is receiving audio frames. 4. Verify that the audio actually contains speech and matches the required format. If you try these steps and still run into issues, please don’t hesitate to reach out—I’m here to help you get this sorted! Sources: livekit-plugins/livekit-plugins-google/livekit/plugins/google/beta/realtime... | How to Set Up Speech-to-Text Input with Text-Only Output in LiveKit Agents | agents/src/transcription.ts
e
how to check 1
how to check input audio transcription is enabled
t
Let's take a moment to check if input audio transcription is enabled for you. You’ll want to look at the 
transcription_enabled
setting within the 
RoomOutputOptions
. This setting is what controls whether the speech-to-text (STT) transcriptions are sent as text streams to your frontend. Here’s a simple way to verify it: 1. Take a look at the 
RoomOutputOptions
configuration. If 
transcription_enabled
is set to 
True
or isn’t specifically set (since it defaults to 
True
), that means transcription output is turned on. 2. If you’re working with 
AgentSession
, you can also check the 
transcription_enabled
property inside the 
AgentOutput
component. For example:
Copy code
if session.output.transcription_enabled:
    print("Input audio transcription (output) is enabled")
else:
    print("Input audio transcription (output) is disabled")
If you have any questions or want me to walk you through this further, just let me know—I’m here to help! Sources: Text and transcriptions | LiveKit Docs | Text and transcriptions | LiveKit Docs | examples/voice_agents/toggle_io.py
e
i am using agent playground
t
In the LiveKit Agent Playground, input audio transcription is usually turned on by default, unless it’s been specifically turned off in the agent setup. Here’s a friendly guide to help you check: 1. Take a look at the UI: If you see spoken words showing up as text in the chat, that means transcription is working. The playground uses the 
lk.transcription
text stream topic to show these transcriptions. 2. Default Settings: Transcription is set to be on by default in the 
RoomOutputOptions
with 
transcription_enabled=True
. So, unless someone has turned it off, your audio input should be getting transcribed. 3. Agent Code Check: If you can peek into the agent’s code, look out for any places where 
set_transcription_enabled(False)
or 
transcription_enabled=False
might be set—that would mean transcription is disabled. 4. Custom Receiver Note: Sometimes, if audio output is off, you might not see transcriptions in the UI. In that case, you can add a custom text stream handler to confirm transcription is happening behind the scenes. If you have any questions or want me to walk you through any of these steps, just let me know—I’m here to help! Sources: examples/voice_agents/toggle_io.py | How to Set Up Speech-to-Text Input with Text-Only Output in LiveKit Agents | .env.example | Agents Playground | LiveKit Docs | Text and transcriptions | LiveKit Docs
e
[221127.561] ERROR (51749): Error running STT task: Error: STT does not support streaming, add a VAD to the AgentTask/VoiceAgent to enable streaming
t
It looks like the error is happening because the STT provider (like OpenAI Whisper) doesn’t support streaming directly. No worries though! To work around this, you can add a Voice Activity Detection (VAD) system alongside the STT using a 
StreamAdapter
. What this does is it buffers the audio until the VAD detects the end of someone speaking, and then it sends that audio for transcription. Here’s a simple example in Node.js to help you get started:
Copy code
import { openai, silero } from '@livekit/agents';

const whisperSTT = new openai.STT();
const vad = await silero.VAD.new();
const vadStream = vad.stream();
const stt = new STTStreamAdapter(whisperSTT, vadStream);
This way, even STT models that don’t support streaming can still be used smoothly in a streaming setup. If you’d like, I’m here to help you step through the implementation—just let me know! Sources: Speech-to-text (STT) integrations | LiveKit Docs | livekit-agents/livekit/agents/stt/stream_adapter.py
Open in Slack
PreviousNext
Powered by