https://livekit.io logo
Join Slack
Powered by
LiveKit not passing all text from an LLM to the TT...
# ask-ai
b
LiveKit not passing all text from an LLM to the TTS
t
Thanks for reaching out, Boris. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I can help troubleshoot why LiveKit isn’t passing the full LLM text to TTS. Quick acknowledgement: I’m responding about the LLM→TTS pipeline where some generated text is missing from speech. To pinpoint the cause I need a bit more info — please provide any that apply: • Are you using a realtime LLM (Realtime API) or a streaming STT→LLM→TTS pipeline? • Do you have custom 
llm_node
or 
tts_node
implementations (yes/no)? If yes, paste the relevant snippets. • Example: the exact expected LLM text vs. what actually gets sent to TTS (or logs showing chunked outputs). • Any errors, timeouts, or cancellations in logs when the response finishes? • Are you using sentence-based flushing or any delimiters before calling TTS? • Which TTS engine are you using? In the meantime, try these quick checks you can run now: • Test with default `llm_node`/`tts_node` (no customization) to see if the full text is synthesized. • Enable verbose logging and verify each LLM chunk is yielded and forwarded to TTS. • If using a realtime LLM, confirm you set modalities to 
["text"]
and are not letting the realtime model bypass TTS. • Check for premature stream termination (timeouts/cancel calls) and for code that filters or returns early from the LLM stream loop. • If you buffer by sentence, ensure final partial sentences are flushed to TTS. Send the answers/snippets above and one short example of expected vs actual output and I’ll give focused fixes and code edits to resolve it. Sources: Pipeline nodes and hooks | LiveKit docs | Modifying LLM output before TTS | LiveKit docs | LLM Output Replacement | LiveKit docs | Live Conversations with AI using ChatGPT and WebRTC | Realtime model plugins | LiveKit Docs
b
STT→LLM→TTS pipeline No customisation In this example, the "J'ai beaucoup travaillé en tant que premier." didnt appear in the audio. Only the last part was said out loud 113220.414 DEBUG livekit.agents received user preflight transcript {"user_transcript": "J'ai beaucoup travaillé en tant que premier.", "language": "en"} 113221.067 DEBUG livekit.agents received user transcript {"user_transcript": "J'ai beaucoup travaillé en tant que premier.", "language": "en", "transcript_delay": 0.04635167121887207} 113223.480 DEBUG livekit.agents flush audio emitter due to slow audio generation 113229.576 DEBUG livekit.agents received user preflight transcript {"user_transcript": "Mais je travaillais à mon compte. Donc je mettais les annonces en ligne et les gens m'appelaient pour que j'arrive chez eux", "language": "en"} 113229.580 DEBUG livekit.agents received user transcript {"user_transcript": "Mais je travaillais à mon compte. Donc je mettais les annonces en ligne et les gens m'appelaient pour que j'arrive chez eux.", "language": "en", "transcript_delay": 0.0593411922454834} 113230.544 DEBUG livekit.agents received user preflight transcript {"user_transcript": "Pour intervenir.", "language": "en"} 113230.749 DEBUG livekit.agents received user transcript {"user_transcript": "Pour intervenir.", "language": "en"} 113237.110 DEBUG livekit.agents flush audio emitter due to slow audio generation 113250.168 INFO livekit.agents shutting down worker {"id": "unregistered"}
t
Thanks — I’m looking at the STT→LLM→TTS pipeline logs you shared and responding about the “flush audio emitter due to slow audio generation” entries and the missing phrase. Diagnosis (what’s happening) • The logs show the pipeline flushed the audio emitter due to slow audio generation. When that flush occurs, pending synthesized audio can be discarded, which explains why the earlier phrase ("J'ai beaucoup travaillé en tant que premier.") never reached the speaker and only the final part was spoken. What to try next (short actionable steps) 1. Enable preemptive generation • Set 
preemptive_generation=True
on your 
AgentSession
so TTS can start synthesizing earlier and reduce backlog. 1. Use or verify a streaming-capable / low-latency TTS • Switch to or confirm a low-latency TTS provider (e.g., Cartesia Sonic) so audio frames are produced faster and flushing is less likely. 1. Confirm chunked/sentence streaming is active • Ensure the agent sends text to TTS incrementally (sentence or chunk boundaries) rather than one large block. This prevents long waits for the first audio frame. 1. Capture TTS input for debugging • Temporarily log what text segments are passed into TTS to confirm whether the missing phrase is reaching the TTS node:
Copy code
async def tts_node(self, text: AsyncIterable[str], model_settings):
    async for segment in text:
        print("TTS input segment:", repr(segment))
        async for frame in Agent.default.tts_node(self, [segment], model_settings):
            yield frame
1. Check for timeouts / worker shutdown • Your logs show a worker shutdown after flushing. Verify no timeout/cancel is closing the session before all audio is generated. 1. Upgrade packages • Upgrade 
livekit-agents
and your TTS plugin to the latest versions; flushing heuristics and TTS performance have improved in recent releases. If you want, send: • Which TTS engine/plugin you’re using • Your 
AgentSession
config (especially 
preemptive_generation
value) • A short trace showing the LLM output chunks (if available) I’ll give the exact code/config suggestion based on those details. Sources: Agent speech and audio | LiveKit docs | Modifying LLM output before TTS | LiveKit docs | Text-to-speech (TTS) models | LiveKit Docs
2 Views
Open in Slack
PreviousNext
Powered by