How to improve TTS naturalness by controlling text...
# ask-aij
jolly-eve-3916
07/30/2025, 5:38 AMHow to improve TTS naturalness by controlling text chunking in AgentSession?
We're experiencing stilted/unnatural TTS speech with our voice agent using AgentSession.
The issue appears to be that text is being sent to TTS in very small chunks, causing
unnatural pauses and intonation.
Current setup:
session = AgentSession(
llm=anthropic.LLM(model="claude-sonnet-4-20250514"),
tts=deepgram.TTS(model="aura-2-cora-en"),
# ... other settings
)
What we're seeing in logs:
Text is chunked unnaturally:
' course! I hear'
' you. I'
' without adding those'
' pauses an'
'\n\nWhat'
This creates:
1. Unnatural pauses between chunks
2. Stilted speech patterns
3. Deepgram STT interpreting fragments as "DOT"
Questions:
1. Does AgentSession support sentence_tokenizer parameter like VoicePipelineAgent does?
2. If not, what's the recommended way to buffer text into complete sentences before
sending to TTS?
3. Is there a way to control the LLM streaming chunk size to get more natural sentence
boundaries?
4. Should we migrate to VoicePipelineAgent for better sentence handling, or is there a way
to achieve this with AgentSession?
We need the text to be buffered into complete sentences before TTS synthesis to create
natural-sounding speech.
Using livekit-agents 1.2.2
t
tall-airline-16756
07/30/2025, 5:39 AMThanks for reaching out, Albert. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
07/30/2025, 5:40 AMTo improve TTS naturalness and control text chunking in AgentSession, here are the key points and recommendations:
1. Sentence Tokenization in AgentSession:
AgentSession does not directly support a
sentence_tokenizertts_nodetts_node Copy code
from livekit.agents.voice import Agent
from livekit.agents import tokenize
class SentenceBufferedAgent(Agent):
async def tts_node(self, text: AsyncIterable[str], model_settings: ModelSettings):
async def process_with_sentence_buffering():
buffer = ""
sentence_tokenizer = tokenize.blingfire.SentenceTokenizer(
min_sentence_len=15,
retain_format=False
)
async for chunk in text:
buffer += chunk
# Tokenize accumulated text
sentences = sentence_tokenizer.tokenize(buffer)
# Yield complete sentences, keep incomplete text in buffer
if len(sentences) > 1:
for sentence in sentences[:-1]:
yield sentence + " "
buffer = sentences[-1] # Keep last incomplete sentence
# Yield any remaining text
if buffer.strip():
yield buffer
return Agent.default.tts_node(self, process_with_sentence_buffering(), model_settings)4 Views