modern-leather-76107
11/24/2025, 8:26 PMasync def monitor_agent_responses():
"""Monitor LLM responses and switch TTS language/voice accordingly"""
last_message_id = None
while True:
try:
await asyncio.sleep(0.5) # Polling interval
state = graph.get_state(config={"configurable": {"thread_id": thread_id}})
if state and state.values.get('messages'):
messages = state.values['messages']
if messages:
last_msg = messages[-1]
# Check if it's a new AI message
if (hasattr(last_msg, 'id') and last_msg.id != last_message_id and
hasattr(last_msg, 'content') and last_msg.content and
not isinstance(last_msg, HumanMessage)):
last_message_id = last_msg.id
text = last_msg.content
# Detect language from AI response
if text and len(text.strip()) >= 3:
detected_results = detect(text, model='auto') # fast_langdetect
if detected_results and detected_results[0]['score'] >= 0.70:
lang_code = detected_results[0]['lang'].lower()
voice_id = LANGUAGE_VOICE_MAPPING.get(lang_code)
if lang_code != current_language["code"]:
<http://logger.info|logger.info>(f":earth_africa: Language switch: {current_language['code']} → {lang_code}")
tts.update_options(language=lang_code, voice_id=voice_id)
current_language["code"] = lang_code
except Exception as e:
logger.debug(f"Monitor error: {e}")
await asyncio.sleep(1)
# Started as background task
monitor_task = asyncio.create_task(monitor_agent_responses())
The problem:
The 0.5s polling interval creates a race condition — TTS often starts speaking before the monitor detects the language change, so the first sentence (or entire response) plays in the wrong voice/language.
What I’ve tried:
1. Wrapping/subclassing elevenlabs.TTS to intercept .stream() — failed because the Agent framework didn’t call my wrapped methods.
2. Monkey-patching tts.stream/tts.synthesize on the instance — methods weren’t invoked.
3. class swizzling (changing the instance’s class at runtime) — failed with class assignment: ‘MultilingualTTS’ object layout differs from ‘TTS’ because the ElevenLabs plugin has a fixed C-level layout.
4. Faster polling (150ms, 50ms) — helps but doesn’t eliminate the race, and increases CPU load.
My question:
Is there a cleaner way to hook into the TTS pipeline before audio generation starts? Ideally something like:
• A callback/event when Agent is about to synthesize text (so I can detect language and update TTS options first)
• A way to wrap or extend SynthesizeStream that the framework will actually use
• An event bus or signal when LLMAdapter yields a new chunk
Or is polling the LangGraph state the only viable approach here?
Setup:
• livekit-agents with Agent, AgentSession, langchain.LLMAdapter
• livekit-plugins-elevenlabs for TTS
• LangGraph for conversation state
• fast_langdetect for language detection
Thanks for any guidance! 🙏refined-appointment-81829
11/24/2025, 10:34 PMtts_node method receives an AsyncIterable[str] of text chunks from the LLM. tts_node.py:38-52 You can process this stream to detect language before forwarding to TTS:
async def tts_node(self, text: AsyncIterable[str], model_settings: ModelSettings):
async def process_text():
accumulated_text = ""
async for chunk in text:
accumulated_text += chunk
# Detect language once you have enough text
if len(accumulated_text.strip()) >= 20: # threshold
detected_lang = detect(accumulated_text)[0]['lang']
if detected_lang != self.current_language:
self.tts.update_options(
language=detected_lang,
voice_id=LANGUAGE_VOICE_MAPPING[detected_lang]
)
self.current_language = detected_lang
yield chunk
return Agent.default.tts_node(self, process_text(), model_settings)