https://livekit.io logo
Join Slack
Powered by
how to intercept the messages before sending it to...
# ask-ai
a
how to intercept the messages before sending it to the TTS?
t
Thanks for reaching out, Aditya. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
You can intercept and preprocess messages before sending them to the TTS (Text-to-Speech) service in LiveKit using different approaches depending on your LiveKit Agents version: • Agents v0.x: Use the 
before_tts_cb
callback when initializing your 
VoicePipelineAgent
. This callback lets you inspect or modify the text before it is sent to TTS.
Copy code
def replace_words(agent, text):
    # Modify text as needed
    return text.replace('example', 'demo')

agent = VoicePipelineAgent(
    ..., 
    before_tts_cb=replace_words,
)
• Agents v1.0+: Override the 
tts_node
method in a custom Agent subclass. You can preprocess streaming or non-streaming text here. For example:
Copy code
class CustomAgent(Agent):
    async def tts_node(self, text, model_settings):
        async def process_text():
            async for chunk in text:
                yield chunk.replace('foo', 'bar')
        return Agent.default.tts_node(self, process_text(), model_settings)
Both approaches allow for tasks such as text normalization, filtering, or pronunciation adjustments before TTS synthesis. For more advanced post-processing (like volume adjustments), the 
tts_node
method allows modifications to the resulting audio buffers as well. Refer to these mechanisms to ensure your logic runs at the correct place in the TTS pipeline. Sources: VoicePipelineAgent | LiveKit Docs | chain-of-thought-tts/README.mdoc | Agents 0.x migration guide | LiveKit Docs | pipeline-tts/tts_node.py | Voice pipeline nodes | LiveKit Docs
a
This is not working, is there any other way for this
t
Hey there! Let me break down some cool ways you can customize the LiveKit pipeline for your project. Think of these like behind-the-scenes magic tricks for tweaking how your agent communicates. • Override `llm_node`: Imagine you want to give your agent a little editorial control. This method lets you peek and modify what the language model is about to say before it becomes speech. Pretty neat, right? • Turn Completion Hook: This is like a quick backstage pass to adjust the conversation context right before your LLM jumps in with a response. Small tweaks can make a big difference! • Transcription Node: While mainly designed for showing what's been said, this hook gives you another lens to look at the conversation. If you're looking for the most reliable way to preprocess, I'd suggest customizing the 
tts_node
method (for Agents 1.0+) or using the 
llm_node
override. If something's not quite clicking, just let me know your Agent version and what you're trying to do, and we'll figure it out together! Sources: Voice pipeline nodes | LiveKit Docs | Agents 0.x migration guide | LiveKit Docs
a
Give me code for modifying the tts node.
t
Hey there! Let me walk you through a super handy way to customize the text-to-speech method in your LiveKit Agent. I've got a friendly code example that shows you exactly how to do this. 🤖
Copy code
from livekit.agents.voice import Agent, ModelSettings
from typing import AsyncIterable

class CustomAgent(Agent):
    async def tts_node(self, text: AsyncIterable[str], model_settings: ModelSettings):
        async def process_text():
            async for chunk in text:
                # Replace 'foo' with 'bar' in all TTS output
                modified_chunk = chunk.replace('foo', 'bar')
                yield modified_chunk
        # Use the default tts_node with your modified text stream
        return await Agent.default.tts_node(self, process_text(), model_settings)
Pro tip: You can swap out that 
chunk.replace('foo', 'bar')
line with whatever text processing magic you need. This method gives you the flexibility to modify or filter your text before it gets turned into speech. Pretty cool, right? 😎
Open in Slack
PreviousNext
Powered by