Can I control the TTS to not speak certain words, like skipping the input when it contains numbers and only speaking the text?

Thanks for reaching out, Yassine. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

but the VoicePipelineAgent is the old version

Hey there! In the latest version of LiveKit, we've swapped out the

before_tts_cb

callback for a more flexible

tts_node

system. This gives you the power to tweak the TTS pipeline by adjusting the text before it gets synthesized. Here’s a friendly example to show you how you can skip certain words or numbers:

async def skip_words_tts_node(text: str | AsyncIterable[str]) -> str | AsyncIterable[str]:
    if isinstance(text, str):
        # Skip specific words or numbers in non-streaming text
        return text.replace("specific_word", "").replace("123", "")
    else:
        # Skip specific words or numbers in streaming text
        async def process_stream():
            async for chunk in text:
                processed_chunk = chunk.replace("specific_word", "").replace("123", "")
                yield processed_chunk
        return process_stream()

agent = VoicePipelineAgent(
    vad=silero.VAD.load(),
    stt=openai.STT(),
    llm=openai.LLM(),
    tts=openai.TTS(),
    tts_node=skip_words_tts_node,  # Add your TTS node callback here
)

This setup works great with both non-streaming and streaming text inputs. If you want me to walk you through it more or need extra examples, just let me know—I’m here to help!

), stt=deepgram.STT(model="nova-2-general", language="fr"), llm=groq.LLM(model="llama3-8b-8192"), tts=cartesia.TTS(model="sonic-2", voice="8832a0b5-47b2-4751-bb22-6a8e2149303d", language="fr"), turn_detection=MultilingualModel(), tts_node=skip_numbers_and_symbols, ) i use it like this but i get this error d {"pid": 33793, "job_id": "AJ_2FgDXoXpaUEQ"} 2025-08-04 044657,617 - ERROR livekit.agents - unhandled exception while running the job task Traceback (most recent call last): File "/home/client/mohamed_project/voice-pipeline-agent-python/venv/lib/python3.12/site-packages/opentelemetry/util/_decorator.py", line 71, in async_wrapper return await func(*args, **kwargs) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/client/mohamed_project/voice-pipeline-agent-python/venv/lib/python3.12/site-packages/livekit/agents/ipc/job_proc_lazy_main.py", line 240, in _traceable_entrypoint await self._job_entrypoint_fnc(job_ctx) File "/home/client/mohamed_project/voice-pipeline-agent-python/agent.py", line 113, in entrypoint agent=Assistant(), ^^^^^^^^^^^ File "/home/client/mohamed_project/voice-pipeline-agent-python/agent.py", line 40, in init super().__init__(

Hey there! It looks like the unhandled exception error you ran into might be due to one of these common hiccups: • Mismatched TTS Settings: Double-check that your TTS setup (like SSML, streaming, or other settings) is valid and works well with the TTS provider you’re using. • Invalid Custom TTS Parameters: Make sure all your parameters (such as model, voice, language) are correctly set and fall within what your TTS provider allows. • Uncaught Errors in TTS Processing: If the TTS provider throws an exception during synthesis, be sure your code is catching and handling it properly. • Issues in Custom Text/Audio Processing: Take another look at your custom

tts_node

logic to spot any errors in how text or audio is being processed. Here are a few friendly tips to help you troubleshoot: 1. Make sure the

skip_numbers_and_symbols

function is implemented correctly and isn’t causing any unexpected errors. 2. Check that your TTS provider (like Cartesia) supports the model, voice, and language settings you’ve chosen. 3. Look out for any extra error logs or stack traces—they often give helpful clues about what’s going wrong. 4. Try wrapping your custom

tts_node

logic in a try-except block to catch and log any surprises during execution. If you’d like, feel free to share more details about the error or how

skip_numbers_and_symbols

is set up—I’m here to help you get this sorted out!

i'm using cartesia and i want it to skip the number when sepeak

Hey there! If you want Cartesia TTS to skip numbers when it’s speaking, a handy way is to clean up the text first by removing those numbers before sending it over to the TTS service. Here’s a simple example to show you how:

import re

def remove_numbers(text):
    """Remove all digits from the text."""
    return re.sub(r'\d+', '', text)

text = "Please send $100 to account 12345."
filtered_text = remove_numbers(text)

After you’ve taken out the numbers, just pass the cleaned-up text to the Cartesia TTS plugin in LiveKit like this:

from livekit.plugins import cartesia
import asyncio

async def main():
    tts = cartesia.TTS(model="sonic-2", voice="8832a0b5-47b2-4751-bb22-6a8e2149303d", language="fr")

    # Process the text to remove numbers
    text = "Please send $100 to account 12345."
    filtered_text = remove_numbers(text)

    # Send the processed text to TTS
    stream = tts.stream()
    await stream.push_text(filtered_text)
    await stream.flush()
    await stream.end_input()

asyncio.run(main())

This way, the TTS will smoothly skip over any numbers when speaking. If you have any questions or need a hand with anything else, just give me a shout!

stt=deepgram.STT(model="nova-2-general", language="fr"), llm=groq.LLM(model="llama3-8b-8192"), tts=cartesia.TTS(model="sonic-2", voice="8832a0b5-47b2-4751-bb22-6a8e2149303d", language="fr"), turn_detection=MultilingualModel(), tts_node=skip_numbers_and_symbols,# TTS node that filters out numbers and special characters from TTS streaming output async def skip_numbers_and_symbols(text_stream: AsyncIterable[str]) -> AsyncIterable[str]: async for chunk in text_stream: cleaned = re.sub(r"[\d\|]+|[^\w\sÀ-ÿ]", "", chunk) yield cleaned can you update my code to use the rght implimentation

Hey there! I've updated your code to make sure it filters out numbers and special characters from the TTS streaming output just like you wanted:

import re
from typing import AsyncIterable

async def skip_numbers_and_symbols(text_stream: AsyncIterable[str]) -> AsyncIterable[str]:
    """Filters out numbers and special characters from the TTS streaming output."""
    async for chunk in text_stream:
        # Remove numbers and special characters, keeping letters, spaces, and accented characters
        cleaned = re.sub(r"[\d\|]+|[^\w\sÀ-ÿ]", "", chunk)
        yield cleaned

# Example usage in your agent setup
from livekit.plugins import cartesia
from livekit.agents.pipeline import VoicePipelineAgent
from livekit.plugins import deepgram, groq

agent = VoicePipelineAgent(
    stt=deepgram.STT(model="nova-2-general", language="fr"),
    llm=groq.LLM(model="llama3-8b-8192"),
    tts=cartesia.TTS(model="sonic-2", voice="8832a0b5-47b2-4751-bb22-6a8e2149303d", language="fr"),
    turn_detection=MultilingualModel(),
    tts_node=skip_numbers_and_symbols,  # Use the updated TTS node here
)

This way, your TTS output will keep the letters, spaces, and accented characters, but filter out those pesky numbers and special symbols. If you want me to tweak anything else or if you have questions, just let me know—I'm here to help!

33793, "job_id": "AJ_2FgDXoXpaUEQ"} 2025-08-04 044657,617 - ERROR livekit.agents - unhandled exception while running the job task Traceback (most recent call last): File "/home/client/mohamed_project/voice-pipeline-agent-python/venv/lib/python3.12/site-packages/opentelemetry/util/_decorator.py", line 71, in async_wrapper return await func(*args, **kwargs) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/client/mohamed_project/voice-pipeline-agent-python/venv/lib/python3.12/site-packages/livekit/agents/ipc/job_proc_lazy_main.py", line 240, in _traceable_entrypoint await self._job_entrypoint_fnc(job_ctx) File "/home/client/mohamed_project/voice-pipeline-agent-python/agent.py", line 113, in entrypoint agent=Assistant(), ^^^^^^^^^^^ File "/home/client/mohamed_project/voice-pipeline-agent-python/agent.py", line 40, in init super().__init__( TypeError: Agent.__init__() got an unexpected keyword argument 'tts_node' {"pid": 33793, "job_id": "AJ_2FgDXoXpaUEQ"} 2025-08-04 044731,095 - INFO livekit.agents - process exitin i'm not use the odl version usevoice pipeline i'm using from livekit.agents import ( Agent, AgentSession, AutoSubscribe, JobContext, JobProcess, WorkerOptions, cli, metrics, Roo

Hey there! The error you're seeing happens because the

Agent

class in the latest LiveKit version doesn’t take a

tts_node

parameter directly. But no worries—you can still use a custom TTS node by subclassing

Agent

and overriding the

tts_node

method. Here’s a quick example to help you out:

from livekit.agents import Agent, AgentSession
from livekit.agents.voice import ModelSettings

class CustomTtsAgent(Agent):
    async def tts_node(self, text: AsyncIterable[str], model_settings: ModelSettings):
        """Override default TTS node behavior"""
        async for chunk in text:
            # Example: Remove numbers and special characters
            cleaned = re.sub(r"[\d\|]+|[^\w\sÀ-ÿ]", "", chunk)
            yield cleaned

async def entrypoint(ctx: JobContext):
    session = AgentSession()
    await session.start(
        agent=CustomTtsAgent(
            instructions="Your agent instructions here",
            tts=cartesia.TTS(model="sonic-2", voice="8832a0b5-47b2-4751-bb22-6a8e2149303d", language="fr"),
        ),
        room=ctx.room
    )

This way, your custom TTS logic fits right in and keeps everything working smoothly with the latest LiveKit version. If you want me to walk you through anything else or have more questions, just let me know—I’m here to help! Sources: livekit-agents/livekit/agents/voice/agent.py | pipeline-tts/short_replies_only.py