How do I enable the TTS to pronounce the mathematical symbol and also make he transcribtion show the right mathematical symbols? The TTS struggles to pronounce it if the raw mathematical symbol is sent to the tts and also i do not want the spoken symbol to be transcribed, I want it ot appear in the frontend in the right mathematical format

Thanks for reaching out, Anigilaje. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i am using this : session = AgentSession( vad=ctx.proc.userdata["vad"], llm=openai.LLM(model="gpt-5-nano", api_key=os.getenv("OPENAI_API_KEY") ), stt=openai.STT(model="whisper-1"), tts=openai.TTS( base_url="https://prepai-tts.publicaai.com/v1", model="chatterbox-trubo", voice="cloned-voice" ), ) usage_collector = metrics.UsageCollector() @session.on("metrics_collected") def _on_metrics_collected(ev: MetricsCollectedEvent): metrics.log_metrics(ev.metrics) usage_collector.collect(ev.metrics) # Start session with standard options await session.start( agent=agent, room=ctx.room, # Using RoomInputOptions for text/audio input room_input_options=RoomInputOptions( text_enabled=True, audio_enabled=True, ), room_output_options=RoomOutputOptions( transcription_enabled=True, audio_enabled=True ) ) ctx.log_context_fields["user_id"] = user_id # Assign session to agent to allow manual triggering agent.session = session logger.info(f"Agent initialized with prompt for subject: {subject or 'default'}, topic: {topic or 'core'}")

Thanks — this is helpful. You’re using OpenAI STT/LLM and a custom TTS endpoint at prepai-tts. I’ll help you get the TTS to speak math symbols naturally while the frontend transcript keeps the actual symbols. Quick clarifying question before I give exact code you can drop into your session: does your prepai-tts endpoint support SSML (or an SSML-like flag such as

enable_ssml_parsing

)? If you don’t know, you can test by sending a small SSML payload (e.g., a

<speak>

wrapper) and seeing if it accepts it or returns an error. Two implementation paths depending on that answer: 1) If prepai-tts supports SSML (preferred) • Keep the original text (with symbols) for transcription. • When sending text to TTS, wrap symbols using SSML instructions so the TTS pronounces them as words. For example, use

<say-as>

or

<phoneme>

to force pronunciation (provider syntax may vary). • Implementation concept: a. In your pipeline, create a small wrapper that converts the outgoing TTS input into SSML while leaving the transcript source unmodified. b. Example flow: • Transcript input (and saved transcript) stays as: "2 + 2 = 4" • TTS input becomes:

<speak>2 <say-as interpret-as="verbatim">+</say-as> 2 equals 4</speak>

(exact tags vary by provider) a. Ensure you set any provider flag required to enable SSML parsing (e.g.,

enable_ssml_parsing=True

) when constructing the TTS client if supported. Tell me if prepai-tts accepts SSML and I’ll give a concrete example using the session code you posted. 2) If prepai-tts does NOT support SSML • Implement simple preprocessing for speech and preserve the original for transcription: ◦ Add a custom

tts_node

(or a small wrapper around your TTS client) that converts math symbols into their spoken-word equivalents when sending to the TTS. Example replacements: ▪︎

+

→

plus

▪︎

-

→

minus

▪︎

*

or

×

→

times

▪︎

/

→

divided by

▪︎

=

→

equals

▪︎

√

→

square root of

▪︎

∑

→

sigma

or

sum of

▪︎

^

→

to the power of

◦ Keep the transcription pipeline unchanged (or in your

transcription_node

, explicitly re-insert symbols if your STT normalizes them to words). • Implementation concept: a. When message text arrives, stash the original string for transcript output. b. Build a speech_text by replacing symbols with words (use a mapping/dict). c. Send speech_text to TTS; send the original string to the frontend for display. • I can provide a compact Python snippet you can add to your AgentSession to do this replacement and route the two different strings appropriately. Other helpful options and considerations • If you need word-level timing or highlighting in sync with audio (e.g., highlight the “+” symbol while “plus” is spoken), ask whether you need timings; I can show how to align spoken-word tokens to symbol positions using simple mapping or provider timings. • Provide the list of math symbols you care about (basic arithmetic vs. LaTeX-level symbols) so I can give you a ready-to-use mapping table. Next step: tell me whether prepai-tts accepts SSML. If you’re unsure, say so and I’ll show both: • an SSML example you can test, • and a small custom

tts_node

Python snippet that performs symbol→word replacement while keeping the original for transcription.

alos, provide the github lik where ic a get more exaples , i ean the livekit github link

Thanks — great details. Below I’ll do two things: (1) give the LiveKit GitHub link you asked for, and (2) offer the two concrete implementation choices again (SSML vs. preprocessing) with small, copy‑pasteable examples tailored to your AgentSession setup so you can pick whichever fits your prepai-tts capabilities. 1) LiveKit GitHub link • LiveKit Agents repo: https://github.com/livekit/agents This repo contains voice agent examples, node overrides (

tts_node

,

transcription_node

), and sample voice-agent code you can adapt. 2) Two practical approaches you can use with your session code A. If prepai-tts supports SSML (recommended) • Keep the original text for transcription. • Send an SSML-wrapped string to TTS so symbols are spoken as words while the transcript retains symbols. Example concept (pseudo-code you can integrate):

def make_ssml_for_math(original_text):
    # Simple example: replace a few symbols with SSML hints; adjust tags to your TTS provider
    # e.g., turn "2 + 2 = 4" into "<speak>2 <say-as interpret-as='characters'>+</say-as> 2 equals 4</speak>"
    ssml = original_text.replace("+", "<say-as interpret-as='verbatim'>+</say-as>")
    ssml = ssml.replace("=", " equals ")
    return f"<speak>{ssml}</speak>"

# When sending to TTS:
original = "2 + 2 = 4"
ssml_input = make_ssml_for_math(original)
# send ssml_input to prepai-tts while storing original for transcription/display

• Test a small SSML payload to confirm whether the endpoint accepts

<speak>

or an

enable_ssml_parsing

flag. If it accepts, use SSML tags appropriate to the provider (some support

<phoneme>

or provider-specific attributes). B. If prepai-tts does NOT support SSML (fallback) • Preprocess text for TTS (replace symbols with spoken words) and keep the original text for the transcript/frontend. Example mapping + integration snippet:

MATH_MAP = {
    "+": "plus",
    "-": "minus",
    "*": "times",
    "×": "times",
    "/": "divided by",
    "=": "equals",
    "√": "square root of",
    "∑": "sum of",
    "^": "to the power of"
}

def symbols_to_speech(text):
    out = text
    for sym, word in MATH_MAP.items():
        out = out.replace(sym, f" {word} ")
    return " ".join(out.split())  # normalize whitespace

# Usage in your pipeline:
original = "∑_{i=1}^n i = n(n+1)/2"
speech_text = symbols_to_speech(original)
# send speech_text to prepai-tts; keep `original` as the transcript displayed in frontend

3) Handling sync/highlighting (optional) • If you want to highlight symbols in the transcript while the TTS says the word, ask and I’ll show: ◦ how to map spoken-word positions back to original-symbol positions ◦ or how to use provider word-level timing (if available) to align highlights 4) Next actions I can take for you • If you confirm whether prepai-tts accepts SSML (or share a short error from an SSML test), I’ll give a precise SSML payload tuned to your endpoint and a ready-to-drop snippet integrated with your AgentSession. • Or, if SSML is not supported, I can provide a complete small wrapper (a

tts_node

override) to place into your session that performs replacements and routes the original text to the transcription/frontend. Which would you like me to provide next — the SSML-ready code (if SSML is supported) or the full

tts_node

wrapper that does symbol→word preprocessing and preserves the original transcript? Sources: Meet KITT | LiveKit | TTS Comparison | LiveKit Documentation | Tool definition and use | LiveKit Documentation | Text and transcriptions | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation