https://livekit.io logo
Join Slack
Powered by
I'm using pipeline STT-> llm -> TTS. In a r...
# ask-ai
l
I'm using pipeline STT-> llm -> TTS. In a room if a participant send audio this pipeline is okay . if a participant is send text input i want to send the text ouput instead of TTS output. how to do that
✅ 1
t
Thanks for reaching out, Rajadurai. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I understand you want the pipeline to use TTS for audio inputs but return plain text when a participant sends text. You can do this by detecting text input and disabling audio output before sending the reply. Two approaches (pick one): • Global text-only (simple) ◦ When you start the session, set 
text_input=True
and 
audio_output=False
in 
RoomOptions
. All agent replies will be text-only (no TTS). • Hybrid (audio for speech, text-only for typed input) a. Detect input type: subscribe to the room text stream (
lk.chat
/ 
lk.transcription
) or use your text input callback. b. When input is text: • Disable audio output for that response: call 
session.output.set_audio_enabled(False)
(or set 
audioEnabled: false
in 
outputOptions
for Node). • Send the LLM reply as text (the agent will emit it on the transcription/text stream). • Optionally re-enable audio for subsequent audio inputs: 
session.output.set_audio_enabled(True)
. a. When input is audio: leave audio output enabled so the LLM reply is sent via TTS. Notes and pointers: • Use 
registerTextStreamHandler
(or equivalent) on the client to receive the agent’s text responses. • If you implement a 
text_input_cb
, you can intercept text, disable TTS, and return text-only responses from there. If you tell me which SDK/language you’re using (Node or Python) I can give a short code snippet showing the exact calls. Sources: Text and transcriptions | LiveKit Documentation | How to Set Up Speech-to-Text Input with Text-Only Output in LiveKit Agents, with interactive elements
l
write a python entrypoint code to handle this dynamicalyy. if a input is audio we should reply in audio. if it is text input. it should be text
t
Below is a concise Python entrypoint example that shows how to start a LiveKit Agent session and dynamically switch between audio (TTS) replies for audio input and text replies for typed input. It uses a text input callback and toggles 
session.output.set_audio_enabled()
around the response so audio is only produced for audio-originated inputs. Adjust config objects (STT/LLM/TTS) and imports to match your environment.
Copy code
import asyncio
from livekit_agent import AgentSession, RoomOptions, TextInputOptions, AgentContext

# Replace these placeholders with your real configurations
STT_CONFIG = {...}
LLM_CONFIG = {...}
TTS_CONFIG = {...}

class MyAgent:
    async def on_message(self, ctx: AgentContext, message: str):
        # Here you can call your LLM; return the reply text
        # Synchronous placeholder - replace with actual LLM call
        reply = f"Agent reply to: {message}"
        return reply

async def main():
    agent = MyAgent()

    async def text_input_cb(ctx: AgentContext, text: str):
        """
        This callback is invoked when a participant sends text (lk.chat).
        We want to reply with text only (no TTS).
        """
        # Ensure audio output is disabled so response is text-only
        await ctx.session.output.set_audio_enabled(False)

        # Process the message with your agent/LLM and send text reply
        reply = await agent.on_message(ctx, text)

        # Send text reply back via the session's text/transcription stream
        # Use session.send_text or other API your SDK exposes for text output
        await ctx.session.send_text(reply)

        # Optionally leave audio disabled; if you want to re-enable for future audio inputs,
        # you can re-enable here or re-enable on audio callback
        # await ctx.session.output.set_audio_enabled(True)

    # RoomOptions: enable both text input and audio output by default (hybrid)
    room_options = RoomOptions(
        text_input=True,
        audio_output=True,  # default; we'll toggle per input
        text_input_options=TextInputOptions(
            callback=text_input_cb
        )
    )

    session = AgentSession(
        stt=STT_CONFIG,
        llm=LLM_CONFIG,
        tts=TTS_CONFIG,
    )

    # Start the session (attach to an existing room)
    # Replace 'room_name' and any auth/connection params with your values
    await session.start(
        agent=agent,
        room="your_room_name",
        room_options=room_options,
    )

    # Example: also handle audio inputs via an audio callback or use STT events
    # If your SDK has an audio callback, ensure audio output is enabled there.
    async def on_audio_input(ctx: AgentContext, audio_event):
        """
        This function handles audio-originated input. STT pipeline will feed text to LLM.
        For audio-originated interactions we want TTS output enabled.
        """
        await ctx.session.output.set_audio_enabled(True)

        # STT may already transcribe and forward text; get the transcribed text
        transcribed_text = audio_event.transcript  # adjust per SDK
        reply = await agent.on_message(ctx, transcribed_text)

        # Use session.say or other TTS-send method so reply is spoken
        await ctx.session.say(reply)

    # If SDK requires registering audio handler, register here:
    # session.register_audio_handler(on_audio_input)

    # Keep the session running
    try:
        await session.wait_until_closed()
    finally:
        await session.stop()

if __name__ == "__main__":
    asyncio.run(main())
If you want, tell me which LiveKit Python SDK functions you have available (exact method names for sending text and TTS like 
send_text
, 
say
, or 
register_audio_handler
) and I’ll adapt this snippet to match those APIs.
Open in Slack
PreviousNext
Powered by