Is it possible to stream on LLM side?

Thanks for reaching out, Ashish. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

In python. Where you have stt, tts, LLM part. I want to stream text

Hey there! If you want to stream text using STT (Speech-to-Text), TTS (Text-to-Speech), and LLM (Large Language Model) in Python with LiveKit, here’s a friendly step-by-step guide to help you out: 1. *First, import the necessary modules:*`from livekit import agents, rtc from livekit.agents.voice import Agent, AgentSession from livekit.plugins import deepgram, openai` 2. *Next, set up your agent like this:*`class MyAgent(Agent): def __init__(self): super().__init__( instructions="You are a helpful voice assistant.", stt=deepgram.STT(), llm=openai.LLM(model="gpt-4"), tts=openai.TTS(), )` 3. *Then, create an

AgentSession

and connect it to a room:*`async def entrypoint(ctx: agents.JobContext): session = AgentSession() await session.start( agent=MyAgent(), room=ctx.room )` 4. *Finally, run your agent with:*`if name == "__main__": agents.cli.run_app( WorkerOptions(entrypoint_fnc=entrypoint) )` Here’s what this setup does for you: • STT: It turns audio into text using plugins like Deepgram or OpenAI. • LLM: Processes that text and crafts a helpful response using OpenAI’s GPT. • TTS: Converts the response back into audio with plugins like OpenAI or ElevenLabs. If you have any questions or want me to walk you through any part in more detail, just let me know—I’m here to help!

What are the available variables for LLM and how can I stream. In roomoutputoption I have audio enabled as false and sync transcription as False. The text I’m getting is a dump. So I want to stream it such that it looks good on ui

If you want to stream text for UI display with audio turned off and sync transcription disabled, here’s a friendly guide on how to set up your

RoomOutputOptions

and use

LLMStream

to get real-time text streaming going: 1. Turn Off Audio Output: Just set

audio_enabled=False

in your

RoomOutputOptions

to make sure no audio is sent out. 2. Turn Off Sync Transcription: Set

sync_transcription=False

so transcriptions come through as soon as they’re ready, without waiting to sync with any audio. 3. Stream the Text: Your agent will send text responses to the

lk.transcription

topic. On the frontend, you can catch these messages using

registerTextStreamHandler

for the

lk.transcription

topic and then display the text however you like. Here’s a quick example configuration to get you started:

session = AgentSession(
    llm=openai.LLM(model="gpt-4o-mini"),
)
await session.start(
    agent=MyAgent(),
    room=ctx.room,
    room_input_options=RoomInputOptions(text_enabled=True, audio_enabled=False),
    room_output_options=RoomOutputOptions(
        transcription_enabled=True,
        audio_enabled=False,
        sync_transcription=False
    ),
)

And on the frontend, you can handle the stream like this:

room.registerTextStreamHandler('lk.transcription', async (reader, participantInfo) => {
    const message = await reader.readAll();
    console.log(`New message from ${participantInfo.identity}: ${message}`);
});

This way, you’ll have text streaming smoothly to your UI without any audio or sync delays. If you have any questions or want to tweak this further, just let me know — I’m here to help! Sources: LLM | LiveKit Agents | Text and transcriptions | LiveKit Docs | Text and transcriptions | LiveKit Docs