how to set Gemini 2.5 Flash-Lite llm in streaming ...
# ask-ai
b
how to set Gemini 2.5 Flash-Lite llm in streaming mode or is it streaming by default
t
Thanks for reaching out, Harsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Just to let you know, the
gemini-2.5-flash
LLM doesn’t stream by default. But no worries! You can enable streaming by calling the
chat()
method, which gives you an
LLMStream
object. This method uses the
generate_content_stream
function to create a streaming connection, so you get responses in real-time. If you're aiming for real-time voice interactions, the
RealtimeModel
class is actually built for streaming and low-latency use. Please feel free to reach out if you'd like a hand setting this up—I’m here to help! Sources: livekit-plugins/livekit-plugins-google/livekit/plugins/google/models.py | Hume TTS integration guide | LiveKit Docs | Gemini Live API integration guide | LiveKit Docs | plugins/google/src/models.ts
b
where do i add this code? give me example
t
Here’s a friendly example to help you get started with the
chat()
method using the
gemini-2.5-flash
model to enable streaming mode:
Copy code
from livekit.agents import ChatContext
from livekit.plugins import google

# Initialize the LLM with the gemini-2.5-flash model
llm = google.LLM(model="gemini-2.5-flash")

# Create a chat context and add a user message
chat_ctx = ChatContext()
chat_ctx.add_message(role="user", content="Hello, how are you today?")

# Initiate a streaming chat session
async with llm.chat(chat_ctx=chat_ctx) as stream:
    async for chunk in stream:
        if chunk.delta.content:
            print("Streaming response chunk:", chunk.delta.content)
To break it down a bit: • The
chat()
method kicks off a streaming session for you. • The
LLMStream
object you get back lets you handle response chunks as they come in, in real-time. • Just remember to initialize the LLM with the
gemini-2.5-flash
model to make this work smoothly. If anything feels unclear or you’d like me to walk you through it more, please don’t hesitate to ask. I’m here to help! Sources: livekit-plugins/livekit-plugins-google/livekit/plugins/google/beta/gemini_t... | plugins/openai/src/llm.ts | Large language model (LLM) integrations | LiveKit Docs