When I use

AvatarSession

, the agent replies the first time I connect. But after disconnecting and reconnecting, the agent stops replying (take my voice print the text, avatar showing, but not responding) (I need to restart the agent). I’m using the OpenAI Realtime model with TTS fallback.

Thanks for reaching out, imad. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I'm using the starter code provided by livekit and I'm adding avatar session, so I'm not overloading the chat context, it's just simple application

Thanks so much for clarifying! To help us get to the bottom of this together, could you please share how you're setting up the Realtime model and TTS fallback in your application? Specifically: • Are you using

modalities=["audio"]

or

modalities=["text"]

when configuring the Realtime model? • Are you manually attaching TTS listeners for text responses, or are you going with the default setup? Understanding this will really help us figure out whether the issue is with the configuration or the reconnection process. Looking forward to your reply!

default setup import logging from dotenv import load_dotenv from livekit.agents import ( NOT_GIVEN, Agent, AgentFalseInterruptionEvent, AgentSession, JobContext, JobProcess, MetricsCollectedEvent, RoomInputOptions, RunContext, WorkerOptions, cli, metrics, ) from livekit.agents.llm import function_tool from livekit.plugins import noise_cancellation, openai, silero,bey logger = logging.getLogger("agent") load_dotenv() class Assistant(Agent): def __init__(self) -> None: super().__init__( instructions="""You are a helpful voice AI assistant. You eagerly assist users with their questions by providing information from your extensive knowledge. Your responses are concise, to the point, and without any complex formatting or punctuation including emojis, asterisks, or other symbols. You are curious, friendly, and have a sense of humor.""", ) def prewarm(proc: JobProcess): proc.userdata["vad"] = silero.VAD.load() async def entrypoint(ctx: JobContext): session = AgentSession( llm=openai.realtime.RealtimeModel(voice="sage"), tts=openai.TTS(model="tts-1-hd", voice="sage") ) # # Add a virtual avatar to the session, if desired avatar=bey.AvatarSession(avatar_id='694c83e2-8895-4a98-bd16-56332ca3f449') await avatar.start(session, room=ctx.room) # Start the session, which initializes the voice pipeline and warms up the models await session.start( agent=Assistant(), room=ctx.room, room_input_options=RoomInputOptions( noise_cancellation=noise_cancellation.BVC(), ), ) # Join the room and connect to the user await ctx.connect() if name == "__main__": cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm))

Thank you so much for sharing your setup with me. I understand how important it is for everything to work smoothly! The default configuration of LiveKit's

AvatarSession

with the OpenAI Realtime model and TTS fallback does include built-in reconnection handling, which is great. That said, the issue you’re experiencing might be happening because the TTS pipeline or event listeners for text responses aren’t being re-attached after a reconnection. To help you move forward, here are a few things you can try: • Make sure the TTS listeners (like for

response.text.delta

) are explicitly re-attached once the connection is restored. • Keep an eye on the

session_reconnected

event so you can reinitialize the TTS pipeline and confirm everything recovers properly. • It might also help to use

modalities=["text"]

explicitly in your Realtime model configuration to keep the text output consistent for TTS processing. Please don’t hesitate to reach out if you’d like me to walk you through these steps or if you need any help debugging the reconnection process. I’m here to support you! Sources: livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/stt.py | plugins/openai/src/realtime/realtime_model.ts

Yes, guide me please

Let me guide you through how to make sure your TTS pipeline is re-attached smoothly and how to keep an eye on reconnection events in your setup: 1. Re-attach TTS Listeners After Reconnection: It’s helpful to add a listener for the

session_reconnected

event so you can reinitialize the TTS pipeline automatically. Here’s a simple example:

@session.on("session_reconnected")
async def on_reconnected():
    <http://logger.info|logger.info>("Session reconnected, reinitializing TTS pipeline")
    # Re-attach TTS listeners here

1. Explicitly Configure the Realtime Model: To make sure your TTS processing gets consistent text output, include

modalities=["text"]

in your Realtime model configuration. This is how you can update your session initialization:

session = AgentSession(
    llm=openai.realtime.RealtimeModel(modalities=["text"]),
    tts=openai.TTS(model="tts-1-hd", voice="sage")
)

1. Debugging and Validation: Turning on debug logging can really help you confirm that the

session_reconnected

event fires as expected and that your TTS listeners are re-attached properly. Keep an eye on the logs for any errors or events that might have been missed. Give these steps a try, and please don’t hesitate to reach out if you run into any issues or want a hand with anything else. I’m here to help!

but when user disconnect the session is closed, I'm not connecting to the same session, because session is already ended

Thank you so much for clarifying that a new session is created after the user disconnects. To help you handle TTS fallback and reconnection smoothly in this situation, here are some friendly steps you can follow: 1. Use a TTS Fallback Adapter: Try setting up a

FallbackAdapter

with multiple TTS providers to keep things resilient. For example:

from livekit.agents import tts
from livekit.plugins import elevenlabs, groq

tts_adapter = tts.FallbackAdapter(
    tts=[
        elevenlabs.TTS(),
        groq.TTS(),
    ],
    max_retry_per_tts=2
)

1. Preserve the Fallback Adapter Instance: It’s important to reuse the same

FallbackAdapter

instance across new sessions. This helps maintain the health status of your TTS providers and avoids retrying services that recently failed. 2. Monitor TTS Events: Keep an eye on

tts_availability_changed

events. This way, you can track the status of each provider and gently inform users if all TTS services happen to be temporarily unavailable. 3. Handle Session Termination: When you detect a session closing with

CloseEvent

, you can get ready for reconnection by reinitializing the session and reattaching the TTS pipeline. Please feel free to reach out if you'd like more help implementing these steps or managing session-specific state across new sessions. I’m here to support you! Sources: livekit-plugins/livekit-plugins-upliftai/livekit/plugins/upliftai/tts.py | Connecting to LiveKit | LiveKit Docs | CHANGELOG.md | Events and error handling | LiveKit Docs

the session is automatically created, I'm not creating a new session, and by the way the avatar was working before and since a week it's start to do this, so maybe a new package of livekit has been released?

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!

2025-09-17 142504,866 - WARNING livekit - livekit:rtc engine453livekit:rtc_engine - received session close: "signal client closed: \"stream closed\"" UnknownReason Resume