We are noticing with the Agentjs OAI Realtime sess...
# ask-ai
p
We are noticing with the Agentjs OAI Realtime session, periodically the open ai session will disconnect, and then LiveKit will try to reconnect,
OpenAI Realtime API connection failed, retrying in 0.1s
, however after the reconnection happens, livekit has connected to a new session it appears, and all OAI session history is wiped out. looks like OAI docs said you have to add the context history back in to the new session, i was wonderingg if LK has functionality to do that automatically
t
Thanks for reaching out, Scott. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I hear you: you're seeing OpenAI Realtime disconnects, LiveKit reconnects to a new OAI session, and the OAI history is gone. You're asking if LiveKit can restore that context automatically. Short answer: LiveKit does not automatically restore OpenAI Realtime session context after a disconnect. When LiveKit reconnects, a new OAI session is created and prior conversational context is not automatically reloaded. What you can do (recommended approach): • Capture and store the conversation history during the active session (for example, persist relevant messages from
session.history
to your backend or a cache). • On reconnection detect that the OAI session was replaced and re-inject the stored history into the new RealtimeModel session by loading those messages as the chat context. • Note: loading long history can force the model into text-only mode (so test limits and modality changes), and you may want to only re-inject a summarized subset or the most recent N messages. • If you use Azure/OpenAI variants, initialize the new RealtimeModel and TTS plugin appropriately (e.g., using
with_azure
or equivalent for your provider) before loading history. If you'd like, tell me: • which SDK/language you're using (browser JS, Node, or server), • whether you already persist
session.history
, and • whether you need to preserve audio/TTS state or only text context— and I can give a concise code sketch for capturing and re-injecting history. Sources: Realtime model plugins | LiveKit Docs | OpenAI Realtime API plugin guide | LiveKit docs | Agents and handoffs | LiveKit docs | Azure OpenAI Realtime API and LiveKit | LiveKit docs | Azure AI Services and LiveKit | LiveKit Docs
p
is there a Livekit event for the reconnection happening?
t
Yes — LiveKit emits a Reconnected event when the client successfully restores its connection after a disruption (it follows the Reconnecting event). You can listen for it on the Room object and run your recovery logic (for example, re-injecting OpenAI conversation history) there. Quick questions to help me give a precise code example: • Which SDK/environment are you using (browser JS, Node, or another client)? • Do you want a simple listener or an example that retrieves
session.history
and replays it into a new OpenAI Realtime session? Sources: Connecting to LiveKit | LiveKit docs
p
node
t
Thanks — you're using Node. Responding to your question about a reconnection event and how to handle it. Yes — Agent/Room sessions emit reconnection events in Node: •
AgentSessionEventTypes.Reconnecting
fires when reconnect attempts start. •
AgentSessionEventTypes.Reconnected
fires when reconnection succeeds. Use these to trigger recovery logic (for example, reload stored OpenAI history into a new RealtimeModel). Example:
Copy code
import { voice } from '@livekit/agents';

// session is your AgentSession instance
session.on(voice.AgentSessionEventTypes.Reconnecting, () => {
  console.log('Agent is reconnecting...');
  // optional: mark state, pause UI, persist in-memory history
});

session.on(voice.AgentSessionEventTypes.Reconnected, async () => {
  console.log('Agent has reconnected');
  // example recovery steps:
  // 1) retrieve persisted chat history from your backend/cache
  // 2) create or obtain new RealtimeModel if OpenAI session was replaced
  // 3) re-inject recent/summarized messages into the model context
  // 4) resume TTS/audio if needed
});
Notes and recommendations: • LiveKit does not automatically reload OpenAI Realtime conversation into a new OpenAI session. If OpenAI creates a new session on reconnect, you must re-inject the prior context from your persisted
session.history
or your own store. • Persist the most relevant recent messages (or a summary) rather than the full history to avoid modality/text-only issues and hitting input limits. • On Reconnecting you can snapshot
session.history
to a fast cache; on Reconnected, replay that snapshot into the new RealtimeModel before resuming audio output. If you want, tell me: • how you currently persist history (if at all), and • whether you need a concrete Node code sample showing persistence + re-injection into OpenAI Realtime — I’ll provide that. Sources: Connecting to LiveKit | LiveKit docs | How We Built a Globally Distributed WebRTC Mesh Network | How We Built a Globally Distributed WebRTC Mesh Network | Handling events | LiveKit docs | Azure OpenAI Realtime API and LiveKit | LiveKit docs
p
is this event available in specific agent nodejs version?
t
Short answer: reconnection events (Reconnecting / Reconnected) are available in current LiveKit client/agent Node.js builds, but I need your package version to confirm precisely. Please tell me the exact version(s) from your package.json for: •
@livekit/agents
•
@livekit/rtc-node
or
@livekit/client
(whichever you have) If you prefer, paste your package.json dependencies and I’ll confirm whether your version includes the AgentSession/Room reconnection events and recommend an upgrade command if needed. Sources: Connecting to LiveKit | LiveKit docs | Job lifecycle | LiveKit docs | client-sdk-js | livekit/livekit_composite | DeepWiki
p
v1.0.24
t
Thanks — with v1.0.24 you’re good. • The
Reconnecting
and
Reconnected
events are available (you can listen on the AgentSession / Room as shown earlier). • LiveKit does NOT automatically restore OpenAI Realtime conversation context after an OpenAI session is replaced; you must persist and re-inject history yourself on
Reconnected
. Minimal Node example for v1.0.24:
Copy code
import { voice } from '@livekit/agents';

// `session` is your AgentSession instance
session.on(voice.AgentSessionEventTypes.Reconnecting, () => {
  console.log('Reconnecting — snapshot session.history to cache');
  // snapshot: persist session.history to your cache/backend
});

session.on(voice.AgentSessionEventTypes.Reconnected, async () => {
  console.log('Reconnected — restore OpenAI context if needed');
  // 1) retrieve persisted history
  // 2) create/obtain new RealtimeModel for OpenAI if session replaced
  // 3) re-inject recent/summarized messages into the model before resuming TTS
});
If you want, I can provide a concrete code snippet showing: • how to snapshot
session.history
to memory or Redis on Reconnecting, and • how to re-inject that history into a new OpenAI RealtimeModel on Reconnected. Which storage do you prefer (in-memory, Redis, or DB)?