how to get the provider session for realtime model...
# ask-ai
a
how to get the provider session for realtime models? in particular for Grok Voice
✅ 1
t
Thanks for reaching out, Sebastián. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you're asking how to get the provider session for realtime models, specifically Grok Voice. Here's what I can tell you. The Grok Voice integration in LiveKit is exposed through the xAI plugin as a
RealtimeModel
used inside an
AgentSession
(for example:
AgentSession(llm=xai.realtime.RealtimeModel(voice="Ara"))
). The SDK abstracts the realtime provider connection and does not document an API to retrieve the underlying provider session (websocket/low-level handle) directly. If you need provider-level telemetry or debugging: • Check whether the model emits session events or metrics (some realtime models emit
metrics_collected
events). • Let me know the exact telemetry or control you need (e.g., raw audio frames, socket events, connection state, or custom logging) and I’ll suggest the best workaround or the next steps., with interactive elements
a
i want session ids
t
Acknowledge: You're asking for session IDs for Grok Voice realtime sessions. Short answer: LiveKit does not expose provider session IDs for Grok Voice. The xAI
RealtimeModel
used inside
AgentSession
abstracts the provider connection and no documented
session_id
property or API exists for Grok Voice. If you need session IDs for a specific purpose (debugging, logs, correlation), tell me what you're trying to do and I’ll suggest alternatives (for example, using request/response correlation IDs in your app, capturing LiveKit session/room IDs, or using models that expose session IDs like AssemblyAI’s
SpeechStream.session_id
). Sources: AssemblyAI STT | LiveKit Documentation, with interactive elements
a
i want them for support, with OpenAI we have specific openai events of the session, i want the same for grok
t
Thanks — you're asking for Grok Voice provider session events/IDs like you get for OpenAI. We do not currently expose a provider session ID or OpenAI‑style session events for Grok Voice. The
xai.realtime.RealtimeModel
used in an
AgentSession
abstracts the provider connection and no documented session_id/event stream for Grok Voice exists. If you need equivalent support telemetry, here are practical alternatives — tell me which fits your workflow and I’ll provide steps: • Use your own correlation ID: generate a request/session UUID in your application and attach it to logs, agent messages, or metadata so you can correlate LiveKit room/track activity with provider conversations. • Correlate with LiveKit identifiers: use LiveKit room IDs, participant IDs, or track IDs as support keys when investigating issues. • Switch to a model that exposes session IDs (e.g., AssemblyAI’s
SpeechStream.session_id
) for sessions where provider session visibility is required. If you want, tell me: • Which specific events you need (connect/disconnect, turn start/end, errors), and • Whether you can add a correlation header/metadata in your app. I’ll then give concrete implementation steps for the best option. , with interactive elements
a
for OpenAI Realtime, we're doing a specific event: "@_self_.assistant.realtime_llm_session.on("openai_server_event_received")"
t
Thanks — you're asking to get the same provider-session event hook for Grok Voice that you have for OpenAI Realtime (e.g.,
assistant.realtime_llm_session.on("openai_server_event_received")
). Short answer: Grok Voice does not expose an OpenAI‑style provider session event or session_id in the LiveKit API today. The
xai.realtime.RealtimeModel
inside
AgentSession
abstracts the provider connection and no equivalent event is available. Options I can help with next (tell me which you prefer): • I can walk you through adding a correlation ID you emit from your app (attach to agent messages/metadata and logs) so support can link LiveKit activity to a Grok conversation. • I can show how to use LiveKit room/participant/track IDs as support keys and add structured logs for easier triage. • I can file a feature request to expose provider session events/session_id for Grok Voice (include which events you need: connect/disconnect, turn start/end, errors, raw server events). Which option do you want to proceed with, and which specific events or session fields do you need for support? , with interactive elements
a
how to register on an agent a post activity event listener? for example:
Copy code
self.realtime_llm_session.on(
            "openai_server_event_received", self.handle_openai_event
        )
t
You're asking how to register a post-activity event listener on an agent (like your OpenAI example) and whether the same hook exists for Grok Voice. Short answer: • To register session-level listeners, use
session.on(...)
on your
AgentSession
(works for Grok and OpenAI). Example events:
agent_state_changed
,
user_state_changed
,
conversation_item_added
,
user_input_transcribed
,
close
. • For OpenAI Realtime you can also subscribe to the low-level provider event via the LLM object:
session.llm.on("openai_server_event_received", handler)
. • Grok Voice (xAI
RealtimeModel
) does not expose that OpenAI-style provider event or a provider
session_id
today. How to register a post-activity listener on the AgentSession (Python example):
Copy code
@session.on("conversation_item_added")
async def on_conversation_item_added(ev):
    # ev.item.role == "assistant" for agent outputs
    if ev.item.role == "assistant":
        await handle_agent_output(ev.item)
If you specifically need the provider-level events for Grok (connect/disconnect/raw server events) here are alternatives I can help implement: • Add a correlation ID you attach to agent messages/metadata and your logs so support can trace conversations. • Use LiveKit room/participant/track IDs as support keys and emit structured logs on the same correlation ID. • File a feature request to expose Grok provider events/session_id (tell me which events you need: connect/disconnect, turn start/end, errors, raw server events). Which alternative would you like to proceed with, and which exact events or fields do you need for support? Sources: Turns overview | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation | Agent session | LiveKit Documentation | How to Build a Background Observer for Voice AI Guardrails | LiveKit | Debug voice agents in real time with Agent Console | LiveKit, with interactive elements
a
but can't that be done from the agent class?
t
Short answer: no — register those listeners on the AgentSession, not inside the Agent class. Explanation and what to do next: • Use
session.on(...)
to listen for post-activity/session events (e.g.,
conversation_item_added
,
agent_state_changed
,
user_state_changed
,
user_input_transcribed
,
close
). Example:
Copy code
@session.on("conversation_item_added")
async def on_conversation_item_added(ev):
    if ev.item.role == "assistant":
        await handle_agent_output(ev.item)
• For provider-level raw events you use the LLM session (e.g.,
session.llm.on("openai_server_event_received", ...)
) — that exists for OpenAI only. Grok Voice (
xai.realtime.RealtimeModel
) does not expose the OpenAI‑style provider event or session_id today. If you need provider-level visibility for support, I can: • Show how to add a correlation ID into agent messages/metadata and logs so support can trace conversations, or • Show how to emit structured logs with LiveKit room/participant/track IDs, or • File a feature request to expose Grok provider events/session_id. Which of these do you want to proceed with, and which specific events/fields do you need? Sources: Turns overview | LiveKit Documentation | Pipeline nodes and hooks | LiveKit Documentation | Agent Builder | LiveKit Documentation | Job lifecycle | LiveKit Documentation | Debug voice agents in real time with Agent Console | LiveKit, with interactive elements
a
but on handoff i think this is only registered for the first agent, every handoff need to re-register
t
You're right to be concerned — listeners registered on an
AgentSession
do not automatically carry over if a handoff creates a new session. Quick summary and options: • Fact: Event listeners are attached to the
AgentSession
. If a handoff instantiates a new
AgentSession
, you must re-register listeners for that new session. • If you reuse the same
AgentSession
across agents, listeners persist and do not need re-registration. Practical approaches: • Create a helper that registers all required listeners and call it whenever you create a session:
Copy code
def register_listeners(session):
    @session.on("conversation_item_added")
    async def on_conversation_item_added(ev):
        ...
    @session.on("agent_state_changed")
    def on_agent_state_changed(ev):
        ...
# call after session creation
register_listeners(session)
• Or centralize session creation so handoffs reuse the same
AgentSession
when possible. • If reusing isn't possible, attach a correlation ID (UUID) to agent messages/metadata so support can trace conversations across sessions. Which pattern fits your architecture — reusing sessions or creating new sessions per handoff? I can give a tailored code example for that choice. , with interactive elements