Agent Session Configuration - How to Apply Dynamic User Settings from Participant Metadata Platform: Python SDK with livekit-agents Issue Description: We're building a voice agent where users can configure voice timing settings (endpointing_ms, min_endpointing_delay) through our app UI. These settings are stored in our database and need to be applied dynamically to each agent session. Current Implementation: 1. User settings are passed via participant metadata when creating the LiveKit token 2. In our agent's entrypoint function, we create an AgentSession with Deepgram STT configuration 3. We attempt to parse voice settings from participant metadata to configure the session Problem: The timing of participant connection vs session creation is causing issues: - If we create the AgentSession immediately in the entrypoint, we don't have access to participant metadata yet (participant hasn't connected) - When the participant connects later (via the "participant_connected" event), the AgentSession is already created with default settings - The STT configuration (endpointing_ms, min_endpointing_delay) cannot be updated after session creation Code Structure: async def entrypoint(ctx: JobContext): # At this point, ctx.room.remote_participants is often empty # We need participant metadata to get user's voice settings # Default settings endpointing_ms = 6000 min_endpointing_delay = 6.0 # Try to get from participants (often empty at this stage) if ctx.room.remote_participants: # Parse participant metadata for voice settings... # Create session with whatever settings we have session = AgentSession( stt=deepgram.STT( endpointing_ms=endpointing_ms, # Need dynamic value from user # ... ), min_endpointing_delay=min_endpointing_delay, # Need dynamic value ) # Later, participant connects with their metadata @ctx.room.on("participant_connected") def on_participant_connected(participant): # Now we have the metadata with voice settings # But session is already created! Questions: 1. What's the recommended pattern for waiting for participant metadata before creating the AgentSession? 2. Is there a way to update STT configuration after the session is created? 3. Should we use room metadata instead of participant metadata for settings that affect the agent? 4. Is there a way to ensure the participant is connected before the agent's entrypoint function proceeds to session creation? What we've tried: - Checking ctx.room.remote_participants at entrypoint (often empty) - Using participant_connected event (too late, session already created) - Passing settings via room metadata (not sure if this is the right approach) Any guidance on the best practice for applying user-specific configuration to agent sessions would be greatly appreciated! ---

Thanks for reaching out, Albert. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Clarification on Metadata Change Listener - session_update() Method Not Found Platform: Python SDK with livekit-agents Context: Following up on your previous response about handling dynamic user settings from participant metadata. Your Suggestion: You mentioned listening for metadata changes and updating the session on-the-fly: @ctx.room.on("participant_metadata_cha nged") def on_metadata_changed(old_metadata, participant): new_metadata = json.loads(participant.metadata) new_config = parse_session_config(new_metadata) session.session_update( endpointing_ms=new_config.endp ointing_ms, min_endpointing_delay=new_conf ig.min_endpointing_delay ) Issue: After searching through the LiveKit Python SDK source code, we cannot find the session_update() method on the AgentSession class. We found: - update_options() exists but is just a pass statement (empty implementation) - update_agent() exists but only updates the Agent instance, not session configuration - No session_update() method exists Our Investigation: # In livekit/agents/voice/agent_session.py def update_options(self) -> None: pass # Line 655-656 def update_agent(self, agent: Agent) -> None: # Only updates the agent, not session config The only session_update references we found were in the OpenAI Realtime API plugin, not the core AgentSession. Questions: 1. Is there actually a way to update STT configuration (endpointing_ms, min_endpointing_delay) after the AgentSession is created? 2. If not, should we just rely on wait_for_participant() and skip the metadata change listener entirely? 3. Are session configuration parameters intentionally immutable after creation? 4. Was the session_update() method perhaps confused with the OpenAI Realtime API's session update functionality? Our Current Understanding: Based on our research, it seems that: - Session configuration is set at creation time and cannot be modified - The wait_for_participant() solution is sufficient for our needs - Listening for metadata changes wouldn't help since we can't update the running session Could you please clarify whether dynamic session updates are possible, or if we should stick with just waiting for the participant before creating the session? Thank you for your help!

Hey there! Just a quick heads-up: once an

AgentSession

is created, you can't make dynamic updates to its configuration, like changing the STT settings. The

session_update()

method you mentioned isn’t actually part of the core

AgentSession

class, and the

update_options()

method is currently just a placeholder without any real functionality. With that in mind, the best way to go about this is to use

ctx.wait_for_participant()

to make sure the participant metadata is ready before you create the session. That way, you can set everything up correctly right from the start. If you have any more questions or want to chat through this further, just let me know—I’m here to help! Sources: Gladia integration guide | LiveKit Docs | livekit-plugins/livekit-plugins-baseten/livekit/plugins/baseten/stt.py