I understand how the

user_state_changed

event and

user_away_timeout

triggering during bot speech can be a bit tricky, but don’t worry—we can handle this smoothly by using the built-in state management of the

AgentSession

. The great news is that the framework automatically cancels the

user_away_timeout

timer whenever the agent isn’t in the "listening" state, so you won’t need to manage this manually. To help prevent any issues from the

user_state_changed

callback, here are a few friendly tips you can try: • Set

allow_interruptions=False

in your

AgentSession

configuration. This will disable user interruptions during the bot’s speech, keeping things smooth. • If you’d like, you can also adjust

min_interruption_duration

to require the user to speak for a minimum amount of time before triggering an interruption. • In your

user_state_changed

event handler, simply check the agent’s state before acting. For example, you can ignore the event if the agent isn’t in the "listening" state. Here’s a little example to guide you through setting up your session:

session = AgentSession(
    vad=silero.VAD.load(),
    llm=openai.LLM(),
    stt=deepgram.STT(),
    tts=cartesia.TTS(),
    user_away_timeout=15,
    allow_interruptions=False  # Prevent interruptions during bot speech
)

@session.on("user_state_changed")
def on_user_state_changed(ev):
    if session.agent_state != "listening":
        return  # Ignore events when the agent is not listening
    # Handle the event only when appropriate

This setup helps make sure your bot’s speech won’t cause any unwanted behavior, while keeping your session management neat and tidy. If you have any more questions or need further assistance, I’m here to help! Sources: livekit-plugins/livekit-plugins-ultravox/livekit/plugins/ultravox/realtime/... | Events and error handling | LiveKit Docs | Turn detection and interruptions | LiveKit Docs