How to catch agent state changes like "listening" "thinking" with a listener?

To catch agent state changes such as "listening" or "thinking," you can use the

participant_attributes_changed

event to monitor changes in the agent's state. According to the LiveKit documentation, agent states are published as participant attributes, which can be accessed and monitored in your application. Would you like more details on setting up the listener or handling these state changes? Sources: - LiveKit documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

can i not catch it in an agent.on() event of some kind? @millions-hair-24956

The

agent.on()

method is typically used for specific events like

agent_started_speaking

or

agent_stopped_speaking

. However, for state changes like "listening" or "thinking," these are usually reflected as participant attributes rather than direct events. Therefore, you would need to monitor the

participant_attributes_changed

event to catch these state changes. If you need more detailed guidance on implementing this, please let me know. Sources: - LiveKit documentation 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

k show me how to do it with participant_attributes_changed

To handle agent state changes using the

participant_attributes_changed

event, you can set up a listener to detect changes in participant attributes. Here's a basic example using JavaScript:

room.on(RoomEvent.ParticipantAttributesChanged, (changed, participant) => {
  console.log('Participant attributes changed:', changed);
  if (changed['agent_state']) {
    console.log('Agent state changed to:', changed['agent_state']);
  }
});

This code listens for changes in participant attributes and logs the new agent state when it changes. You can replace

'agent_state'

with the specific key used for the agent's state in your application. Would you like more detailed information on this implementation? Sources: - Participant attributes and metadata 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

in this format

@self.agent.on("agent_speech_committed")
        def on_agent_speech_committed(msg: llm.ChatMessage):
            log_message(logger, "[AGENT]", msg.content)
            asyncio.create_task(send_debug_log(self.ctx, "AGENT", msg.content))

        @self.agent.on("agent_speech_interrupted")
        def on_agent_speech_interrupted(msg: llm.ChatMessage):
            log_message(logger, "[AGENT INTERRUPTED]", msg.content)
            asyncio.create_task(send_debug_log(self.ctx, "AGENT (INTERRUPTED)", msg.content))
            asyncio.create_task(send_data_message(self.ctx, {"agent_state": "agent_interrupted"}))

        @self.agent.on("metrics_collected")
        def on_metrics_collected(mtrcs: metrics.AgentMetrics):
            self.usage_collector.collect(mtrcs)
            metrics.log_metrics(mtrcs, logger=logger)

@millions-hair-24956

It seems likely that you want to handle agent state changes using a similar event-driven approach as shown in your example. However, based on the provided data sources, there isn't a direct

agent.on()

event for state changes like "listening" or "thinking." These states are typically managed through participant attributes. To achieve similar functionality, you might need to monitor participant attributes for changes in the agent's state. Unfortunately, the data sources do not provide a direct method to do this using

agent.on()

. Instead, you would typically use a participant attributes change listener as previously described. If you need more detailed guidance on implementing this, please let me know. Sources: - https://docs.livekit.io/python/livekit/agents/multimodal/multimodal_agent.html#livekit.agents.multimodal.multimodal_agent.MultimodalAgent 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

doesn't have to be agent on, just need to do it in my agent backend

To handle agent state changes in your agent backend, you can use the

participant_attributes_changed

event to monitor changes in the agent's state. Here's an example in Python:

@ctx.room.on("participant_attributes_changed")
def on_attributes_changed(changed_attributes, participant):
    if 'agent_state' in changed_attributes:
        new_state = changed_attributes['agent_state']
        print(f"Agent state changed to: {new_state}")
        # Perform actions based on the new state

This code sets up a listener for the

participant_attributes_changed

event, checking for changes in the

agent_state

attribute. When a change is detected, it logs the new state and can trigger additional actions based on the state. Would you like more detailed information on this implementation? Sources: - https://docs.livekit.io/agents/voice-agent/ - https://docs.livekit.io/python/livekit/agents/multimodal/multimodal_agent.html#livekit.agents.multimodal.multimodal_agent.MultimodalAgent 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

isn't it like lk_ something i should be listening to?

To listen for agent state changes in your backend, you should monitor the

lk.agent.state

attribute, which is used to track the agent's state. Here's how you can set up a listener for this in Python:

@ctx.room.on("participant_attributes_changed")
def on_attributes_changed(changed_attributes, participant):
    if 'lk.agent.state' in changed_attributes:
        new_state = changed_attributes['lk.agent.state']
        print(f"Agent state changed to: {new_state}")
        # Perform actions based on the new state

This code listens for changes in the

lk.agent.state

attribute, which reflects the agent's current state, such as "listening" or "thinking." When a change is detected, it logs the new state and can trigger additional actions based on the state. Would you like more detailed information on this implementation? Sources: - https://docs.livekit.io/components-android/livekit-compose-components/io.livekit.android.compose.ui/-compose-visibility/size.html - https://docs.livekit.io/agents/openai/client-apps/ 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

lk.agent.state