are there events in livekit to know when the user speech finished and when the agent started speaking?

Thanks for reaching out, Clovis. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

same for if I wanna get when the user started speaking?

Hey there! 👋 Let me break down how LiveKit helps you detect when someone starts talking. It's actually pretty cool and depends on what you're building. For LiveKit Agents: • If you're using a VoicePipelineAgent, you'll get a handy

user_started_speaking

event that lets you know when someone starts chatting. • In version 1.0, you can also use the

user_state_changed

event to keep an eye on things. Check out this neat example:

@session.on("user_state_changed")
def on_user_state_changed(ev: UserStateChangedEvent):
    if ev.new_state == "speaking":
        print("User started speaking")

For Client SDKs: • The

ParticipantEvent.IsSpeakingChanged

is your go-to for tracking when someone starts or stops talking. Here's a quick example that shows how easy it is: `participant.on(ParticipantEvent.IsSpeakingChanged, (speaking: boolean) => { console.log(

${participant.identity} is ${speaking ? 'now' : 'no longer'} speaking. audio level: ${participant.audioLevel}

); });` For Low-Level Speech Detection: • Want more detailed control? Voice Activity Detection (VAD) events like

on_start_of_speech

have got you covered. • Speech-to-Text services also chip in with

START_OF_SPEECH

events to help you out. The right event really depends on what you're building - whether it's an agent, a client app, or you need super precise speech detection. Hope this helps you find exactly what you need! 😊 Sources: SpeechEventType | LiveKit Agents | VoicePipelineAgent | LiveKit Docs | VoicePipelineAgent | LiveKit Docs | VPAEvent | LiveKit Agents | Turn detection and interruptions | LiveKit Docs

and what about the tool calls, is there any event related to them to know exactly when they are executed or it's outside of livekit?

LiveKit has this super cool feature called the function_tools_executed event that helps you keep track of what's happening with your tool calls. 🌟 Here's the scoop on the Event: • What's inside: ◦ `function_calls`: A friendly list of

FunctionCall

objects that show which tools were used. ◦ `function_call_outputs`: The results from those tool calls - think of it like getting the receipts for what just happened! ◦ `created_at`: A timestamp that tells you exactly when this event popped up. Wanna see how it works? Check this out:

from livekit.agents.voice.events import FunctionToolsExecutedEvent

def on_tools_executed(event: FunctionToolsExecutedEvent):
    for call, output in event.zipped():
        print(f"Tool {call.name} executed with output: {output}")

session.on("function_tools_executed", on_tools_executed)

The coolest part? That zipped() method that pairs up your function calls with their outputs - it's like matching socks, but for code! 😄 This event is your behind-the-scenes buddy, letting you know exactly what tools have been called and what they discovered. Pretty neat, right? Sources: livekit-agents/livekit/agents/llm/llm.py