how do I plug into VAD SOS/EOU events in my python agent?

Thanks for reaching out, rohit. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I want the start of speech event, and can't seem to find the right code to plug into it

I'm happy to help you with handling the start of speech (SOS) event in your Python agent! You have a couple of great options using Voice Activity Detection (VAD) or Speech-to-Text (STT) engines. Here’s a friendly guide on how you can implement this: 1. Using VAD: • VAD will emit a

VADEvent

with the type

VADEventType.START_OF_SPEECH

when it detects speech. • You can listen for these events in your agent and handle them with the

on_start_of_speech

method.

from livekit.agents.vad import VADEventType

async def process_vad():
    async for vad_event in agent.vad_stream:
        if vad_event.type == VADEventType.START_OF_SPEECH:
            print("User started speaking")

2. Using STT Engines: • Some STT engines, like Deepgram, Azure, or Google Cloud, also emit

START_OF_SPEECH

events when speech is detected. • This can be a helpful fallback if VAD isn’t available or isn’t working perfectly for your needs. 3. Session-Level Events: • LiveKit agents also provide higher-level session events, such as

user_started_speaking

, which you can subscribe to for managing state or logging.

@session.on("user_started_speaking")
def on_user_speak():
    print("User began speaking")

If you’d like, I’m here to help with more examples or to walk you through your specific use case—just let me know! Sources: livekit-agents/livekit/agents/voice/agent_activity.py | agents/src/vad.ts | home_assistant/homeautomation.py