https://livekit.io logo
Join Slack
Powered by
What are the VAD levels event we can subscribe to ...
# ask-ai
l
What are the VAD levels event we can subscribe to ?
t
Thanks for reaching out, David. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
I'm happy to help you explore the VAD (Voice Activity Detection) events you can subscribe to in LiveKit! Here’s a quick overview to guide you: • START_OF_SPEECH: This event is triggered right when speech begins. It provides useful details like the sample index, timestamp, and how long the speech lasts. • INFERENCE_DONE: After each inference step, this event fires, sharing information such as speech probability, how long the inference took, and relevant thresholds. • END_OF_SPEECH: This one happens when speech ends following a period of silence, giving you timing and speech segment details to keep track of. • METRICS_COLLECTED: Periodically, this event reports performance metrics like idle time and the total duration of inferences, helping you monitor system efficiency. Also, don’t forget about the higher-level user_state_changed event! It’s great for staying updated on changes in a user’s state, such as when they start speaking, switch to listening, or become away. Sources: plugins/silero/src/vad.ts | livekit-plugins/livekit-plugins-silero/livekit/plugins/silero/vad.py | Events and error handling | LiveKit Docs
l
show me how to use it
t
I'd love to help you get started with using VAD (Voice Activity Detection) events in LiveKit! Here's a simple step-by-step guide to walk you through it: 1. Initialize the VAD Plugin: Begin by loading the VAD model with the 
silero.VAD.load()
method. You can customize it by setting parameters like 
min_speech_duration
and 
min_silence_duration
to suit your needs. 2. Create a VAD Stream: Next, create a VAD stream using the 
.stream()
method. This will help you process audio frames smoothly. 3. Push Audio Frames: Send your audio frames to the VAD stream using 
push_frame()
in Python or 
pushFrame()
in Node.js — whichever fits your setup. 4. Handle VAD Events: Finally, listen for important events like 
START_OF_SPEECH
, 
END_OF_SPEECH
, and 
INFERENCE_DONE
to detect when speech starts and ends. Here's a quick example:
Copy code
async for event in vad_stream:
    if event.type == VADEventType.START_OF_SPEECH:
        print("Speech started")
    elif event.type == VADEventType.END_OF_SPEECH:
        print("Speech ended")
If you're working with Node.js, you can try this:
Copy code
for await (const event of vadStream) {
    if (event.type === VADEventType.START_OF_SPEECH) {
        console.log("Speech started");
    } else if (event.type === VADEventType.END_OF_SPEECH) {
        console.log("Speech ended");
    }
}
Please feel free to reach out if you'd like me to walk you through more examples or if you have any questions along the way. I'm here to help! Sources: Silero VAD plugin | LiveKit Docs | Silero VAD plugin | LiveKit Docs | examples/primitives/echo-agent.py | plugins/silero/src/vad.ts | basics/echo_transcriber_agent.py
l
give me python listeners with decorators
t
Thanks for reaching out! I wanted to let you know that LiveKit doesn’t natively support Python decorators for VAD event listeners just yet. But no worries—you can easily register event handlers using the 
.on()
method. Here’s a little example to help you get started:
Copy code
from livekit.plugins import silero
from livekit.agents import vad

class MyAgent:
    def __init__(self):
        self.vad = silero.VAD.load()

        def on_start_of_speech(event: vad.VADEvent):
            print("Speech started")

        def on_end_of_speech(event: vad.VADEvent):
            print("Speech ended")

        self.vad.on("start_of_speech", on_start_of_speech)
        self.vad.on("end_of_speech", on_end_of_speech)
If you’re interested in using decorators, we can definitely explore creating a custom wrapper together. Just let me know, and I’d be happy to guide you through it! Sources: Silero VAD plugin | LiveKit Docs | Working with plugins | LiveKit Docs | metrics/send-metrics-to-3p/send_metrics_to_3p.py | tests/fake_vad.py
l
'VAD' object has no attribute 'VADEvent'"
t
Hi there! It looks like the error is happening because 
VADEvent
isn’t actually an attribute of the 
VAD
object. Instead, 
VADEvent
is a separate data structure that helps represent events emitted by the VAD system. To work with these events, you’ll want to listen to the events coming from the VAD stream. Here’s a little example to guide you:
Copy code
from livekit.plugins import silero
from livekit.agents import vad

vad_instance = silero.VAD.load()
vad_stream = vad_instance.stream()

async for event in vad_stream:
    if event.type == vad.VADEventType.START_OF_SPEECH:
        print("Speech started")
    elif event.type == vad.VADEventType.END_OF_SPEECH:
        print("Speech ended")
If anything isn’t clear or you’d like a hand with something else, please don’t hesitate to ask—I’m here to help! Sources: agents/src/voice/audio_recognition.ts | livekit-plugins/livekit-plugins-silero/livekit/plugins/silero/__init__.py
Open in Slack
PreviousNext
Powered by