Hi, I was wondering whether it was possible to kno...
# agents
m
Hi, I was wondering whether it was possible to know when realtime models detect an interrupt (ideally, through an event) because they have their own VAD. I couldn't find a clear way of doing it in the docs and even the slack AI agent couldn't help much. If it's currently not possible, I would like to hear what the team thinks about it because I would be interested in contributing if the feature makes sense. Thanks
I looked through the source code and it seems like this event (
input_speech_started
) is being emitted when the realtime's model-sided VAD detects an interrupt but it's not clear whether this is the only way this event is used because the name is a bit confusing and it sounds like the event is overloaded. For example, here I can see it being used when Gemini live api detects an interrupt
l
Interesting, I haven't seen that request before but it makes sense. Like you say, I don't think it is currently possible but I'm happy to be proven wrong! Cursor helped me test the
input_speech_started
from my agent but as you saw, it fires for BOTH interrupts and user starting to speak:
Copy code
# Access the RealtimeSession through the session's internal activity
    # The input_speech_started event is from server-side VAD in the realtime model
    # Note: This event may only fire when server-side VAD is active
    if hasattr(session, '_activity') and session._activity:
        rt_session = getattr(session._activity, '_rt_session', None)
        if rt_session:
            @rt_session.on("input_speech_started")
            def on_input_speech_started():
                print("**** Input speech started")
        else:
            print("Warning: Could not access RealtimeSession. Event may not fire if using client-side VAD.")
    else:
        print("Warning: Could not access session activity.")