https://livekit.io logo
Join Slack
Powered by
How to monitor EOU (End-of-Utterance) probabilitie...
# ask-ai
b
How to monitor EOU (End-of-Utterance) probabilities and turn detection decisions in real-time? We're using the following configuration based on your recommendations: from livekit.plugins.turn_detector.english import EnglishModel session = AgentSession( llm=openai.LLM(model="gpt-4o"), stt=deepgram.STT( model="nova-3", language="en-US", interim_results=True, endpointing_ms=25, punctuate=True, smart_format=True, filler_words=True ), tts=custom_baseten_tts, turn_detection=EnglishModel(), vad=silero.VAD.load(), min_endpointing_delay=0.5, max_endpointing_delay=6.0 ) Specific questions: 1. Are there any session events we can listen to for monitoring EOU probabilities in real-time? Something like: @session.on("eou_probability_updated") def on_eou_update(event): print(f"EOU probability: {event.probability}") 2. Is there a way to access the turn detection model's confidence scores when it makes endpointing decisions? 3. Are metrics like eou.end_of_utterance_delay only available through the metrics system, or can we access them in real-time during the conversation? We want to log these values to debug why our turn-taking still feels slightly unnatural despite implementing all the recommended settings.
t
Thanks for reaching out, Andrew. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To monitor EOU (End-of-Utterance) probabilities and turn detection decisions in real-time, here are the available options and methods: 1. Real-time EOU Probability Monitoring: • You can create a custom turn detector wrapper to access EOU probabilities in real-time. For example:
Copy code
class CustomTurnDetector:
    def __init__(self, userdata: TurnTakingUserData = None):
        self._model = MultilingualModel()
        self.last_eou_probability = None
        self.userdata = userdata

    async def predict_end_of_turn(self, chat_ctx: llm.ChatContext) -> float:
        probability = await self._model.predict_end_of_turn(chat_ctx)
        self.last_eou_probability = probability
        
        # Send to frontend via RPC
        await self._send_eou_to_frontend(probability)
        return probability
2. RPC Communication for Real-time Updates: • Send EOU updates to a frontend using RPC. Example:
Copy code
await room.local_participant.perform_rpc(
    destination_identity=client_participant.identity,
    method="client.eou_update",
    payload=json.dumps({
        "eou_probability": probability,
        "percentage": probability * 100
    })
)
• The frontend can register an RPC handler to receive these updates:
Copy code
const handleEouUpdateRpc = async (rpcInvocation) => {
    const payload = JSON.parse(rpcInvocation.payload);
    if (payload && typeof payload.percentage === 'number') {
        setEouProbability(payload.percentage);
    }
};

room.localParticipant.registerRpcMethod("client.eou_update", handleEouUpdateRpc);
3. Turn Detection Model Confidence Scores: • The turn detection models return probability scores directly from their 
predict_end_of_turn
method. These scores can be logged or sent to a frontend for debugging. 4. Metrics Access: • Metrics like 
eou.end_of_utterance_delay
are available through the telemetry system. These include: ◦ `ATTR_EOU_PROBABILITY`: The end-of-turn probability ◦ `ATTR_EOU_DELAY`: The endpointing delay applied ◦ `ATTR_END_OF_UTTERANCE_DELAY`: Time from end of speech to turn completion • These metrics are captured during EOU detection spans and can be accessed for debugging. By combining these methods, you can monitor and debug EOU probabilities and turn detection decisions effectively in real-time. Sources: livekit-agents/livekit/agents/telemetry/trace_types.py | Logs, metrics, and telemetry | LiveKit Docs | complex-agents/turn-taking/turn-taking-frontend/app/page.tsx
6 Views
Open in Slack
PreviousNext
Powered by