Is there a way to ask for correction based on the confidence of the STT transcription? Is there even a way to get the confidence value of the STT transcription? I'd like for the agent to ask the user to repeat what they said when the confidence is low enough.

Thanks for reaching out, Luiz. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Yes, I'd love it! What's the setup to get and use the confidence values?

Hey there! If you want to set up and use confidence values in LiveKit STT transcription to trigger actions, here’s a friendly step-by-step guide to help you out: 1. Configure the STT Plugin: Start by using an STT plugin like AssemblyAI and set a confidence threshold that works for you. For example:

from livekit.plugins import assemblyai

# Initialize AssemblyAI STT with confidence threshold
stt = assemblyai.STT(
    end_of_turn_confidence_threshold=0.8,
    min_end_of_turn_silence_when_confident=160,
    max_turn_silence=2400,
)

1. Listen for Transcription Events: Use the

SpeechEvent

to catch those transcriptions and check their confidence levels.

async def on_transcript(event: SpeechEvent):
    if event.type == SpeechEventType.FINAL_TRANSCRIPT:
        confidence = event.alternatives[0].confidence
        transcript = event.alternatives[0].text
        
        if confidence >= 0.8:
            # Trigger action based on high confidence
            trigger_action(transcript)

# Set up event listener
stt.on('SpeechEventType.FINAL_TRANSCRIPT', on_transcript)

1. Trigger Actions Based on Confidence: Here’s where you decide what happens when the confidence hits your threshold. Just plug in your action logic:

def trigger_action(transcript: str) -> None:
    # Implement your action logic here
    print(f"Triggering action with transcript: {transcript}")

If you have any questions or want a hand with the setup, just let me know—I’m here to help!