how do i know when the stt is done transcribing an...
# ask-ai
c
how do i know when the stt is done transcribing an utterance with finality, not just partials? ie, how do i make sure im getting the following streams, but know that the last one is final: i i love play i love playing in the park i love playing in the park with my i love playing in the park with my daughter
✅ 1
t
Thanks for reaching out, Faraz. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — you’re asking how to know when an STT stream has produced a final (stable) transcript vs. interim partials. Good question. Listen for the STT event type that signals finality. LiveKit STT emits distinct SpeechEvent types: • INTERIM_TRANSCRIPT: partial/in-progress text (may change). • FINAL_TRANSCRIPT: stable, final transcription for that utterance — safe to act on. Implement by filtering events and only treating text from
FINAL_TRANSCRIPT
as definitive. The engine typically moves to FINAL_TRANSCRIPT after it detects end-of-speech (pause/endpointing) or a model-based turn-end. If you want, tell me which integration or language SDK you’re using (JS, Python, or server-side plugin) and I’ll show a short code snippet for filtering final transcripts. Sources: Speechmatics STT plugin guide | LiveKit Documentation | Speech-to-text (STT) models overview | LiveKit Documentation | Turn Detection for Voice Agents: VAD, Endpointing, and Model-Based Detectio..., with interactive elements
c
give me a code example of receiving these in LLM from STT
t
Do you want the example in JavaScript (browser/Node), Python, or another language? Also: are you using LiveKit’s real-time STT events (SpeechEvent stream) or a server-side transcription plugin that forwards events into an LLM pipeline? , with interactive elements
c
pythin
t
Thanks — I’ll show a concise Python example that: • Listens to LiveKit STT SpeechEvent stream, • Filters for FINAL_TRANSCRIPT vs INTERIM_TRANSCRIPT, • Sends only final transcripts to an LLM (example uses OpenAI-style async call). Replace the stubbed LiveKit stream and LLM call with your actual implementations/clients.
Copy code
import asyncio
from enum import Enum
from typing import AsyncIterable

# --- Mock types (replace with your LiveKit SDK types) ---
class SpeechEventType(Enum):
    INTERIM_TRANSCRIPT = "INTERIM_TRANSCRIPT"
    FINAL_TRANSCRIPT = "FINAL_TRANSCRIPT"
    START_OF_SPEECH = "START_OF_SPEECH"
    END_OF_SPEECH = "END_OF_SPEECH"

class Alt:
    def __init__(self, text: str):
        self.text = text

class SpeechEvent:
    def __init__(self, type: SpeechEventType, alternatives: list[Alt]):
        self.type = type
        self.alternatives = alternatives

# --- Example: process STT stream and call LLM on final transcripts ---
async def call_llm(prompt: str) -> str:
    """
    Replace with your LLM client call (OpenAI/other). This is an async stub.
    """
    await asyncio.sleep(0.1)  # simulate network
    return f"LLM response for: {prompt}"

async def process_stt_stream(stream: AsyncIterable[SpeechEvent]):
    async for event in stream:
        # ignore events with no alternatives
        if not getattr(event, "alternatives", None):
            continue

        text = event.alternatives[0].text

        if event.type == SpeechEventType.INTERIM_TRANSCRIPT:
            # optional: update UI or buffer
            print(f"Interim: {text}")
        elif event.type == SpeechEventType.FINAL_TRANSCRIPT:
            print(f"Final: {text}")
            # send final transcript to LLM
            llm_response = await call_llm(text)
            print(f"LLM -> {llm_response}")
        elif event.type == SpeechEventType.START_OF_SPEECH:
            print("Start of speech")
        elif event.type == SpeechEventType.END_OF_SPEECH:
            print("End of speech")

# --- Example mock stream to demonstrate behavior ---
async def mock_stt_events():
    events = [
        SpeechEvent(SpeechEventType.INTERIM_TRANSCRIPT, [Alt("i")]),
        SpeechEvent(SpeechEventType.INTERIM_TRANSCRIPT, [Alt("i love play")]),
        SpeechEvent(SpeechEventType.INTERIM_TRANSCRIPT, [Alt("i love playing in the park")]),
        SpeechEvent(SpeechEventType.INTERIM_TRANSCRIPT, [Alt("i love playing in the park with my")]),
        SpeechEvent(SpeechEventType.FINAL_TRANSCRIPT,   [Alt("i love playing in the park with my daughter")]),
    ]
    for e in events:
        await asyncio.sleep(0.05)
        yield e

# --- Run demo ---
async def main():
    await process_stt_stream(mock_stt_events())

if __name__ == "__main__":
    asyncio.run(main())
If you want, tell me which LLM client (OpenAI, Anthropic, etc.) and which LiveKit Python object you receive the SpeechEvent stream from, and I’ll adapt the snippet to real SDK calls. Sources: Build a multilingual voice agent that automatically switches languages | Li... | Speech-to-text (STT) models overview | LiveKit Documentation, with interactive elements
l
do you push from stt -> llm in your code itself or does livekit trigger the stt node and llm node separately when it deems to
t
Good question — regarding how STT -> LLM is triggered: LiveKit emits STT SpeechEvent events (interim and final). LiveKit does not automatically invoke your LLM for you — you receive those events and your code decides what to do with them. In practice you: • Listen to the SpeechEvent stream from LiveKit. • Ignore or show INTERIM_TRANSCRIPT events as needed. • When you receive a FINAL_TRANSCRIPT event, push that final text to your LLM pipeline (or trigger whatever downstream processing you need). If helpful, I can adapt the Python snippet I shared to your exact setup (which LLM client you use and which LiveKit object/SDK you get SpeechEvents from). Which LLM and LiveKit Python API are you using? , with interactive elements