Thanks — I’m responding to your request for simple code examples to evaluate metrics for STT, LLM, VAD, and TTS. Below are compact, ready-to-run examples (Python) that show common evaluation metrics you can capture and log. Each example focuses on the minimal metric set useful for quick evaluation. 1) Speech-to-Text (STT) • Metrics: transcription text, latency, audio duration, and end-of-utterance (EOU) delay.

# STT metrics example
import time

def evaluate_stt(recognize_fn, audio):
    start = time.time()
    transcription = recognize_fn(audio)        # returns text
    end = time.time()

    metrics = {
        "transcription": transcription,
        "latency_s": end - start,
        "audio_duration_s": len(audio) / 16000.0,  # example: samples / sample_rate
        "eou_delay_s": end - start                 # simple proxy for EOU delay
    }
    print(metrics)
    return metrics

2) Large Language Model (LLM) • Metrics: time-to-first-token (TTFT), total response time, prompt/completion token counts (if available), and tokens/sec.

# LLM metrics example
import time

def evaluate_llm(call_llm_fn, prompt):
    t0 = time.time()
    stream = call_llm_fn(prompt, stream=True)

    # time to first token
    first_token_time = None
    tokens = []
    for token in stream:
        if first_token_time is None:
            first_token_time = time.time()
        tokens.append(token)

    t_end = time.time()
    prompt_tokens = len(prompt.split())
    completion_tokens = len(" ".join(tokens).split())

    metrics = {
        "ttft_s": (first_token_time - t0) if first_token_time else None,
        "duration_s": t_end - t0,
        "prompt_tokens": prompt_tokens,
        "completion_tokens": completion_tokens,
        "tokens_per_second": (completion_tokens / (t_end - (first_token_time or t0))) if first_token_time and t_end>first_token_time else None
    }
    print(metrics)
    return metrics

3) Voice Activity Detection (VAD) • Metrics: speech segments count, total speech time, idle time, inference latency per frame.

# VAD metrics example
import time

def evaluate_vad(vad_fn, audio_frames):
    speech_segments = []
    total_speech_time = 0.0
    total_inference_time = 0.0
    idle_time = 0.0
    last_speech_end = 0.0

    for frame_idx, frame in enumerate(audio_frames):
        t0 = time.time()
        is_speech = vad_fn(frame)   # returns True/False
        t1 = time.time()

        total_inference_time += (t1 - t0)

        if is_speech:
            # simple segment counting by contiguous speech frames
            speech_segments.append(frame_idx)
            total_speech_time += len(frame) / 16000.0
            last_speech_end = frame_idx
        else:
            idle_time += len(frame) / 16000.0

    metrics = {
        "speech_frame_count": len(speech_segments),
        "total_speech_time_s": total_speech_time,
        "idle_time_s": idle_time,
        "avg_inference_time_per_frame_s": total_inference_time / max(1, len(audio_frames))
    }
    print(metrics)
    return metrics

4) Text-to-Speech (TTS) • Metrics: time-to-first-byte (TTFB) / time-to-first-sample, generation duration, audio duration, and characters generated.

# TTS metrics example
import time

def evaluate_tts(synthesize_fn, text):
    t0 = time.time()
    audio_stream = synthesize_fn(text, stream=True)  # yields chunks or samples
    t_first = None
    audio_samples = []

    for chunk in audio_stream:
        if t_first is None:
            t_first = time.time()
        audio_samples.extend(chunk)

    t_end = time.time()

    metrics = {
        "ttfb_s": (t_first - t0) if t_first else None,
        "generation_duration_s": t_end - (t_first or t0),
        "audio_duration_s": len(audio_samples) / 16000.0,
        "characters": len(text)
    }
    print(metrics)
    return metrics

Quick notes and suggestions • Replace placeholder functions (e.g.,

recognize_fn

,

call_llm_fn

,

vad_fn

,

synthesize_fn

) with your STT/LLM/VAD/TTS library calls or LiveKit agent hooks. • Use actual sample rates (e.g., 16000) that match your audio. • For production monitoring, send these metrics to a metrics backend (Prometheus, Datadog) instead of printing. • If you want examples integrated with LiveKit Agents event hooks or the exact metric objects (e.g.,

STTMetrics

,

LLMMetrics

) I can provide those next. Would you like: • LiveKit Agent-based snippets that hook into the built-in metric events, • or examples that integrate these metrics into a monitoring system (Prometheus/Datadog)? Sources: STT Metrics | LiveKit Documentation | LLM Metrics | LiveKit Documentation | VAD Metrics | LiveKit Documentation | TTS Metrics | LiveKit Documentation