I want to track each turn overall latency.. how to map all metric to a turn

Thanks for reaching out, Abhishek. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

i guess we do not get speech id in all metric_collected event

You're right to notice that not every

metrics_collected

metric will include a

speech_id

. Here's how to handle that and still map metrics to turns. When

speech_id

appears •

speech_id

is added to metrics emitted by turn-related components: End‑of‑Utterance (EOU), STT (when used), LLM, and TTS. Those are the primary metrics you should rely on for per‑turn correlation. Why some metrics lack

speech_id

• Some metrics are session- or system-level (transport, CPU, memory, network, etc.) and aren't tied to a specific turn, so they won’t include

speech_id

. • A metric may be emitted before a

speech_id

is assigned (early events) or after turn finalization, depending on timing. How to correlate metrics when

speech_id

is missing • Timestamps: use metric timestamps to join session-level metrics to the nearest turn window (e.g., between EOU start and response complete). • Request/trace identifiers: if a

request_id

or trace/span is present, use it to link metrics across services. • Custom turn IDs: emit or attach your own turn identifier at the earliest point you can (for example, generate a

turn_id

when speech detection starts) and include it with any events you control. • Heuristics: associate a session metric to the active turn if its timestamp falls within that turn’s start/end interval. Practical collection strategy 1. Index all metrics that include

speech_id

by

speech_id

. 2. For metrics without

speech_id

, attempt these in order: ◦ Match on `request_id`/trace id. ◦ Assign to the nearest

speech_id

by timestamp (define a max gap). ◦ Store as session-level metrics and surface them separately in dashboards. 1. Persist a turn record keyed by

speech_id

(or your

turn_id

) containing timestamps for start, EOU, LLM start/finish, and TTS start/finish. Use this timeline to map probables for any unmapped metrics. If you want, I can: • Suggest a concrete schema for storing per-turn timelines and unmapped metrics. • Provide code for timestamp-based matching with gap thresholds. Which would be more helpful?

does EOU metric event contains speedch_id

Yes — EOU (End‑of‑Utterance) metric events include a

speech_id

. Use that

speech_id

to link the EOU event to the corresponding turn and correlate it with STT, LLM, and TTS metrics that share the same

speech_id

. If you’d like, I can show a short example of extracting

speech_id

from EOU events and storing a turn timeline.

stt event does not have speech id. class STTMetrics(BaseModel): type: Literal["stt_metrics"] = "stt_metrics" label: str request_id: str timestamp: float duration: float """The request duration in seconds, 0.0 if the STT is streaming.""" audio_duration: float """The duration of the pushed audio in seconds.""" streamed: bool """Whether the STT is streaming (e.g using websocket).""" metadata: Metadata | None = None

You're correct: STT metrics (

STTMetrics

) do not include

speech_id

. Here are practical ways to correlate STT to a turn and a short example. How to map STT -> turn • request_id: STT includes

request_id

. If other components (or your code) propagate the same

request_id

through the pipeline, use it to link STT to the turn. • Timestamps: match STT `timestamp`/`audio_duration` to the turn timeline (e.g., between turn start and EOU) with a configurable max gap. • Custom turn_id: generate a

turn_id

when speech detection starts and attach it to any outgoing requests you control (STT requests, LLM calls, TTS), then index metrics by that ID. • Heuristics fallback: if no IDs match, assign STT to the nearest

speech_id

by timestamp, and mark it as inferred. Minimal example (timestamp + request_id fallback)

# stored by speech_id: {speech_id: {start_ts, eou_ts, ...}}
# incoming stt: STTMetrics with request_id, timestamp, audio_duration

def map_stt_to_turn(stt):
    # 1) try request_id -> speech_id mapping
    speech_id = request_id_to_speech_id.get(stt.request_id)
    if speech_id:
        return speech_id

    # 2) timestamp match: find active turn window containing stt.timestamp
    for sid, t in turns.items():
        if t['start_ts'] <= stt.timestamp <= t['eou_ts'] + MAX_POST_EOU_GAP:
            return sid

    # 3) nearest-turn fallback
    nearest = min(turns.items(), key=lambda it: abs(it[1]['eou_ts'] - stt.timestamp))
    if abs(nearest[1]['eou_ts'] - stt.timestamp) < MAX_NEAREST_GAP:
        return nearest[0]

    return None  # treat as session-level STT metric

Recommendation • If you can, propagate

request_id

or add a

turn_id

to STT requests — this is the most reliable. Otherwise use timestamp windowing with sane gap thresholds and mark inferred mappings. Would you like a suggested DB schema for storing turns and unmapped STT metrics or a ready-to-use matching implementation?

okk. how to map speech_id to items in chat_ctx

You're asking how to map

speech_id

to items in

chat_ctx

. Below are direct options and a concise example to implement the mapping. Options to map

speech_id

-> chat_ctx • Capture

speech_id

at transcription time: ◦ Listen to

user_input_transcribed

(or equivalent) which includes

speech_id

and the transcript; attach that

speech_id

to the chat message metadata when adding it to

chat_ctx

. • Use `conversation_item_added`: ◦ When a conversation item is committed, add

speech_id

into the item’s metadata (if the event provides it) or update the item with the

speech_id

you captured earlier. • Maintain an external mapping: ◦ Keep a dictionary keyed by

speech_id

storing the chat item id, transcript, timestamps, and any metadata. • Timestamp correlation fallback: ◦ If

speech_id

isn’t present on the chat item, match by timestamps (transcript timestamp ↔️ turn start/EOU) with a configurable max gap and mark the mapping as inferred. Minimal implementation example 1. Capture transcription -> store mapping:

speech_to_ctx = {}

@session.on("user_input_transcribed")
def on_user_input_transcribed(ev: UserInputTranscribedEvent):
    # ev.speech_id, ev.transcript, ev.timestamp
    # create chat item and store mapping
    chat_item = add_to_chat_ctx(text=ev.transcript, metadata={"speech_id": ev.speech_id})
    speech_to_ctx[ev.speech_id] = {"chat_item_id": chat_item.id, "timestamp": ev.timestamp}

1. Correlate metrics -> look up chat item:

@session.on("metrics_collected")
def on_metrics_collected(ev: MetricsCollectedEvent):
    for m in ev.metrics:
        sid = getattr(m, "speech_id", None)
        if sid and sid in speech_to_ctx:
            chat_item_id = speech_to_ctx[sid]["chat_item_id"]
            store_latency_for_chat_item(chat_item_id, m)
        else:
            # fallback: try request_id mapping or timestamp matching
            handle_unmapped_metric(m)

Practical tips • Prefer embedding

speech_id

directly into chat item metadata when creating the item—this is the most reliable mapping. • Propagate

request_id

across STT/LLM/TTS calls if you need an alternate stable key. • Persist the

speech_id

↔️

chat_item_id

mapping in your DB so dashboards and post‑session analytics can join metrics to chat history. • When using timestamp heuristics, store explicit start/EOU timestamps for each turn to reduce false matches. If you want, I can provide: • A suggested DB schema for persisting mappings, or • A timestamp-based matching function with configurable gap thresholds. Which do you prefer? Sources: Observability data hooks | LiveKit docs | Events and error handling | LiveKit docs