p99, p90 and p50 e2e latency for agent reply?

Thanks for reaching out, Saurabh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

both, TTFT and full reply completion

Thanks — you want p50/p90/p99 for both TTFT (time to first token/playback) and full reply completion. I don’t have precomputed percentiles to share, but I can walk you through exactly how to collect and compute them. A couple quick questions so I give the right, targeted steps: 1. Are you running LiveKit Agents on LiveKit Cloud with Agent Observability, or self-hosted? 2. Where do you prefer to compute percentiles? ◦ Exported traces (CSV/JSON) that you process locally, or ◦ Metrics backend (Prometheus/Grafana) where we can query percentiles directly? 1. Do you want the TTFT definition to be: time from user audio arrival at LiveKit server to the first audio token sent to the client (playback start)? And do you want full reply completion defined as: time from user audio arrival to final audio playback finished (or final token emitted)? If your definition differs, tell me the exact start/end events you want measured. Once you confirm those three items I’ll provide: • The exact event timestamps to capture from Agent Observability (which fields to use). • Step-by-step export or query instructions for your chosen path (sample PromQL or a simple script to compute p50/p90/p99 from CSV/JSON). • Practical tips to ensure accuracy (clock sync, filtering incomplete sessions, and excluding warm-up requests).

i am using livekit cloud

Thanks — I’ll help you get p50/p90/p99 for both TTFT and full-reply completion on LiveKit Cloud. A couple quick confirmations so I give the precise steps you can run right away: 1. Do you want to compute percentiles from: ◦ exported session traces (CSV/JSON) you’ll process locally, or ◦ LiveKit Cloud’s metrics backend (Prometheus/Grafana) so you can query percentiles directly? 1. Please confirm your exact event definitions: ◦ TTFT: is this "time from user audio arrival at the LiveKit server to the first agent audio token sent to the client (playback start)?" ◦ Full reply completion: is this "time from user audio arrival to the agent finishing the final audio token/playback end?" If you want different start/end events (e.g., user speech end vs arrival), tell me which. 1. Do you want percentiles computed across: ◦ all sessions in a time window (e.g., last 24h), or ◦ filtered by room/agent/model or other tags? If filtered, list the filters (room IDs, agent IDs, model names). Once you confirm 1–3 I will provide: • Exact fields/timestamps to extract from Agent Observability in LiveKit Cloud. • Step-by-step export or PromQL/Grafana queries to compute p50/p90/p99. • A small example script (CSV → percentiles) and practical tips (clock sync, excluding partial sessions).

for a conversation transcript, I need formula to calculate mentioned metrics

Thanks — I’m responding to your request for a formula/method to compute p50, p90, and p99 for both TTFT (time-to-first-token/playback) and full reply completion from a conversation transcript. Summary answer • You need to extract a single numeric latency value per agent reply (one for TTFT and one for full-reply completion). • Put those values into a list and compute percentiles (50th, 90th, 99th) with any standard percentile function. • I’ll show the exact timestamps to extract from transcripts/metrics, a clear formula, and a short example script. 1) Which timestamps to extract Choose consistent start/end events. Common, recommended definitions: • TTFT (Time To First Token / playback start) ◦ Start: user speech end timestamp (or “user audio arrival” if that’s how your transcripts mark it) ◦ End: agent first audio token emitted timestamp (or first agent audio playback start timestamp) ◦ Per-turn TTFT = EndTTFT - StartUserEnd • Full reply completion ◦ Start: user speech end timestamp (or user audio arrival, if you prefer) ◦ End: agent final audio token emitted timestamp (or agent playback end timestamp) ◦ Per-turn FullLatency = EndFull - StartUserEnd Pick the same “start” for both metrics (user speech end vs audio arrival) across all turns so values are consistent. 2) Basic percentile formula (concept) • Collect N latency samples: L = [l1, l2, ..., lN] • Sort L ascending → L_sorted • Percentile p (as percentage, e.g., 50, 90, 99): ◦ Using the common interpolation method (numpy style): index = (p/100) * (N - 1) If index is integer → value = L_sorted[index] Else → interpolate between floor(index) and ceil(index) • p50 = percentile(L, 50); p90 = percentile(L, 90); p99 = percentile(L, 99) 3) Practical steps to compute from a conversation transcript 1. For each agent turn in the transcript: ◦ Extract timestamps for start and end events per your chosen definition. ◦ Compute: ◦ ttft_i = timestamp_agent_first_token - timestamp_user_end ◦ full_i = timestamp_agent_final_token - timestamp_user_end 1. Filter out invalid turns: ◦ Discard turns missing timestamps, cancelled responses, or aborted streams. ◦ Optionally remove warm-up turns or unusually long outliers depending on analysis goals. 1. Compute percentiles on the remaining lists: ◦ TTFT_percentiles = percentile(ttft_list, [50,90,99]) ◦ Full_percentiles = percentile(full_list, [50,90,99]) 4) Example (Python) — adapt to your transcript fields

import numpy as np

# Example arrays of per-turn latencies in seconds (you will compute these from transcript timestamps)
ttft_values = [0.45, 0.32, 0.67, 0.51, 0.49, 0.88, 0.41, 0.53, 0.71, 0.62]
full_values = [1.20, 0.98, 1.50, 1.10, 1.05, 1.90, 0.95, 1.12, 1.40, 1.28]

def compute_percentiles(values):
    if not values:
        return {}
    p50 = np.percentile(values, 50)
    p90 = np.percentile(values, 90)
    p99 = np.percentile(values, 99)
    return {"p50": p50, "p90": p90, "p99": p99}

print("TTFT:", compute_percentiles(ttft_values))
print("Full reply:", compute_percentiles(full_values))

5) Notes & accuracy tips • Ensure timestamps use the same clock and timezone. If events are produced by different services, ensure clock sync (NTP) or use server-side event ordering fields. • Use the same start event across all samples (speech end vs audio arrival). • Exclude partial/errored turns (e.g., cancelled or interrupted) unless you explicitly want to include them. • For small N, p99 may not be meaningful; report sample size alongside percentiles. • If you want to compute percentiles over a time window or per-room/agent, group samples accordingly before computing. 6) Want me to produce a ready-to-run extractor? I can provide a short extractor script that: • Parses a JSON/CSV conversation transcript, • Computes per-turn ttft and full reply times using the exact field names you have, • Outputs p50/p90/p99 and sample counts. If so, tell me: • The exact field names in your transcript for user speech end, agent first token, and agent final token (examples:

user_end_ts

,

agent_first_audio_ts

,

agent_last_audio_ts

), • Whether timestamps are Unix seconds, milliseconds, or ISO strings. Would you like that extractor script? Sources: Text and transcriptions | LiveKit Documentation | examples/frontdesk | livekit/livekit_composite | DeepWiki | Agents (Python) | livekit/livekit_composite | DeepWiki | LLM Metrics | LiveKit Documentation | TTS Metrics | LiveKit Documentation