Thanks — I hear your frustration that Scribe_v2_realtime is showing latency tens of times higher than the ~150 ms you expected. Below I’ll explain the most common causes and give focused, actionable checks you can run to find where the delay is coming from. What I’m responding to: your observation that Scribe_v2_realtime’s real-world latency is much higher than the advertised ~150 ms. Most likely causes (and why they matter) • Network RTT and geography: Each hop (client → LiveKit server/agent → ElevenLabs STT endpoint) adds round-trip time. If any components are in different regions, 30–100+ ms per hop can accumulate into large end-to-end delay. • CPU throttling on burstable VMs: Running agents or workers on burstable instance types (e.g., t3/t4) can cause big latency spikes when credits are exhausted — operations that normally take 50–160 ms can stretch to seconds. • Buffering / audio frame batching: If your pipeline batches or buffers audio before sending (instead of streaming 20 ms frames), that buffering adds fixed latency before transcription starts. • WebRTC jitter / packet loss: Jitter, reordering, or packet loss in the media path can delay audio arrival and the STT stream’s ability to produce tokens in real time. • Agent/service placement and internal hops: Agent dispatch and media routing add overhead. If the agent isn’t co-located with the STT service or the LiveKit server, each hop increases latency. • Endpoint congestion or throttling: Under load, STT providers sometimes buffer or throttle to maintain stability, which raises initial token delay (TTFT). Quick diagnostic steps (component-level checks) 1. Measure component latencies: a. Capture timestamp when audio is recorded on the client. b. Timestamp when LiveKit/agent receives that audio. c. Timestamp when the STT provider receives audio and when the first token is returned. This isolates whether the delay is network, agent compute, or STT processing. 1. Check RTTs and region placement: ◦ Ping or run traceroutes between client → LiveKit server → agent → STT endpoint. ◦ If components are cross-region, try co-locating the agent in the same region as the STT endpoint. 1. Verify instance types: ◦ Confirm agents/workers are on non-burstable instances (e.g., c5/m5). If on t3/t4, try a non-burstable instance to rule out CPU credit throttling. 1. Inspect audio framing and buffering: ◦ Ensure audio is streamed in small frames (20 ms is common) without extra aggregation on the client or agent before sending to the STT provider. 1. Test under controlled conditions: ◦ Run a local test (client + agent + STT in same region infra) to get a baseline. If latency drops to expected levels locally, the issue is likely network/geography. 1. Observe TTFT (Time To First Token): ◦ If TTFT is under ~300 ms in a good baseline but much higher in production, focus on network and instance placement. Practical mitigations you can try now • Move agent/workers into the same cloud region as the STT endpoint (typically US East for some providers). • Use dedicated CPU instances (c5/m5 families) for consistent performance. • Ensure minimal buffering in the streaming pipeline—send 20 ms frames in near-real time. • Run short controlled latency tests (local region) to create a baseline, then compare production numbers. • If you need alternative STT comparisons, try a low-latency model (e.g., Deepgram Nova-2 or AssemblyAI) to see if the behavior is provider-specific. If you want, I can help narrow this down — please tell me: • Where your LiveKit server, agent/workers, and clients are geographically located (cloud regions). • The instance types you’re running for agents/workers. • Whether you’re batching frames before sending to the STT provider (yes/no). • A sample of measured timestamps (client capture, agent receive, STT first token) if available. I’ll use that to suggest the next targeted tests or configuration changes. Sources: How We Built a Globally Distributed WebRTC Mesh Network | Troubleshooting Latency and Timeout Errors with Turn Detection on AWS | Live Conversations with AI using ChatGPT and WebRTC | Agent dispatch | LiveKit Documentation | examples/frontdesk | livekit/livekit_composite | DeepWiki