the livekit server node sdk, takes around 600-900ms to make a dispatch

Thanks for reaching out, Sebastián. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

is the sdk call on the DispatchService

Thanks — I see you’re calling the DispatchService (

twirp/livekit.AgentDispatchService

) and your screenshot shows ~1.19s for that request. I’m responding to your question about that SDK call and why you’re seeing high latency. Quick confirmation: • Yes — dispatch is performed via the AgentDispatchService call you pictured. That RPC is responsible for routing/assigning a session to an agent. Why you might see ~600–1200 ms (and what to check) • Network RTT / geography: long round-trips between client, LiveKit server, and agents/models will increase the RPC time. Are the client, server, and agents in the same region? • Initialization vs dispatch: confirm whether your measurement includes agent startup or model load. First requests (cold starts) are often slower. • Host sizing/cpu: burstable or under-provisioned VMs (e.g., t3/t4g) can introduce delays if CPU credits are exhausted. • Co-location with model providers: if models are remote, model latency adds to the dispatch window. • Measurement scope: ensure your timer is only the RPC (dispatch) and not following handshake/connection steps. Next diagnostics I can help with (please provide as many as you can) 1. Are you on LiveKit Cloud or self-hosted? If self-hosted, what instance types/regions are you using? 2. Is the latency consistent or mostly on the first request? 3. How are you measuring the 600–900 ms (client-side timing, server logs, or APM)? Share the exact timestamps/events included. 4. Can you enable Agent Observability / per-turn

metrics

and share the dispatch-related metrics or logs (e2e_latency, time-to-first-token, etc.)? 5. Does the delay persist when you call a minimal dispatch-only path (no model or TTS/STT work)? Immediate suggestions you can try • Measure a minimal dispatch RPC to isolate routing time from init/load time. • Co-locate agents/models and server in the same region. • Avoid burstable instance types or increase instance size to rule out CPU credit issues. • Keep at least one agent warm if you suspect cold-starts are the cause. Tell me your answers to diagnostic questions 1–3 and I’ll give concrete next steps and command/config suggestions. , with interactive elements

delay is only for the dispatch request, not the agent time to wake up agent is in a different region

Thanks — you clarified the delay is only on the dispatch RPC and your agent is in a different region. That strongly points to network round‑trip latency as the main cause. What you can do next • Short term: co-locate the agent with the LiveKit server (same region) — this will remove the cross‑region RTT and should cut dispatch time substantially. • If co‑location isn’t possible: deploy routing or region‑aware dispatch so requests target agents in the nearest region to the server to reduce RTT. • Measure precisely: capture client → server → agent RTTs for the dispatch RPC (client-side and server-side timestamps) to confirm how much of the 600–900 ms is pure network latency versus server-side processing. One quick question so I can give concrete steps: are you on LiveKit Cloud or self‑hosted, and which region is your LiveKit server running in? , with interactive elements

im in livekit cloud

Thanks — since you're on LiveKit Cloud and the agent is in a different region, that likely explains the dispatch-only RTT. Quick questions so I can give exact steps: 1. Which LiveKit Cloud region is your server running in? 2. Which region is your agent currently deployed in? 3. Can you move the agent to the same region, or do you need a region-aware routing setup instead? If you tell me the two regions and whether you can relocate the agent, I’ll provide concise next steps for co‑locating the agent or configuring region-aware dispatch. , with interactive elements

we have brazil (sao paulo) and us (washington and sf) for our server agent is US the delay is from agent

Thanks — I’m replying to your note that the dispatch delay is isolated to the DispatchService RPC and your LiveKit Cloud servers include São Paulo while the agent is in the US. Root cause • The dispatch RPC latency is caused by cross‑region network RTT between the São Paulo server and the US agent. Cross‑region RTT (São Paulo ↔️ US) commonly adds several hundred ms, which matches your 600–900 ms observation. Options to reduce latency • Co‑locate agent and server (fastest): ◦ Move the agent to the same region as the LiveKit server, or move the LiveKit server to the agent’s region (e.g., US‑east or US‑west). This removes the inter‑region RTT. • If you must keep server in São Paulo: ◦ Self‑host an agent in São Paulo and connect it to LiveKit Cloud (removes cross‑region hop). ◦ Or deploy agents in the nearest supported region (us‑east) and use region‑aware dispatch; this reduces but does not eliminate the Brazil↔️US RTT. • Short term diagnostic: measure raw network RTT between the São Paulo server and the US agent (ping/traceroute from a host in each region) to confirm network portion of the delay. Next step Which approach do you want to try? • Move/repoint your LiveKit server to a US region (I can give the exact Cloud dashboard/CLI steps), or • Self‑host an agent in São Paulo (I can provide the minimal deployment commands and connection config), or • Keep current setup and I’ll show how to enable region‑aware dispatch and measure expected RTT. Tell me which option you prefer and I’ll provide the concise commands/configuration for it. Sources: Agent dispatch | LiveKit Documentation | Why WebRTC beats WebSockets for realtime voice AI | LiveKit | Agents on LiveKit Cloud | Deploy voice AI on global infrastructure | LiveKi... | Self-hosted deployments | LiveKit Documentation, with interactive elements