what is best practice when i have multiple workers registering to the same livekit project, but in different regions?

Thanks for reaching out, Aivis. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

lets say I want Europe, US East and would use OpenAI for LLM , Google Cloud and/or Elevenlabs for TTS and would have inbound calls to US and UK numbers

Thanks — this is helpful context. I'll recommend a concrete deployment and scaling plan for Europe + US East, using OpenAI for LLM and Google Cloud/ElevenLabs for TTS, with inbound calls to US and UK numbers. Deployment overview • Use a single

agent_name

for all workers so they form one pool and can fail over across regions. • Deploy workers in two regions: Europe (e.g., eu-west1) and US East (e.g., us-east1). Run at least three workers total per

agent_name

, ideally distributed 2/1 or 2/2 depending on traffic (see scaling below). • Prefer region placement that is nearest both callers and the external AI/TTS providers to reduce latency. Region mapping for your providers & callers • Inbound calls to US numbers: prioritize US East workers. • Inbound calls to UK numbers: prioritize Europe workers. • OpenAI (LLM): OpenAI endpoints are globally reachable but latency varies. Place workers close to callers and use the region nearest your high-volume callers to reduce round-trip time. • Google Cloud TTS: deploy workers in the same Google region as your TTS project for lowest TTS latency. • ElevenLabs TTS: choose worker regions closest to ElevenLabs endpoints (or where your traffic is) — treat as similar to Google Cloud but validate region performance. Concrete instance counts (starting point) • Moderate traffic estimate: ◦ US East: 3 workers ◦ Europe: 2 workers • High traffic or critical availability: ◦ US East: 4–6 workers ◦ Europe: 3–4 workers Start with the moderate plan and scale up based on metrics below. Worker sizing and configuration • Keep identical worker code, dependencies, env vars (except region-specific config like TTS endpoint or API key if required). • Ensure each worker authenticates with the same

agent_name

and has correct AI/TTS API keys injected securely via `--secrets-file`/`--secrets`. • If a provider has region-specific endpoints or keys (e.g., Google Cloud project region), configure worker environment to use that endpoint. Routing & affinity • Rely on LiveKit geographic affinity so calls are dispatched to nearby workers by default. • For stricter routing (e.g., ensure UK calls go to Europe), add a lightweight routing layer that: a. Detects inbound number country (US vs UK). b. Tags the job or sets a preference so dispatch favors the corresponding region. • Ensure fallbacks so if preferred-region workers are saturated, jobs route to the other region. TTS / LLM latency considerations • Measure Time to First Token (TTFT) for OpenAI and TTS response times for Google/ElevenLabs from each region. • If OpenAI latency from Europe is noticeably higher, consider routing European calls to US East only when necessary or using a caching/response-buffering strategy for predictable prompts. • For TTS, prefer Google Cloud in regions where it has the lowest latency; use ElevenLabs where it performs better or for voice quality needs. Implement per-region preference logic. Autoscaling & capacity planning • Autoscale workers per region based on: ◦ Concurrent active calls per worker ◦ Average CPU / memory and external API wait times ◦ Job queue length or dispatch rejections • Keep a minimum of 2–3 warm workers per region to avoid cold-start latency. • Add headroom (e.g., 20–30%) over expected peak concurrency to prevent rejection. Secrets, failover & reliability • Inject secrets via CLI `--secrets-file`/`--secrets`. Do not hardcode keys. • Monitor worker heartbeats and dispatch failures; ensure automatic restart and replacement. • Use retries and exponential backoff for TTS/LLM calls; degrade gracefully (e.g., lower-quality TTS or shorter LLM prompts) if external provider latency spikes. Monitoring & observability • Track: ◦ Dispatch latency and job routing by region ◦ TTFT/TTFB for LLM and TTS per region/provider ◦ Per-worker CPU/memory, active sessions, error rates ◦ Inbound call distribution by number country • Create alerts for sustained elevated TTFT/TTFB, high error rates, or dispatch rejections. Next steps I can help with • Help pick specific cloud regions (e.g., eu-west1, europe-west2, us-east1) based on your phone provider regions. • Provide sample autoscaling thresholds and a simple routing snippet to prefer region based on inbound number. • Recommend monitoring metrics and alert thresholds tailored to your expected concurrency. Which next step would you like me to start with?

should I use any Dispatch rules?

Short answer: Yes—use Dispatch rules to control routing for US vs UK/Europe calls. Why: Dispatch rules let you map incoming calls (by phone number, room prefix, or metadata) to specific agent names/regions so calls prefer nearby workers while still allowing fallback to other regions if capacity is low. Recommended setup for your US East + Europe scenario • Use a single logical agent pattern (e.g.,

voice-agent

) but create region-specific agent names for routing control:

voice-agent-us-east

and

voice-agent-eu

. • Create dispatch rules that match inbound phone numbers or room prefixes: ◦ US numbers → rule that assigns

voice-agent-us-east

◦ UK numbers → rule that assigns

voice-agent-eu

• Include a room naming convention (e.g.,

call-us-<id>

,

call-eu-<id>

) to make rules deterministic and auditable. • Configure each region to run multiple workers with the same region agent name (≥3 total across regions, at least 2 per region recommended) for HA and round-robin within a region. Fallbacks and capacity handling • Add a fallback rule that allows dispatch to the other region if preferred-region agents are saturated. • Monitor dispatch rejections and job queue lengths and autoscale region workers accordingly. Provider affinity • For LLM/TTS latency, prefer workers in regions nearest the provider endpoints: ◦ Use Google Cloud TTS from the region where your Google project is hosted. ◦ Use ElevenLabs where it shows better latency/quality for that region. • If OpenAI latency from Europe is high, allow EU jobs to failover to US East when needed. Operational tips • Inject API keys via `--secrets-file`/`--secrets` and keep worker configs identical except for region-specific endpoints. • Track metrics per dispatch rule/agent: dispatch latency, TTFT/TTFB, errors, and rejections. • Keep a small warm pool (2–3) per region to avoid cold starts. If you’d like, I can: • Draft the exact JSON dispatch rules or example config for LiveKit Phone Numbers/SIP that map US/UK numbers to the two agent names. • Suggest autoscaling thresholds and sample monitoring alerts. Which would you like next? Sources: Agent dispatch | LiveKit docs | Agents telephony integration | LiveKit docs | sip | livekit/livekit_composite | DeepWiki | https://blog.livekit.io/introducing-livekit-phone-numbers-zero-to-ringing-in-60-seconds/#/portal/signup| | Custom agent deployments | LiveKit docs

• Draft the exact JSON dispatch rules or example config for LiveKit Phone Numbers/SIP that map US/UK numbers to the two agent names.

Thanks — below are ready-to-use JSON examples and a short explanation for mapping your US and UK numbers to region-specific agents (

voice-agent-us

and

voice-agent-eu

). Use via the CLI (

lk sip dispatch create <file>

) or the API. Notes: • Replace the phone numbers, agent names, and metadata values with your real values. • For LiveKit Phone Numbers,

trunkIds

can be left as an empty array. • Metadata is a JSON string your workers can parse from JobContext. US number → US agent (room prefix "us-call")

{
  "dispatch_rule": {
    "rule": {
      "dispatchRuleIndividual": {
        "roomPrefix": "us-call"
      }
    },
    "trunkIds": [],
    "criteria": {
      "calledNumber": [
        "+15551234567"
      ]
    },
    "roomConfig": {
      "agents": [
        {
          "agentName": "voice-agent-us",
          "metadata": "{\"region\":\"us-east\",\"llm\":\"openai\",\"tts\":\"elevenlabs\"}"
        }
      ]
    }
  }
}

UK number → EU agent (room prefix "uk-call")

{
  "dispatch_rule": {
    "rule": {
      "dispatchRuleIndividual": {
        "roomPrefix": "uk-call"
      }
    },
    "trunkIds": [],
    "criteria": {
      "calledNumber": [
        "+442079460958"
      ]
    },
    "roomConfig": {
      "agents": [
        {
          "agentName": "voice-agent-eu",
          "metadata": "{\"region\":\"europe\",\"llm\":\"openai\",\"tts\":\"google-cloud\"}"
        }
      ]
    }
  }
}

Fallback rule (allow other region if preferred region saturated)

{
  "dispatch_rule": {
    "rule": {
      "dispatchRuleIndividual": {
        "roomPrefix": "call-fallback"
      }
    },
    "trunkIds": [],
    "criteria": {
      "calledNumber": [
        "+15551234567",
        "+442079460958"
      ]
    },
    "roomConfig": {
      "agents": [
        {
          "agentName": "voice-agent-us",
          "metadata": "{\"region\":\"us-east\",\"llm\":\"openai\",\"tts\":\"elevenlabs\"}"
        },
        {
          "agentName": "voice-agent-eu",
          "metadata": "{\"region\":\"europe\",\"llm\":\"openai\",\"tts\":\"google-cloud\"}"
        }
      ]
    }
  }
}

If you’d like, I can: • Convert these to the exact CLI commands or a Python snippet to create them via the API. • Adjust rules to match number ranges, SIP trunk IDs, or room naming you prefer. Which would you like next? Sources: sip | livekit/livekit_composite | DeepWiki | examples/voice_agents | livekit/livekit_composite | DeepWiki | Agents telephony integration | LiveKit docs | Agent dispatch | LiveKit docs | Introducing LiveKit Phone Numbers: Zero to Ringing in 60 Seconds

how does this approach work with the previous suggestion to Use a single

agent_name

for all workers so they form one pool and can fail over across regions? Isnt this contradicting?

Great question — not contradictory, just two different patterns with different trade-offs. Here’s a clear breakdown and recommendations so you can pick what fits your needs. Short answer • Single

agent_name

→ best for maximum simplicity and seamless failover (one unified pool across regions). • Region-specific `agent_name`s + Dispatch rules → best for strict region affinity, compliance, or per-region provider preferences. You can also combine approaches: prefer region-specific routing via dispatch rules but allow fallback to the global pool for availability. Details & trade-offs • Single

agent_name

◦ Pros: automatic round-robin across all workers, simple ops, easiest failover if any region loses capacity. ◦ Cons: less control over which region handles a call (may pick a farther worker, increasing latency or using the wrong TTS endpoint). • Region-specific `agent_name`s (e.g.,

voice-agent-us

,

voice-agent-eu

) with Dispatch rules ◦ Pros: deterministic routing so callers are served by nearby workers and region-specific provider endpoints; easier to attach region metadata (TTS preferences). ◦ Cons: needs explicit fallbacks and autoscaling per region to avoid rejections; slightly more config/ops. How to combine them (recommended for your use case) • Primary routing via Dispatch rules to region-specific agents (US calls →

voice-agent-us

, UK calls →

voice-agent-eu

) to keep latency and provider affinity optimal. • Also register a small number of workers using a global

agent_name

(e.g.,

voice-agent-global

) or include both agent names in a fallback dispatch rule so jobs can be handled if a region is saturated or offline. ◦ Example patterns: ▪︎ Preferred: Dispatch rule assigns

voice-agent-us

first; fallback rule lists

voice-agent-eu

or

voice-agent-global

. ▪︎ Alternate: All workers register both agent names (not recommended unless code/config is identical) — simpler failover but harder to enforce provider affinity. • Ensure each region runs multiple instances per agent name (≥2 warm, 3+ recommended overall) and autoscale per-region. Practical config suggestions • Use dispatch rules that list preferred agent(s) in order (preferred region first, then fallback). • Include region and provider preferences in

metadata

so the worker can pick the right TTS/LLM endpoint. • Monitor dispatch rejections and TTFT/TTFB to tune fallbacks and autoscaling. If you want, I can: • Draft JSON dispatch rules that implement preferred-region + fallback to a global pool. • Show the exact metadata and worker registration pattern to use for safe failover. Which would you like me to prepare?