Hello! Any way to properly trace delays and server locations between pipeline "steps"? I'm not in the US and I have this concern that I might have some SEA-US roundtrips due to auto-routing. I know for 100% that 11labs I'm connecting to is US-only, but at the same time I often see auto-designated Singapore server for AST/TTS, which I'd rather avoid. My goal is to know how many roundtrips there are between SEA-US during one exchange of bytes, ideally I'd like to only have one. But I fear due to some routing auto-designations I have multiple.

Thanks for reaching out, Stepan. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

I have:

stt=deepgram.STT(
    model="nova-2-phonecall",
    language="en",
    api_key=settings.DEEPGRAM_API_KEY,
),
tts=elevenlabs.TTS(
    voice_id="aSXZu6bgEOS8MXVRzjPi",
    api_key=settings.ELEVENLABS_API_KEY,
),
llm=groq.LLM(
    model="openai/gpt-oss-120b",
    api_key=settings.GROQ_API_KEY,
),

How do I enable tracing? I already do observability with logfire. There are steps like: agent_session start_agent_activity on_enter user_turn eou_detection agent_turn llm_node llm_request llm_request_run gen_ai.choice tts_node tts_request And so on, but it shows 6-8 seconds because it's the whole duration of the turn, not a specific request-travel.

Thanks — I’m responding to your question about how to enable tracing so you can see per-request/network latency between steps (not just whole-turn duration). What to change now 1. Enable Agent Observability for the session: ◦ Set

record=True

when starting the session:

await session.start(agent=MyAgent(), room=ctx.room, record=True)

• Also ensure Observability is enabled in your LiveKit Cloud project Settings → Data and privacy. 1. Use LiveKit Inference models instead of the direct plugins you’re instantiating now. Replace your current definitions with the

inference

wrappers so LiveKit emits per-request metrics (TTFB, TTFT, durations, region metadata): ◦ example pattern:

session = AgentSession(
         stt=inference.STT(model="deepgram/nova-2-phonecall:en"),
         llm=inference.LLM(model="groq/openai/gpt-oss-120b"),
         tts=inference.TTS(model="elevenlabs/aSXZu6bgEOS8MXVRzjPi"),
         record=True,
     )

What you’ll get • Fine-grained trace spans for each external call (start/end) with timestamps and region metadata. • Per-call metrics:

ttfb

,

ttft

,

duration

you can log or forward to Logfire to separate network transit vs compute time. • Trace view in LiveKit Cloud Agent Insights where you can compare timestamps of

llm_request_start

/

llm_request_end

(and same for

tts_request

) to pinpoint cross-region travel. Why your current spans show whole-turn time • The plugin-style spans (your

llm_request_run

,

tts_node

, etc.) aggregate processing; they don’t include LiveKit’s per-request network telemetry. Switching to

inference.*

is required to surface those network roundtrip timings. Quick diagnostics to run after enabling: • Inspect

llm

metrics: if provider-side TTFT is small but LiveKit

ttft

is large, the extra time is network transit. • Look for gaps between span end of one node and start of next (e.g.,

llm_request_end

→

tts_node_start

) — gaps mean data crossed regions. • Check region metadata on spans to confirm where each step ran. If you want, tell me: • Do you want example code for wiring the

inference

events into your existing Logfire logging? • Or would you like steps to pin region (so agents run in US or SEA) after you verify the roundtrips? Sources: https://blog.livekit.io/streamline-troubleshooting-with-agent-observability/#/portal| | Streamline Troubleshooting with Agent Observability | https://blog.livekit.io/streamline-troubleshooting-with-agent-observability/#/portal/signup| | Introducing LiveKit Inference: A unified model interface for voice AI | Introducing LiveKit Inference: A unified model interface for voice AI

What's the difference between livekit's inference and plugins? Does livekit's inference ensure all the steps are in the US and the delay between steps is minimal?

Thanks — I’m responding to your question about how LiveKit’s Inference differs from plugin usage, and whether Inference guarantees US-only execution and minimal inter-step delay. Short answer • Plugins are direct client-side integrations you call from your agent code; Inference is a LiveKit-managed, unified path through LiveKit’s infrastructure. • Inference does not universally “force every step to run in the US” by default, but it gives LiveKit the control and telemetry needed to keep steps co-located and minimize cross-region hops when configured to do so. Key differences • Plugins (what you’re using now:

deepgram.STT

,

groq.LLM

,

elevenlabs.TTS

): ◦ Your agent directly calls provider endpoints. ◦ You see higher-level spans and aggregated turn times; per-request network telemetry (TTFB/TTFT, region metadata) is not emitted by LiveKit. ◦ Routing and network path depend on where your agent runs and the provider’s public endpoint (so an agent in SG calling a US-only ElevenLabs will cause SEA↔️US roundtrips). • LiveKit Inference (

inference.STT

,

inference.LLM

,

inference.TTS

): ◦ LiveKit acts as the client to the model provider within LiveKit’s infrastructure. ◦ Emits fine-grained metrics and trace spans (per-request

ttfb

,

ttft

, durations, region metadata) visible in Agent Observability. ◦ Enables global co-location and provisioned capacity (LiveKit can route calls over private backbones or provider peering where available), which reduces network hops and queuing. ◦ Because LiveKit controls where inference calls run, you can achieve co-location of agent + model calls more reliably than with direct plugins. Does Inference guarantee all steps in the US and minimal delay? • Guarantee: No automatic global guarantee that everything will run in the US for every session. Region placement depends on project/agent routing and your configuration. • Minimizes delays: Yes, when you configure region affinity or use LiveKit Cloud regions and Inference together, LiveKit can co-locate agent execution with inference services (and use private/backbone paths), which significantly reduces cross-region roundtrips. For US-only providers (like ElevenLabs), running the agent in a US region ensures a single SEA→US roundtrip rather than SEA→SG→US→SG→SEA. What to do to ensure US execution and lowest latency • Use LiveKit Inference for STT/LLM/TTS so LiveKit emits per-call telemetry and can co-locate work. • Enable Agent Observability (

record=True

) and inspect `ttfb`/`ttft` and region metadata to confirm where calls run. • Set region pinning / region affinity for the agent or project to

us

(so LiveKit will reject non-US placements and keep steps in US). Note: region pinning requires Scale plan enablement. • If a provider is global (Cartesia, Inworld, etc.), consider switching to a provider available in SEA to eliminate transits entirely. If you want, I can: • Give the exact code changes to switch your current plugin instantiation to

inference.*

and hook metrics into your Logfire logs, or • Show where in Agent Insights to check the per-call `ttfb`/`ttft` and region tags so you can confirm current routing. Which would you prefer?