Thanks — I hear you: you’re seeing frequent TTFT spikes (~1.5s) with GPT‑4.1 on LiveKit that don’t match your expectation from the benchmark page. Let’s isolate causes and get you a path to fix. Quick checklist — please confirm any that apply so I can narrow this down: • Are your agents and inference endpoints deployed in the same region as the LiveKit inference model? • Do you see the spikes at particular times of day or under higher concurrency? • Are you using tool calls or long conversation context before the LLM call? (tools and large prompts add prep time) • What proportion of sessions show the spike (you said >20%) and do you have per-request TTFT traces from Agent Observability? What’s likely and what you can do now • Region/co‑location: If your agent isn’t co‑located with the inference endpoint, network RTT can add large, variable TTFT. Verify and, if needed, move the agent to the model’s region. • Tool calls & pipeline steps: Each tool or blocking pre-processing step delays generation start. Limit sequential tool steps or move non‑critical work off the request path. • Context size: Large prompts or long histories increase preprocessing time. Trim or summarize history before sending. • Concurrency / instance limits: Worker CPU throttling or instance burst limits under load cause delays. Check instance types and scale workers during peaks. • Dynamic routing / backend load: Provisioning or upstream provider congestion can cause variability. Using a fallback/alternate model or LiveKit’s routing strategies can smooth latency. Immediate diagnostics to run 1. Pull TTFT traces in Agent Observability for affected requests to see whether delay is in the LLM stage or earlier (STT/tool/prep). 2. Compare TTFT by region and by time-of-day to spot routing or load patterns. 3. Reproduce a slow request with minimal context and no tools — if it’s still ~1.5s, likely a network/provisioning issue. 4. Run a concurrency stress test to see if TTFT correlates with session count or CPU saturation. Short-term mitigations • Route some traffic to a lower‑latency fallback model for sensitive flows. • Reduce context and tool steps for real‑time paths. • Scale worker instances or move to non‑burstable instance types to avoid CPU throttling. If you confirm the answers to the checklist items (region, concurrency pattern, tool usage) and share one TTFT trace sample from Agent Observability (timestamp + request id), I’ll recommend the exact next steps tailored to your setup. Sources: Understand and Improve Agent Latency | LiveKit | Voice Agent Architecture: STT, LLM, and TTS Pipelines Explained | LiveKit | Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit | Build Your First AI Voice Agent in Python: Complete Tutorial | LiveKit, with interactive elements