Does the livekit agent hosted on livekit cloud in python stream or send the entire response? More specifically, does the livekit TTS node run after the LLM node has run or does the TTS node start right after the first token from the LLM is received?

Thanks for reaching out, Muhammad. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

If I'm trying to reduce time-to-first-audio latency and testing out reducing the prompt size from 15000 tokens to 1000 tokens, should I worry about the latency in

llm_request

span or

llm_request_run

span? I use OpenAI for the llm_node and elevenlabs for TTS

Thanks — I’m responding to your question about which span to worry about when reducing prompt size to lower time‑to‑first‑audio: the

llm_request

span or the

llm_request_run

span. Short answer • Focus on the

llm_request_run

span for time‑to‑first‑audio. That span represents the actual network/LLM call and streaming of tokens; reducing prompt size shortens what needs to be sent and processed there and will cut the time before the first tokens arrive at TTS. Why •

llm_request

is the higher‑level span that covers the whole LLM request lifecycle (including preparation, token counting, orchestration). It’s useful for overall profiling and cost/token accounting (your trace shows 1,669 → 37 tokens there). •

llm_request_run

is the lower‑level span that includes the POST to OpenAI and the LLM’s streaming output. Time inside

llm_request_run

is the latency before tokens are emitted to downstream nodes (like your TTS node). That directly controls when TTS can start producing audio. Practical notes for your OpenAI + ElevenLabs setup • If OpenAI is used in streaming mode, TTS can begin once the first tokens are received from the

llm_request_run

. So improvements that reduce the time in

llm_request_run

(shorter prompt, faster network, smaller model, or using OpenAI streaming endpoints) will reduce time‑to‑first‑audio. • If you’re not using OpenAI’s streaming output, the LLM will return the full response only after completion — then

llm_request_run

still matters but will include the full generation time (so streaming must be enabled to get early tokens). • Network POST time (your nested

POST

span ~0.69s) is part of `llm_request_run`; consider network latency and connection reuse (keep‑alive) as levers. Recommendations (next steps) 1. Verify you’re using OpenAI streaming endpoints — if not, enable them so the LLM can emit partial tokens to TTS. 2. Measure

llm_request_run

while testing prompt size changes — that’s where you should see the reduction in time‑to‑first‑token. 3. Also watch the TTS request/run spans (ElevenLabs): once tokens arrive, TTS startup time affects first audio. If ElevenLabs supports streaming synthesis, ensure you’re using it. 4. If you want, I can help you: ◦ Show what to look for in the trace to confirm streaming is working, or ◦ Suggest quick checks to isolate network vs LLM generation latency. Which of those would be most helpful right now? , with interactive elements

How do i enable the openai streaming endpoint? I've got this right now:

return openai.LLM(model="gpt-5.1-chat-latest")

Thanks — I’m responding to your snippet and question about enabling OpenAI streaming. Good news: with LiveKit Agents you don’t need to add a special flag — streaming is already enabled by default when you use the OpenAI LLM within an AgentSession. Your code:

return openai.LLM(model="gpt-5.1-chat-latest")

already supports streaming tokens to downstream nodes (so TTS can begin as soon as the first tokens arrive). Notes and options • Ensure you’re running the LLM inside an AgentSession (the Agent pipeline is what wires streaming from LLM → TTS). • If you prefer the Responses API or want alternate wiring, you can use: ◦ openai.responses.LLM for the Responses API, or ◦ LiveKit inference.LLM with provider="openai" • Confirm that your OpenAI provider settings and network allow streaming (no intermediate proxy buffering). If OpenAI streaming is blocked, you’ll see the LLM return only a full response and TTS will wait. If you’d like, I can: • Show a minimal AgentSession example that demonstrates streaming end-to-end, or • Help you check your trace to confirm tokens are streaming (what to look for in the

llm_request_run

span). Which would be most helpful? Sources: OpenAI STT plugin guide | LiveKit Documentation | OpenAI LLM models | LiveKit Documentation | Large language models (LLM) overview | LiveKit Documentation, with interactive elements