One way I’ve been trying to measure latency (of th...
# ask-ai
s
One way I’ve been trying to measure latency (of the LLM) is by using the “console –text” and looking at the logs. But I believe that is not dumped into “Agent Insights”. Is this correct?
t
Thanks for reaching out, Marc. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — good question. You're correct: latency logs you see when running the agent in
console --text
are not sent to Agent Insights. •
console --text
runs locally and does not upload session data to LiveKit Cloud, so those local console logs (including any LLM latency lines) aren't collected by Agent Insights. • Agent Insights only captures data from recorded agent sessions that are running with observability/recording enabled in LiveKit Cloud. If you want latency visible in Agent Insights or centralized telemetry, here are two options you can use: • Run the agent as a recorded session in LiveKit Cloud with Agent Insights/observability enabled. That will capture session-level metrics and traces instead of just local console output. • Add explicit telemetry that reports LLM timings to your observability stack (for local dev or production). For example, emit timing events to: ◦ your existing metrics/telemetry backend (Prometheus, Datadog, etc.), or ◦ a custom event upload that your service sends to LiveKit Cloud when running remote/recorded sessions. If you tell me which approach you prefer (enable recorded sessions for Agent Insights, or add custom telemetry), and whether you’re testing locally or in Cloud, I can give step-by-step guidance for the next steps. Sources: Server startup modes | LiveKit Documentation | Why is my Agent insights data missing? | LiveKit | Streamline troubleshooting with Agent Observability
s
yes but that means I cannot do only text in an easy way no? if I do "uv run main.py dev" and connect via playground and send text, it will run STT-TTS
t
Thanks — good clarification. You're right to be thinking about how the dev/run modes interact with STT/TTS. Short answer: you can run text-only easily in dev mode and avoid STT/TTS. Running
uv run main.py dev
and sending text from the Playground does not trigger STT. Text sent from the frontend is published on the
lk.chat
text stream and handled as text. TTS only runs if audio output is enabled. What to do to measure LLM latency with text-only: 1. Disable audio input and output when you start the AgentSession so the agent neither listens for audio (no STT) nor synthesizes speech (no TTS). 2. Keep text input/output enabled so the Playground sends text and the agent replies over the
lk.transcription
text stream. Example configurations: • Python
Copy code
await session.start(
    agent=MyAgent(),
    room=ctx.room,
    room_options=room_io.RoomOptions(
        audio_input=False,   # no STT
        audio_output=False,  # no TTS
        text_input=True,
        text_output=True,
    ),
)
• Node.js
Copy code
await session.start({
    agent: new MyAgent(),
    room: ctx.room,
    inputOptions: { audioEnabled: false, textEnabled: true },
    outputOptions:{ audioEnabled: false, transcriptionEnabled: true },
});
How to receive agent text replies in the frontend: • Listen to the
lk.transcription
text stream (agent responses are published there when audio output is disabled). If you want latency measurements in Agent Insights: • Agent Insights only collects data from recorded sessions running in LiveKit Cloud with observability enabled. Local/dev
console --text
logs are not uploaded to Agent Insights. • If you need centralized latency metrics for text-only runs, either run the session as a recorded cloud session with observability enabled, or emit your own telemetry (timing events) to your metrics backend during the text-only flow. If you want, tell me: • Whether you want step-by-step changes for your current code (share the session start snippet), or • Whether you want to capture latency in Agent Insights (so I can explain how to run a recorded session and enable observability), or • Which telemetry backend you use so I can suggest how to emit timing events for local dev. Sources: Text and transcriptions | LiveKit Documentation | How to build an agent with speech-to-text input and text-only output | Live...
s
on another note. What kwargs can I pass to openai 5.3 ?
t
Thanks — I’m following up about your question on “OpenAI 5.3” kwargs. Quick clarification before I list parameters: by “OpenAI 5.3” do you mean • the OpenAI model ID (e.g., a hypothetical model named gpt-5.3), or • the OpenAI SDK/client version 5.3? I ask because the LiveKit docs don’t mention a model explicitly named “5.3.” Once you confirm which you mean, I can give accurate guidance. In the meantime, here’s what I can tell you that will likely be useful either way: • LiveKit lets you pass provider-specific parameters to LLMs via
extra_kwargs
on the
inference.LLM()
constructor (or equivalent provider config). Typical OpenAI parameters you can pass include: ◦
model
— the model ID to use. ◦
temperature
— controls randomness (common range 0–2). ◦
max_tokens
/
max_completion_tokens
— limit output length. ◦
top_p
,
presence_penalty
,
frequency_penalty
— sampling controls. ◦
stop
— stop sequences. ◦ Any provider-specific flags (for example:
reasoning_effort
,
tool_choice
,
parallel_tool_calls
) if supported by the integration. • If you want exact, model-specific kwargs (for a particular OpenAI model), confirm the model name and I’ll list the parameters you can pass through LiveKit and how to supply them (e.g.,
extra_kwargs={...}
). Which did you mean — model gpt-5.3 or SDK version 5.3? Also tell me whether you’re configuring the LLM in Python or Node, and I’ll give a concrete example. Sources: OpenAI LLM models | LiveKit Documentation