Does the default OpenTelemetry tracing in LiveKit Agents, when wired to Langfuse, actually solve for application and LLM observability, or is it mainly infrastructure observability? We are seeing a mismatch in practice. The default setup treats an entire voice session as a single trace, with nested spans for each node like STT, LLM, TTS, and agent_turn. This makes per turn latency metrics difficult because everything aggregates to the full session duration. It also makes it hard to represent turn boundaries, barge in, and conditional node execution in a way that supports turn level analysis. More importantly, there appears to be a mismatch between how LiveKit Agents emits trace data via OpenTelemetry and what an LLM observability platform like Langfuse expects. In Langfuse we do not see INPUT and OUTPUT populated anywhere, not at the session level, not at the trace level, and not even on the LLM generation. As a result we cannot easily inspect per turn INPUT/OUTPUT, run turn level evaluations, or isolate whether a specific turn degraded versus the overall session. This also impacts failure mode analysis and N plus one style scoring against conversation history, since the trace structure does not naturally align with conversational turns. Is this a known limitation of the current OpenTelemetry integration, or am I misunderstanding how it is intended to be shaped for LLM observability rather than just infrastructure tracing? Is custom instrumentation for turn level generations the expected pattern here?

In Langfuse the otel semantics are different… I believe they look something like

trace (turn) > span (stt) > span (llm) > span (tts)

but in LiveKit Agents we have

trace (session) > span (turn) > span (stt) > span (llm) > span (tts)

…so yes, there’s a difference. But I don't think anything is missing that would prevent your analysis. Also, you may find LiveKit Agent Insights helpful as you explore this: https://blog.livekit.io/streamline-troubleshooting-with-agent-observability/

@refined-appointment-81829 thanks! So this Agent Observability is only available when I use Agent Cloud Hosting?

It's also available for self hosting The latency is reported to langfuse I believe as TTFT (shown in Langfuse as TTFT kind of banner), and the duration of a turn is what Langfuse calls the "latency" somewhat confusingly, but I think the data is there: whilst they are all subspans within a "session" the data isn't aggregated and discarded Livekit does provide a on_metrics call back, so in principle you can attach extra metrics you like to the spans in your own client code I believe

awesome! @colossal-airport-32984 i'm looking now in my UI but my UI i different then the Youtube video on the above link... e.g. when I go to "Agents" in LiveKit Cloud ... and then go to one of my Agent workers, there is no "session id" there is "worker id" but it doesn't allow me to click on it, like the video is able to

You have to enable it in your settings first

FYI i'm paid customer of LiveKit Cloud, but currently i'm self hosting the agents, I still need and want to migrate it to Hosted Agents btw, just haven't had the time for it yet 🙂

thank you! 🙌

FYI, once you enable that is can take a minute or so to actually be enabled in the backend. So you don't get your trace. Try a new session in a little bit.

trying it out now, really cool! Btw my original question, I still would need a different kind of LLM Observability for evaluating my responses, and be able to use evaluators and fine the prompt based on the scores of those evaluators, you know what I mean? This what agent observability i'm now enabling (all thought that's also very cool and awesome to have additionally!), I would still also want a specific observability in Langfuse (or Langsmith) that is more focused on talk turns and containing input and outputs

At the risk of not actually answering your question have you looked at Agent evals? https://docs.livekit.io/agents/start/testing/

yeah I think this is only for offline tests... ideally I would also be able to use it realtime with online tests

I think the cache is for 5-10 mins

cool it's working now! 🙏