https://livekit.io logo
Join Slack
Powered by
what is langfuse and how can it be integrated, wha...
# ask-ai
b
what is langfuse and how can it be integrated, what are the benefits of using langfuse?
t
Thanks for reaching out, Utkarsh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’ll explain what Langfuse is, how it can be integrated with LiveKit-based workflows, and the benefits you’ll get. What Langfuse is • Langfuse is an open-source observability platform for LLM applications. • It provides a dashboard to trace sessions, inspect prompts and model responses, and analyze latency and cost across agent interactions. How Langfuse can be integrated (with LiveKit-related notes) • Use traces to correlate LLM/agent activity with your LiveKit sessions by setting identifying metadata on spans: ◦ You can set 
user_id
and 
session_id
on spans so Langfuse groups traces by user and session in the dashboard. ◦ In Langfuse integration v1.2.0+ you can customize those fields by adding a custom span processor that sets 
user_id
and 
session_id
during span start. • Implementation approach (high level): a. Add the Langfuse/OpenTelemetry SDK to your service that runs the LLM/agent. b. Implement a 
SpanProcessor
that sets 
user_id
and 
session_id
on spans when they start (this lets Langfuse associate traces with LiveKit users/rooms). c. Export traces to Langfuse (configure the exporter/endpoint and API key). d. In your LiveKit application, ensure you surface the same 
user_id
and 
session_id
values (for example, user IDs and room/session IDs) to the LLM service so they can be attached to spans. What you can do now • If you want to get started, implement a custom span processor in the service that calls the LLM/agent and set 
user_id
and 
session_id
on spans (this is supported starting with the Langfuse integration v1.2.0). • Export traces to your Langfuse instance and verify the traces show the user/session grouping. Benefits of using Langfuse with LiveKit-driven voice/agent experiences • Faster debugging: view full request/response traces to find where prompts or model outputs cause issues. • Session-level visibility: by tagging spans with 
user_id
and 
session_id
, you can trace specific user interactions and replay or inspect them in Langfuse. • Performance insights: measure latencies per call and see where slowdowns occur. • Cost monitoring: analyze model usage and cost per session or user. • Correlation of media and agent behavior: linking LiveKit session identifiers with Langfuse traces helps correlate audio/video events with LLM decisions and responses, improving root-cause analysis. Limitations / current notes • Direct, built-in metadata setting on spans is planned for a future release; for now, use a custom span processor to set 
user_id
and 
session_id
. • The documentation specifically references customizing 
user_id
and 
session_id
via a custom span processor in the Langfuse integration (v1.2.0+): https://kb.livekit.io/articles/5193083926-setting-custom-user-id-and-session-id-in-langfuse-traces If you’d like, tell me: • Which language/runtime your LLM service uses (Node, Python, etc.) and whether you already have tracing set up — I can give a short code example for a custom span processor or point out the exact configuration steps. Sources: Setting custom user_id and session_id in Langfuse traces
b
in python
t
Thanks — I’ll show a concise, practical Python approach to integrate Langfuse traces with your LiveKit sessions and explain the benefits. What we’ll do • Create a custom OpenTelemetry SpanProcessor that sets 
user_id
and 
session_id
on spans when they start. • Configure an OpenTelemetry tracer to use that processor and export traces to Langfuse. • Ensure your LiveKit code and your LLM/agent service share the same 
user_id
and 
session_id
so traces correlate to LiveKit sessions. Why this works • Langfuse groups and surfaces traces by 
user_id
and 
session_id
when those attributes are present. Starting with the Langfuse integration v1.2.0, you can customize these fields via a custom span processor. Python example (conceptual) • This example shows a minimal custom 
SpanProcessor
that sets 
user_id
and 
session_id
on span start. Replace exporter configuration with your Langfuse endpoint/credentials.
Copy code
from opentelemetry.sdk.trace import SpanProcessor, TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry import trace
from opentelemetry.trace import Span

class LiveKitSessionSpanProcessor(SpanProcessor):
    def __init__(self, get_user_and_session):
        # get_user_and_session is a callable that returns (user_id, session_id)
        self.get_user_and_session = get_user_and_session

    def on_start(self, span: Span, parent_context) -> None:
        user_id, session_id = self.get_user_and_session()
        if user_id:
            span.set_attribute("user_id", user_id)
        if session_id:
            span.set_attribute("session_id", session_id)

    def on_end(self, span: Span) -> None:
        pass

    def shutdown(self) -> None:
        pass

    def force_flush(self, timeout_millis: int = 30000) -> bool:
        return True

# Usage example
def get_user_and_session():
    # Return the LiveKit user and session identifiers used by the request.
    # Implement this to read from request context, thread-local, or function args.
    return ("user_123", "session_abc")

# Configure tracer provider
provider = TracerProvider()
trace.set_tracer_provider(provider)

# Add your custom span processor
provider.add_span_processor(LiveKitSessionSpanProcessor(get_user_and_session))

# Add exporter to Langfuse (configure the OTLP exporter/endpoint for your Langfuse instance)
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
otlp_exporter = OTLPSpanExporter(endpoint="<https://your-langfuse-endpoint>", headers=(("x-api-key","YOUR_LANGFUSE_API_KEY"),))
provider.add_span_processor(BatchSpanProcessor(otlp_exporter))

tracer = trace.get_tracer(__name__)

# Example traced operation
with tracer.start_as_current_span("llm_call"):
    # call LLM / agent here
    pass
Important integration notes • get_user_and_session must return the LiveKit user and session/room identifiers for the current request or context. Typical places to source them: ◦ From request headers or session context in your web service. ◦ From thread-local/request-local storage populated when LiveKit events arrive. • Verify traces in Langfuse show 
user_id
and 
session_id
. Langfuse will group traces by those fields. • Direct built-in metadata support on spans is planned in a future Langfuse release; the custom span processor is the recommended approach today (available since Langfuse integration v1.2.0). Benefits you’ll get (recap) • Session-level traceability: link LLM calls to LiveKit users and rooms. • Faster debugging: inspect prompt/response flows per user session. • Performance and cost visibility: latency and model usage per session. • Easier correlation between media events (LiveKit) and agent behavior (LLM). Next steps I can help with • Provide a fuller Python example that reads LiveKit IDs from a Flask/FastAPI request context. • Show how to attach additional metadata (e.g., call_type, media_event) to spans. • Help validate your OTLP exporter settings for Langfuse. Which would you like next? If you tell me your web framework (Flask, FastAPI, Django) and how you receive LiveKit session info, I’ll give a tailored code snippet.
Open in Slack
PreviousNext
Powered by