Hey guys, how can <prompt linking via langfuse> be...
# agents
l
Hey guys, how can prompt linking via langfuse be implemented with livekit agents when using an
llm_node
function in STT->LLM->TTS pipeline? I tried settings attributes on the
llm_node
span but that does not enable langfuse to link traces to specific prompts in the UI. I believe the prompt name and prompt version attributes need to the set on the
llm_request
generation (defined at https://github.com/livekit/livekit_composite/blob/cd1a2500/livekit/agents/livekit-agents/livekit/agents/llm/llm.py), is there a nice way of doing this? Thoughts?
c
Prompt linking assumes you are using a Langfuse managed prompt id and loading it via the Langfuse Python SDK as far as I can see. We do it in our organisation by simply appending it using Langfuse python sdk This code adds a trace to existing session representing the usage of that prompt id. It does not give any chronological information as I am not tying it to a specific LLM generation, just using it as a way to link the session to the prompt - so its not quite used in the way langfuse would imagine The the code snippet
langfuse_resource
is the prompt loaded via LangFuse SDK
Copy code
with self._client.start_as_current_span(name=f"{langfuse_resource.name}") as span:
            span.update_trace(session_id=self._tracing_metadata.langfuse_session_id)
            name = (
                f"Prompt Used in Conversation Session ID: "
                f"{self._tracing_metadata.conversation_session_id}"
            )
            with self._client.start_as_current_observation(name=name, as_type="generation") as obs:
                obs.update(prompt=langfuse_resource)
If anyone is aware of a better way I am interested too