https://livekit.io logo
Join Slack
Powered by
explain to me the difference between using lang gr...
# ask-ai
h
explain to me the difference between using lang graph as the llm versus the session.start agent
t
Thanks for reaching out, Julian. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
The difference between using LangGraph as the LLM and using 
session.start
lies in the level of customization and conversation flow management: • LangGraph as the LLM: - Allows for structured workflows using a StateGraph, enabling you to define specific nodes and transitions for more control over dialogue flow. - Integrates with the LangChain ecosystem, supporting advanced use cases with multiple steps or integrations. - Offers customization through the 
LLMAdapter
, making it compatible with LiveKit while leveraging LangGraph's structured approach. • Session.start: - Simplifies setup for basic interactions by using LiveKit's built-in LLM support (e.g., OpenAI, Google). - Handles responses directly without requiring predefined structured workflows, making it suitable for straightforward conversation scenarios. Choose LangGraph for structured, multi-step workflows and 
session.start
for simpler, direct interactions.
h
show me a full example of using langgraph with livekit
t
Here’s a complete example of how you can use LangGraph with LiveKit to create a voice agent. I’ve broken it down step-by-step to make it easy to follow: 1. Install Dependencies: 
pip install "livekit-agents[openai,langchain,deepgram,silero,turn_detector]" langchain[openai] langgraph
2. Example Code:
Copy code
import logging
from typing import Annotated, TypedDict

from dotenv import load_dotenv
from langchain.chat_models import init_chat_model
from langchain_core.messages import BaseMessage
from langgraph.graph import START, StateGraph
from langgraph.graph.message import add_messages

from livekit.agents import (
    Agent,
    AgentSession,
    JobContext,
    JobProcess,
    RoomInputOptions,
    WorkerOptions,
    cli,
)
from livekit.plugins import deepgram, langchain, silero
from livekit.plugins.turn_detector.multilingual import MultilingualModel

logger = logging.getLogger("langgraph-agent")

load_dotenv()

class State(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]

def create_graph() -> StateGraph:
    openai_llm = init_chat_model(
        model="openai:gpt-4o",
    )

    def chatbot_node(state: State):
        return {"messages": [openai_llm.invoke(state["messages"])]}

    builder = StateGraph(State)
    builder.add_node("chatbot", chatbot_node)
    builder.add_edge(START, "chatbot")
    return builder.compile()

async def entrypoint(ctx: JobContext):
    graph = create_graph()

    agent = Agent(
        instructions="",
        llm=langchain.LLMAdapter(graph),
    )

    session = AgentSession(
        vad=ctx.proc.userdata["vad"],
        stt=deepgram.STT(model="nova-3", language="multi"),
        tts=deepgram.TTS(),
        turn_detection=MultilingualModel(),
    )

    await session.start(
        agent=agent,
        room=ctx.room,
        room_input_options=RoomInputOptions(
            # Optional: Enable noise cancellation
            # noise_cancellation=noise_cancellation.BVC(),
        ),
    )
    await session.generate_reply(instructions="ask the user how they are doing?")

def prewarm(proc: JobProcess):
    proc.userdata["vad"] = silero.VAD.load()

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, prewarm_fnc=prewarm))
3. Steps to Run:
Copy code
OPENAI_API_KEY=your_openai_key
DEEPGRAM_API_KEY=your_deepgram_keypython langgraph_agent.py
◦ First, create a 
.env
file and add your API keys like this: ◦ Then, just run the script: 1. What to Expect: Once you run it, the agent will join the room, generate replies, and chat with users using voice commands. It’s pretty neat! If you want to tweak anything or have questions about specific parts, just let me know—I’m happy to help!
h
would it be possible to use langgraph in a class like this class MyVoiceAgent(Agent):
t
Absolutely, you can use LangGraph within a custom class like 
OctavianVoiceAgent
. Here’s a simple example to get you started:
Copy code
from langgraph.graph import StateGraph
from livekit.agents import Agent, AgentSession
from livekit.plugins import langchain

# Define your LangGraph workflow
def create_workflow() -> StateGraph:
    """This is where you define your LangGraph workflow."""
    def chatbot_node(state):
        return {"messages": []}
    
    builder = StateGraph()
    builder.add_node("chatbot", chatbot_node)
    builder.add_edge(START, "chatbot")
    return builder.compile()

class OctavianVoiceAgent(Agent):
    def __init__(self, graph: StateGraph):
        super().__init__(
            llm=langchain.LLMAdapter(graph),
            # You can add other features like STT, TTS, etc. here
        )

async def entrypoint(ctx: JobContext):
    # First, initialize the LangGraph
    graph = create_workflow()
    
    # Then create your custom agent instance
    agent = OctavianVoiceAgent(graph)
    
    session = AgentSession(
        # Set up your session with the necessary components
        vad=silero.VAD.load(),
        stt=deepgram.STT(),
        tts=deepgram.TTS(),
    )
    
    await session.start(
        agent=agent,
        room=ctx.room,
    )
Just a few things to keep in mind: • Wrap your LangGraph workflow using 
langchain.LLMAdapter
to smoothly integrate with LiveKit. • Extend the base 
Agent
class and pass your LangGraph workflow when you initialize it. • Make sure you have the necessary plugins installed:
Copy code
pip install "livekit-plugins-langchain~=1.1"
If you want, I’m here to help with any specific parts of your workflow or if you need help setting up extra configurations! Sources: livekit-agents/livekit/agents/voice/agent.py
h
can livekit tools be added to a langgraph agent like the one you just shared
t
Absolutely! You can definitely add LiveKit tools to a LangGraph agent. Here’s a friendly step-by-step on how to do it: 1. Define LiveKit Tools: Start by using the 
@function_tool
decorator to set up your tools. For example:
Copy code
from livekit.agents import function_tool

@function_tool()
async def lookup_weather(context, location):
    """Look up weather information for a given location."""
    return {"weather": "sunny", "temperature_f": 70}
2. Create LangGraph Workflow: Next, define your LangGraph workflow by adding nodes and edges, then compile it:
Copy code
from langgraph.graph import StateGraph

def create_workflow():
    workflow = StateGraph()
    # Add nodes and edges here
    return workflow.compile()
3. Integrate Workflow with LiveKit: Wrap your LangGraph workflow using the 
LLMAdapter
like this:
Copy code
from livekit.plugins import langchain

agent = Agent(
    instructions="",
    llm=langchain.LLMAdapter(graph=create_workflow()),
    tools=[lookup_weather]  # Add tools here
)
4. Add Tools Dynamically: And if you want to add tools later on, no problem! Just do: 
await agent.update_tools(agent.tools + [new_tool])
This way, you get to blend LangGraph workflows with LiveKit’s handy tools to boost what your agent can do. If you want me to walk you through anything else or share more examples, just let me know—I’m here to help!
4 Views
Open in Slack
PreviousNext
Powered by