https://livekit.io logo
Join Slack
Powered by
I want to integrate livekit on my backend and fron...
# ask-ai
c
I want to integrate livekit on my backend and frontend. In backend I am using Django (for api's) and celery (for background task's) For now I will test the functionality with jQuery and html but later I will integrate it will reactjs. Now my points that I need to do is: • I want to allow each user to create their own ai agents and they can voice call it using test button. • I want to know how I can create multiple ai agents for one to one talk with ai • I check the python code but it work with session that I can't use as I am running API's • Can I run the agent on celery as background task or do I need to run agents separate Please suggest and guide
t
Thanks for reaching out, Vishal. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To integrate LiveKit with your Django backend and frontend (jQuery/React), here are the recommended approaches for your requirements: • Allowing users to create and voice call their own AI agents: - Use your Django backend to generate LiveKit tokens and manage API calls for room creation and agent session management. - On the frontend, you can start with the LiveKit JavaScript SDK for jQuery/HTML, and later switch to the React starter apps for a more robust integration. - Each user can have a dedicated agent session, managed by the backend and orchestrated using LiveKit's agent framework. • Creating multiple AI agents for one-to-one conversations: - The AgentSession concept allows you to manage multiple agents, each handling a separate user interaction. - You can implement custom workflows for agent handoff or delegation if needed. • Python SDK and session handling in stateless APIs: - LiveKit's Python SDK is designed for stateful, long-running agent processes that maintain persistent WebRTC connections. - Instead of stateless HTTP sessions, agents are managed as workers that connect to rooms and handle real-time communication. - Your Django API should focus on orchestration (token generation, room creation), while agents run as separate services. • Running agents with Celery or as separate services: - Agents should not be run as Celery background tasks. They are intended to be persistent, stateful services (workers) that maintain live connections. - Use LiveKit's built-in job scheduling and dispatch APIs to manage agent lifecycles and connect users to available agents. For rapid prototyping, you can use the React starter apps and adapt the backend logic from the Flask example to Django. When scaling, deploy agents as separate worker services for optimal performance and reliability. This architecture leverages LiveKit's stateful, real-time communication model for seamless AI agent interactions. Sources: AI Agents Framework | livekit/livekit_composite | DeepWiki | Agents Overview | LiveKit Docs | Integrating with your frontend | LiveKit Docs | complex-agents/ivr-agent/README.md | Web and mobile frontends | LiveKit Docs
c
How I check all created agents. Delete any or modify any I want.
t
Hey there! Let me walk you through managing your AI agents in LiveKit - it's super easy and I'll help you every step of the way. • List all agents: - Want to see what agents you've got? No problem! Just use the 
ListAgents
API or this friendly CLI command: 
lk agent list
- This will give you a complete rundown of all your awesome agents. 😊 • Delete an agent: - Sometimes you need to say goodbye to an agent. No worries! Use the 
DeleteAgent
API or this simple CLI command: 
lk agent delete --id AGENT_ID
- Don't worry, you'll get a chance to double-check before anything gets deleted. We've got your back! 👍 • Modify an agent: - Need to tweak something? We make it easy! Use the 
UpdateAgent
API or this handy CLI command: 
lk agent update --id AGENT_ID [--secrets KEY=VALUE]
- You can update agent metadata, secrets, and regions - whatever you need. 🛠️ The best part? You can do all of these with the Go SDK, the 
lk
CLI tool, or directly through the CloudAgent service APIs. Just remember, you'll need admin-level authentication to make these changes. We're here to help make managing your agents a breeze! 🚀 Sources: agent_client.go | cmd/lk/dispatch.go | livekit/livekit_cloud_agent.pb.go
c
I was checking your agent.py file for quick start I have some questions • How I can pass my arguments instead of hard code instructions. • How I can pass my meta data to Assistant function for later use or track my customers
t
Hey there! Let me walk you through some awesome ways you can make your agents super personalized and smart about tracking customer info. It's easier than you might think! • Passing custom arguments (like instructions): - Want to give your agent specific guidance? No problem! Create a custom Agent class that can take in special parameters. - Instead of hardcoding instructions, you can now pass them when you create your agent, making everything more flexible.
Copy code
class MyAgent(Agent):
    def __init__(self, instructions):
        super().__init__(instructions=instructions, ...)

# Usage
agent = MyAgent(instructions="Your custom instructions here")
- Pro tip: You can even use context variables to make your instructions super dynamic. • Passing and storing metadata for tracking customers: - Think of this like creating a digital profile for each customer. Use the 
userdata
attribute to store custom info you might want to remember.
Copy code
from dataclasses import dataclass

@dataclass
class MySessionInfo:
    user_name: str | None = None
    customer_id: str | None = None

session = AgentSession[MySessionInfo](userdata=MySessionInfo(...))
- This is super handy because the metadata sticks around during the whole session, so you can use it whenever you need. • Passing initial data via job metadata: - When you're sending out an agent, you can include some extra info right in the job. Then grab it in your entrypoint function.
Copy code
import json

async def entrypoint(ctx: JobContext):
    metadata = json.loads(ctx.job.metadata)
    user_id = metadata["user_id"]
    # Use this data to initialize your agent or session
With these methods, you'll be creating super adaptable agents that remember and care about each customer's unique needs. Pretty cool, right? Sources: Job lifecycle | LiveKit Docs | complex-agents/personal_shopper/README.md | Workflows | LiveKit Docs
c
Thats a good way to make it dynamic but the command (python agent.py dev) didn't accept other arguments so how I can send my meta data or instruction data to it or may be some id so I can fetch data later.
t
Hey there! Let me break down how to work with the 
python agent.py dev
command in a way that makes your life easier. When you're developing, this command is pretty straightforward, but you might be wondering how to pass custom data to your agent. No worries—I've got you covered! • Passing metadata or instructions is super simple: - When you're dispatching your agent (through the API or CLI), you can include your custom data as a JSON string in the 
metadata
field. It's like giving your agent a little note with extra details.
Copy code
lk dispatch create \
  --new-room \
  --agent-name your-agent \
  --metadata '{"user_id": "12345", "instructions": "Custom instructions"}'
- Inside your agent's 
entrypoint
function, you can easily grab this metadata using 
ctx.job.metadata
and parse it. Think of it like unpacking a care package!
Copy code
import json

async def entrypoint(ctx: JobContext):
    metadata = json.loads(ctx.job.metadata)
    user_id = metadata["user_id"]
    instructions = metadata["instructions"]
    # Now you can use these values to make your agent super personalized
This approach is like giving your agent a Swiss Army knife—it can adapt to different users and scenarios, making your code flexible and ready for anything. Pretty cool, right? Sources: External data and RAG | LiveKit Docs | Agent dispatch | LiveKit Docs | Workflows | LiveKit Docs | Job lifecycle | LiveKit Docs | README.md
c
I am using python and when I try using this command 
python3 agent.py console --metadata='{"user_id": "12345", "instructions": "Custom instructions"}'
I got an error message Usage: agent.py console [OPTIONS] Try 'agent.py console --help' for help. Error: No such option: --metadata
t
Hey there! I see you're running into a tricky situation with the 
console
command in 
agent.py
. No worries – the 
--metadata
option isn't supported here, which is why you're seeing that "No such option" error. Think of the 
console
command like a quick, no-frills testing tool for local development. Let me walk you through some friendly ways to test your agent with custom metadata: • Dispatch the agent with metadata using the API:
Copy code
import asyncio
from livekit import api

async def create_dispatch():
    lkapi = api.LiveKitAPI()
    await lkapi.agent_dispatch.create_dispatch(
        api.CreateAgentDispatchRequest(
            agent_name="your-agent-name",
            room="test-room",
            metadata='{"user_id": "12345", "instructions": "Custom instructions"}'
        )
    )
    await lkapi.aclose()
• Modify your agent code for console testing: - In your 
entrypoint
function, check if 
ctx.job.metadata
exists. If not, provide default metadata for local testing.
Copy code
async def entrypoint(ctx: JobContext):
    import json
    if ctx.job.metadata:
        metadata = json.loads(ctx.job.metadata)
    else:
        metadata = {"user_id": "console_test", "instructions": "Default instructions"}
    # Use metadata as needed
Pro tip: For the most comprehensive metadata testing, I recommend using the agent dispatch API or CLI in a real LiveKit server environment. The 
console
command is perfect for those quick, simple local tests. Hope this helps you out! Sources: livekit-agents/livekit/agents/cli/cli.py | examples/agent_dispatch.py | Job lifecycle | LiveKit Docs
c
Give me an example of how I can create multiple Agents using create_dispatch API. I want to run the Agent only if some user hit my API /start_call
t
Hey there! Let me walk you through a super cool way to create and run multiple agents dynamically when a user hits your 
/start_call
API. We'll use the 
create_dispatch
API to spin up a new agent for each request. I've got a friendly Python example using FastAPI that'll make this crystal clear:
Copy code
from fastapi import FastAPI, Request, HTTPException
from livekit import api
import asyncio
import json
import uuid

app = FastAPI()
AGENT_NAME = "call-agent"

@app.post("/start_call")
async def start_call(request: Request):
    data = await request.json()
    user_id = data.get("user_id")
    if not user_id:
        raise HTTPException(status_code=400, detail="user_id is required")
    room_name = f"call-{user_id}-{uuid.uuid4().hex[:8]}"
    metadata = json.dumps({
        "user_id": user_id,
        "instructions": data.get("instructions", "Default instructions")
    })
    lkapi = api.LiveKitAPI()
    try:
        dispatch = await lkapi.agent_dispatch.create_dispatch(
            api.CreateAgentDispatchRequest(
                agent_name=AGENT_NAME,
                room=room_name,
                metadata=metadata
            )
        )
        return {"status": "success", "dispatch_id": dispatch.id, "room": room_name}
    finally:
        await lkapi.aclose()
• Every time 
/start_call
is hit, we create a brand new agent with a unique room and personalized metadata just for that user. • Pro tip: Set 
agent_name
in your agent worker's 
WorkerOptions
to make dispatching a breeze. • In your agent's 
entrypoint
, you can easily grab the metadata via 
ctx.job.metadata
and tailor the agent experience for each awesome user. Pretty neat, right? This approach lets you effortlessly scale and manage multiple agent sessions, each one kicked off by a user's API call. It's like having a team of friendly assistants ready to jump into action! Sources: Agent dispatch | LiveKit Docs | telephony/make_call/make_call.py
c
For the above start_call function example can you show me an example file agent.py as well
t
Hey there! Let me walk you through an awesome example of an 
agent.py
file that lets you create a super personalized AI assistant. It uses explicit dispatch and customizes the agent for each user using 
ctx.job.metadata
– how cool is that?
Copy code
import json
from livekit.agents import JobContext, WorkerOptions, cli
from livekit.agents.voice import Agent, AgentSession
from livekit.plugins import openai, deepgram, silero

def build_instructions(user_name, instructions):
    return f"You are a helpful AI assistant for {user_name}. {instructions}"

class CustomAgent(Agent):
    def __init__(self, instructions):
        super().__init__(
            instructions=instructions,
            stt=deepgram.STT(),
            llm=openai.LLM(model="gpt-4o"),
            tts=openai.TTS(),
            vad=silero.VAD.load()
        )

async def entrypoint(ctx: JobContext):
    await ctx.connect()
    try:
        metadata = json.loads(ctx.job.metadata)
        user_name = metadata.get("user_name", "Guest")
        instructions = metadata.get("instructions", "How can I help you today?")
    except Exception:
        user_name = "Guest"
        instructions = "How can I help you today?"
    agent = CustomAgent(build_instructions(user_name, instructions))
    session = AgentSession()
    await session.start(agent=agent, room=ctx.room)

if __name__ == "__main__":
    cli.run_app(
        WorkerOptions(
            entrypoint_fnc=entrypoint,
            agent_name="call-agent",  # Must match the agent_name used in your dispatch
        )
    )
• This friendly agent will kick off with 
create_dispatch
and use the metadata you send (like 
user_name
and 
instructions
) to create a totally personalized experience for each user. • Pro tip: Make sure to set 
agent_name
in 
WorkerOptions
to match the name you use in your dispatch API call.
c
Its a good example for single agent running for a business but for multi business site single agent is not feasible. I want the agent name to be dynamic so I can send the name from my db.
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
c
Can you tell me how I can show a test call button in Django admin to voice call with my running agent?
7 Views
Open in Slack
PreviousNext
Powered by