This message was deleted.
# ask-ai
s
This message was deleted.
t
Thanks for reaching out, Aditya. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
The best way to add an agent (such as a support or automated participant) to a LiveKit room using a Next.js frontend involves a few key steps: 1. Create a secure token endpoint on your Next.js backend. This endpoint generates an access token for each participant (agent or user), granting room access and appropriate permissions. Generation must be done server-side and should never expose your API secret to the client. 2. Install LiveKit SDKs and set up required environment variables in your project, such as
LIVEKIT_API_KEY
,
LIVEKIT_API_SECRET
, and
LIVEKIT_URL
. 3. Integrate connection logic in your frontend component. Your Next.js client fetches the token from your backend, then uses it to connect to the LiveKit room using the LiveKit React SDK. Sample backend (API route):
Copy code
import { AccessToken } from 'livekit-server-sdk';
// ...
const at = new AccessToken(apiKey, apiSecret, { identity: username });
at.addGrant({ room, roomJoin: true, canPublish: true, canSubscribe: true });
return NextResponse.json({ token: await at.toJwt() });
Sample frontend usage:
Copy code
// fetch token, then connect
await room.connect(serverUrl, participantToken);
Key tips: • Keep your token endpoint uncached (
Cache-Control: no-store
). • Handle errors for missing env vars and connection failures. • Each agent/participant needs a unique
identity
when generating a token. • Use
@livekit/components-react
for built-in UI components if you need a room interface. For advanced needs (like AI agents or customized permissions), you can further tailor the token grants or configure agent dispatch in your room setup. Let me know if you want specific code examples for a particular scenario or a more detailed walkthrough based on your agent type or permissions. Sources: examples/webhooks-nextjs/README.md | Next.js quickstart | LiveKit Docs | complex-agents/shopify-voice-shopper/shopify-voice-frontend/app/api/connect... | src/pages/api/token.ts | client/web/src/app/api/connection-details/route.ts
c
can it be done without a backend?
t
Hey there! I totally get why you might want to add participants to a LiveKit room directly, but it's super important to do this safely. The key thing is that LiveKit tokens need your special API secret, and we definitely don't want that floating around where it could cause security issues. Think of it like keeping your house keys safe – you wouldn't leave them out for just anyone to grab, right? Similarly, your LiveKit API secret needs to stay protected. The best way to handle this is by creating those tokens on your server and then passing them securely to your client. If you're using Next.js or another frontend framework, you'll want to set up a backend endpoint to manage this process. Trust me, this approach isn't just a complicated extra step – it's a crucial way to keep your rooms secure and make sure only the right people can join. It's all about protecting your digital space!
c
gotcha, i am using python to achieve this. can you give me an flask endpoint which takes a room name in the POST request and spawns the agent in the room.
t
Hey there! Let me walk you through a super handy Flask endpoint for generating LiveKit tokens for agents. Think of this like creating a special pass that lets your agent join a video room smoothly. It's really straightforward – you'll just send over the room name, and this code does the heavy lifting of creating a secure token.
Copy code
from flask import Flask, request, jsonify
from livekit import api
import os
from datetime import datetime

app = Flask(__name__)
# Put your credentials in environment variables
app.config['LIVEKIT_API_KEY'] = os.getenv('LIVEKIT_API_KEY')
app.config['LIVEKIT_API_SECRET'] = os.getenv('LIVEKIT_API_SECRET')
app.config['LIVEKIT_HOST'] = os.getenv('LIVEKIT_HOST', '<wss://your-livekit-host.com>')

@app.route('/api/agent-token', methods=['POST'])
def generate_agent_token():
    data = request.json
    room_name = data.get('room_name')
    if not room_name:
        return jsonify({'error': 'Missing room_name'}), 400
    agent_identity = f"agent_{datetime.now().timestamp()}"
    
    token_builder = (
        api.AccessToken()
        .with_identity(agent_identity)
        .with_name(agent_identity)
        .with_grants(api.VideoGrants(
            room_join=True, room=room_name, can_publish=True, can_subscribe=True, can_publish_data=True
        ))
        .with_kind('agent')
    )
    
    token = token_builder.to_jwt()
    return jsonify({
        'token': token,
        'room_name': room_name,
        'agent_identity': agent_identity
    })

if __name__ == '__main__':
    app.run(debug=True, port=5000)
• Just send a quick POST request with some simple JSON:
{"room_name": "YOUR_ROOM_NAME"}
• Pro tip: If you need something more specialized, you can totally expand this to include agent dispatch with
RoomAgentDispatch
. This setup makes token generation a breeze and keeps everything super secure. Got something specific in mind or need a custom tweak? I'm all ears – just shoot me a message, and we'll figure out exactly what you need! Sources: examples/avatar_agents/audio_wave/agent_worker.py | Generating tokens | LiveKit Docs | ivr-nagivator/flask_livekit/extension.py | Agent dispatch | LiveKit Docs
c
give the agent a example instruction like "act as a story teller". also i have a pdf which needs to be sent at the very starting as an external data source. Along with that i am using open ai realtime model "gpt-4o-mini-realtime-preview" with azure
t
Hey there! Want to turn your agent into a storyteller? It's super easy - just add a custom instruction when you're creating your agent. Here's a quick example:
Copy code
agent = Agent(
    instructions="You are a storyteller. Spin engaging, interactive stories using the information the user provides, and make the experience fun and dynamic."
)
Integrating your PDF as an external data source: I totally get you're looking to add a PDF to your LiveKit agent. While there's no direct one-click method, don't worry! You can totally work around this by processing the PDF content in your own code and then passing it to the agent. Think of it like preparing a delicious story ingredients list that you'll share with your agent. OpenAI GPT-4o-mini-realtime-preview (Azure): Configuring the Azure OpenAI Realtime API is pretty straightforward. Check out this setup:
Copy code
from livekit.agents import AgentSession, Agent
from livekit.plugins import openai

agent = Agent(instructions="You are a storyteller...")
session = AgentSession(
    llm=openai.realtime.RealtimeModel.with_azure(
        azure_deployment="your-gpt-4o-mini-realtime-deployment",
        azure_endpoint="<wss://your-endpoint.openai.azure.com/>",
        api_key="your-api-key",
        api_version="2024-10-01-preview"
    ),
)
await session.start(agent=agent, room=ctx.room)
• Just swap out those deployment, endpoint, and API key values to match your Azure setup. • For the PDF, you'll want to preprocess the content and then seamlessly blend it into your agent's context. Need a hand walking through an end-to-end PDF parsing example? Just let me know what you're trying to do, and I'll be happy to help! Sources: LiveKit Docs | OpenAI and LiveKit | LiveKit Docs | README.md | Azure OpenAI Realtime API and LiveKit | LiveKit Docs
c
i have the pdf in bytes form and need to send it in agent context as data source. Give me the whole code including a flask endpoint which takes a room_name in it's POST request payload and spawns an agent with "Act as an story teller" instruction which uses openai realtime model with azure. Also get a pdf and include it in agent's context.
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!