How can I create AI agent that goes smoothly thro...
# ask-ai
e
How can I create AI agent that goes smoothly throw three stages: smalltalk, quizz, storytellting
t
Thanks for reaching out, Tetiana. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
I'm excited to guide you through creating an AI agent that smoothly moves through smalltalk, quiz, and storytelling stages in LiveKit. Here's a friendly step-by-step approach to help you along: 1. Define Agent Stages and Flow: Think of each stage—smalltalk, quiz, storytelling—as its own agent class. You can use function tools to gently manage the transitions between these stages, all while keeping the user's context in mind. 2. Maintain Shared User Context: It’s helpful to have a shared UserData object where you can keep track of user details, quiz results, and story elements throughout the journey. 3. Smalltalk Stage: Begin with some casual conversation to get to know the user better. Based on their input, you can naturally move on to the next stage. 4. Quiz Stage: Use function tools to craft and present quizzes. Implement RPCs to send quiz data to the frontend and smoothly handle user responses. 5. Storytelling Stage: Personalize the story using all the wonderful data gathered earlier. Keep the tone conversational and wrap up the story when a specific tool signals it’s time. 6. Frontend Integration: Don’t forget to register RPC methods that will manage quiz and storytelling interactions. Components like QuizContainer are great for creating interactive elements. 7. Deployment: Finally, run your agent using the CLI and set up the necessary environment variables (like LIVEKIT_URL and API keys) to get everything up and running. If you’d like more detailed examples and code snippets, I warmly recommend checking out the multi-agent flow and Tavus Avatar examples in the LiveKit documentation. They’re fantastic resources to support you along the way! Sources: README.md | LiveKit Docs | examples/src/gemini_realtime_agent.ts | avatars/tavus/voice-assistant-frontend/components/QuizContainer.tsx
e
Give me example of code
t
I'd love to share an example of a multi-stage AI agent in LiveKit that gently guides you through smalltalk, a fun quiz, and then storytelling:
Copy code
import logging
from dataclasses import dataclass, field
from typing import Dict, List
from livekit.agents import JobContext, WorkerOptions, cli
from livekit.agents.llm import function_tool
from livekit.agents.voice import Agent, AgentSession

@dataclass
class AgentData:
    user_name: str = ""
    responses: Dict[str, str] = field(default_factory=dict)

class SmallTalkAgent(Agent):
    def __init__(self, job_context: JobContext):
        super().__init__(instructions="Start with smalltalk and gather user info.")

    async def on_enter(self):
        await self.session.say("Hello! What's your name?")

    @function_tool
    async def gather_user_info(self, name: str):
        self.session.userdata.user_name = name
        return QuizAgent(self.session.job_context)

class QuizAgent(Agent):
    def __init__(self, job_context: JobContext):
        super().__init__(instructions="Create and display a quiz.")

    async def on_enter(self):
        await self.session.say("Let's start a quiz!")

    @function_tool
    async def quiz_completed(self):
        return StoryAgent(self.session.job_context, self.session.userdata.user_name)

class StoryAgent(Agent):
    def __init__(self, job_context: JobContext, user_name: str):
        super().__init__(instructions=f"Tell a story for {user_name}.")

    async def on_enter(self):
        await self.session.say(f"Once upon a time, {user_name}...")

async def entrypoint(ctx: JobContext):
    session = AgentSession()
    session.userdata = AgentData()
    await session.start(agent=SmallTalkAgent(ctx))

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
This example warmly walks you through: • Smalltalk Stage: A friendly way to gather some info about you. • Quiz Stage: Engages you with a quiz and smoothly moves to the next part. • Storytelling Stage: Crafts a personalized story just for you using the info you shared. To make everything work beautifully, you'll want to integrate RPCs for handling the quiz and set up the frontend to show quizzes and stories. If you need any help with that, I'm here for you! Sources: avatars/hedra/education_avatar/education-frontend/components/quiz-container... | examples/voice_agents/multi_agent.py
e
The agent doesn't speak at all
t
I'm sorry to hear your AI agent isn't speaking. Let's work through some steps together to get this sorted out: 1. Verify TTS Configuration: First, please check that a Text-to-Speech (TTS) model is properly set up and initialized. The agent needs this to be able to generate speech. 2. Enable Audio Output: Make sure audio output is turned on in your session or RoomIO settings. For instance, confirm that
output.audio_enabled
is set to
True
. 3. Check Logs: Take a look at the logs to see if the agent is entering the "speaking" state. If it is, but you still don’t hear audio, the issue might be related to network or playback settings. 4. Realtime Model Settings: If you’re using a RealtimeModel, double-check that
allow_interruptions
is enabled and that turn detection is configured correctly. 5. Network Issues: Lastly, if you’re using telephony integrations, please verify your SIP trunk credentials and ensure there aren’t any connectivity problems. If you’d like, let me know which of these you’ve already looked into, and I’ll be happy to guide you through the next steps. We’ll get this working together! Sources: SIP troubleshooting guide | LiveKit Docs | How to Enable Agent-to-Agent Speech Detection in LiveKit | Events and error handling | LiveKit Docs | Knowledge Management System | livekit/livekit_composite | DeepWiki | livekit-agents/livekit/agents/voice/agent_activity.py
e
Improve please code to speak with sgent
t
To help your AI agent speak more naturally, you can tweak the text output just before it goes to the Text-to-Speech (TTS) engine. Here are two friendly ways to do this: 1. Using a Callback Function: You can set up a
before_tts_cb
callback to gently clean up or adjust the text before it’s spoken. Here’s a simple example: Sources: chain-of-thought-tts/README.mdoc | Modifying LLM output before TTS | LiveKit Docs