https://livekit.io logo
Join Slack
Powered by
how can we make the agent in realtime openai livek...
# ask-ai
s
how can we make the agent in realtime openai livekit to say something (like sending a message through channel so he says a specific sentence at the start of convo) without changing the instructions, just a trigger
t
Thanks for reaching out, MBMBMB. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To make the agent in LiveKit say a specific sentence at the start of a conversation without changing the instructions, you can use the 
session.say()
method within the 
on_enter
lifecycle method of your agent. This triggers the agent to speak a predefined message immediately when the conversation begins. Here’s an example implementation:
Copy code
class MyAgent(Agent):
    async def on_enter(self):
        await self.session.say("Hello! How can I assist you today?")
Ensure that your agent is configured with a TTS plugin, as 
session.say()
requires TTS to function properly.
s
but my agent is with realtime speech to speech
t
Hey there! If you’re looking to set up a real-time speech-to-speech agent in LiveKit, you can use the 
on_enter
method to have your agent say a friendly greeting right when the conversation starts. Here’s a simple way to do it:
Copy code
from livekit.agents import Agent, AgentSession
from livekit.plugins import openai

class GreetingAgent(Agent):
    async def on_enter(self):
        await self.session.say(
            "Hello! How can I assist you today?",
            allow_interruptions=True
        )

async def entrypoint(ctx: JobContext):
    session = AgentSession(
        tts=openai.TTS()  # Make sure you have a TTS plugin set up
    )
    
    await session.start(
        agent=GreetingAgent(),
        room=ctx.room
    )

if __name__ == "__main__":
    # Just configure and run your LiveKit worker here
    pass
Just a heads-up: your TTS plugin needs to support real-time streaming to make the speech-to-speech flow smooth. Also, the 
allow_interruptions
option is there so users can jump in and interrupt the agent if needed. Hope this helps!
s
im using openai realtime livekit agent
t
If you're working with an OpenAI real-time LiveKit agent and want it to say something right at the start of a conversation, you can do that easily in a couple of ways: 1. `session.say()`: 
await session.say("Hello! How can I help you today?")
2. `agent.say()`: 
await agent.say("Hey, how can I help you today", allow_interruptions=True)
Just a quick heads-up: make sure you have a TTS plugin set up if you're using a real-time model, since it's needed to turn text into speech. Sources: Agent speech and audio | LiveKit Docs | delay-handling/agent.py
s
ok give more detail on this
t
Let me walk you through how to get an OpenAI real-time LiveKit agent to say a specific sentence right at the start of a conversation. It’s pretty straightforward, and I’m here to help you every step of the way! • Prerequisites: ◦ Make sure you have LiveKit and the OpenAI plugins installed. ◦ Set up the OpenAI TTS plugin so your agent can convert text to speech. ◦ Configure your agent with the right settings. • Step-by-Step Implementation: a. Import Necessary Modules:
Copy code
from livekit.agents import Agent, AgentSession
from livekit.plugins import openai
from dotenv import load_dotenv
import asyncio

load_dotenv()  # Load environment variables
b. Configure the TTS Plugin: 
tts = openai.TTS(model="tts-1", voice="nova")
c. Create the Agent Class:
Copy code
class GreetingAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions="You are a helpful assistant.",
            tts=tts
        )
    
    async def on_enter(self):
        try:
            await self.session.say(
                "Welcome! How can I assist you today?",
                allow_interruptions=False
            )
        except Exception as e:
            print(f"Error during greeting: {e}")
d. Initialize and Start the Agent Session:
Copy code
async def entrypoint(ctx: JobContext):
    session = AgentSession()
    await session.start(
        agent=GreetingAgent(),
        room=ctx.room
    )

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
• Key Considerations: ◦ TTS Configuration: Double-check that your TTS plugin is set up correctly because 
session.say()
depends on it. ◦ Error Handling: Wrapping the greeting in try-except blocks helps catch any hiccups gracefully. ◦ Real-time Models: Make sure your TTS is ready for real-time speech synthesis to keep things smooth. With this setup, your agent will warmly greet users with a friendly message as soon as the conversation starts — making the whole experience feel a bit more personal and welcoming.
s
but i already have an agent session though
t
If you already have an agent session set up, you can easily get your agent to say a specific sentence right at the start of a conversation using one of these friendly methods: 1. Using `session.say()`: `await session.say( "Hello! Welcome to our conversation. How can I assist you today?", allow_interruptions=False )`Just a heads-up, this method needs a TTS plugin configured in your session. For example, you might use ElevenLabs or Silero to handle text-to-speech conversion.
Copy code
from livekit.plugins import elevenlabs
from livekit.agents import AgentSession

eleven_tts = elevenlabs.TTS(
    model="eleven_turbo_v2_5",
    voice=elevenlabs.Voice(
        id="EXAVITQu4vr4xnSDxMaL",
        name="Bella",
    ),
    language="en",
    enable_ssml_parsing=False,
)

session = AgentSession(tts=eleven_tts)
2. Using `generate_reply()`: `session.generate_reply( instructions="speak: Hello! Welcome to our conversation. How can I assist you today?" )`The great thing about this one is it works even if you don’t have a TTS plugin — it uses the agent’s language model to handle the instruction. 3. Pre-configuring the Agent: You can also set up initial instructions so the agent automatically greets users like this:
Copy code
model = openai.realtime.RealtimeModel(
    instructions="You are a friendly assistant. Greet the user by saying: 'Hello! Welcome to our conversation. How can I assist you today?'"
)

agent = multimodal.MultimodalAgent(model=model)
session = agent.start(room, participant)
Feel free to pick whichever method feels right for your setup. And if you have any questions or need a hand, just let me know — I’m here to help!
s
ok and how do we clear the cache cache history of livekit openai realtime live conversation so it starts brand new
t
Hey there! If you want to clear the cache or conversation history of a LiveKit OpenAI real-time live conversation and start fresh, here’s a simple way to do it: 1. Delete Conversation Items: First, use the 
conversation.item.delete
method to remove specific items from the conversation history: 
this.conversation.item.delete(itemId);
2. Create a New Empty Message: Once you’ve deleted the items, create a new empty user audio message to reset the conversation: 
this.conversation.item.create(this.#createEmptyUserAudioMessage(1));
3. Clear Audio Buffer: If there’s any buffered audio you want to clear, just use: 
this.inputAudioBuffer.clear();
Quick heads-up: • By default, every new agent session starts with a fresh conversation history. But if you’d like to keep the context, you can pass the 
chat_ctx
parameter in the agent constructor. • Clearing the history makes sure the model starts with a clean slate, so there’s no leftover context from before. If you have any questions or want a hand with setting this up, just let me know—I’m here to help! Sources: OpenAI Realtime API integration guide | LiveKit Docs | Workflows | LiveKit Docs | plugins/openai/src/realtime/api_proto.ts | livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/real...
s
not really i want to clear the cache of a live conversation
t
Hey there! If you want to clear the cache of a live conversation in LiveKit OpenAI real-time agent, here’s a simple way to handle both the audio buffer and the conversation history: 1. Clear Audio Buffer: You can use the 
InputAudioBufferClearEvent
to clear out the audio buffer like this:
Copy code
const clearBufferEvent = {
    type: 'input_audio_buffer.clear'
};
session.queueMsg(clearBufferEvent);
2. Clear Conversation History: To remove specific items from the conversation history, just use the `ConversationItemDeleteEvent`: `const deleteEvent = { type: 'conversation.item.delete', item_id: 'ITEM_ID_HERE' }; session.queueMsg(deleteEvent);`Make sure to swap out 
ITEM_ID_HERE
with the actual ID of the conversation item you want to delete. You’ll want to repeat this for each item to clear everything. Here’s a complete example to make it clearer:
Copy code
// Clear audio buffer
const clearBufferEvent = {
    type: 'input_audio_buffer.clear'
};
session.queueMsg(clearBufferEvent);

// Clear conversation history
const deleteEvent = {
    type: 'conversation.item.delete',
    item_id: 'ITEM_ID_HERE'
};
session.queueMsg(deleteEvent);
If you have any questions or want a hand with implementing this, just let me know—I’m here to help!
s
so i have to select an ID ?
t
Hey there! If you want to clear the entire conversation history in LiveKit OpenAI real-time agent without picking out individual item IDs, you can use a handy function called 
clearChats
. It wipes out all chat records for a user in one go. Here’s the scoop on how it works: 1. Retrieve All Chat IDs: The function grabs all the chat IDs linked to the user. 2. Delete All Chats: Then, it goes through those chat IDs and deletes them all in one smooth operation. 3. Update the User Interface: Finally, it makes sure the UI updates to show the cleared history. If you’re setting this up, you can call the 
clearChats
function either through code or a UI button like 
ClearHistory
. Just give me a shout if you want a hand getting it going! Sources: client/components/sidebar-list.tsx
s
more details
t
Let me walk you through how to use the 
clearChats
function to clear your entire conversation history in the LiveKit OpenAI real-time agent—it’s pretty straightforward! 1. Access the Clear History Component: The 
clearChats
function is part of a 
ClearHistory
React component. This component includes a handy button that lets you clear your chat history whenever you want. 2. Trigger the Clear Action: When you click the "Clear history" button, you’ll see a confirmation dialog pop up. If you confirm, the 
clearChats
function kicks in. 3. Execute the 
clearChats
Function: Here’s what happens next: ◦ It first checks if you’re logged in. ◦ Then, it grabs all your chat records. ◦ Deletes those chats from the database using a transaction pipeline. ◦ Finally, it takes you back to the home page once everything’s cleared. 1. Update the UI: Once your history is cleared, the UI refreshes to show that your chat history is gone, along with a message letting you know there’s nothing saved. This feature is all about giving you control and peace of mind by letting you delete your conversation history whenever you need. If you want a hand setting this up, just let me know—I’m here to help!
s
? why are you talking of UI im using python livekit openai realtime
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
Open in Slack
PreviousNext
Powered by