https://livekit.io logo
Join Slack
Powered by
This message was deleted.
# ask-ai
s
This message was deleted.
t
Thanks for reaching out, Taras. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
To gently wait for a user to start speaking and send a message if no input is detected, here are some friendly approaches you can try in LiveKit: • Use the 
user_away_timeout
Feature: This is a great way to detect when a user hasn’t spoken for a while. You can set the 
user_away_timeout
and listen for the 
user_state_changed
event when the state changes to 
away
. Then, you can respond with a kind message. For example:
Copy code
@agent.on("user_state_changed")
async def on_user_state_changed(old_state, new_state):
    if new_state == "away":
        await agent.say("Are you still there? Let me know if you need help.")
• Customize the Timeout: You can adjust how long the system waits before marking the user as away by setting the timeout when you create the `AgentSession`:
Copy code
session = AgentSession(
    user_away_timeout=10  # Time in seconds before user is marked as away
)
• Implement a Custom Inactivity Handler: If you’d like even more control, you can create a custom handler that watches for inactivity and sends a friendly message after a set period of silence. Here’s an example:
Copy code
handler = InactivityHandler(
    timeout_seconds=10,
    inactivity_message="Are you still there? Let me know if you need any help."
)
handler.start(agent)
These options help you gently check in with your users after some quiet time, showing them you’re here and ready to assist. If you have any questions or want to explore this further, just let me know—I’m happy to help! Sources: Events and error handling | LiveKit Docs | agent_extensions/agent_extensions/handlers/inactivity_handler.py | Events and error handling | LiveKit Docs
a
Can you provide InactivityHandler class example for the agent version 1.0 where we use AgentSession instead of voicepipelineagent?
t
I'd love to walk you through an example of implementing an 
InactivityHandler
for LiveKit Agents version 1.0 using `AgentSession`:
Copy code
import asyncio
import logging

from livekit.agents import AgentSession, JobContext, UserStateChangedEvent, cli
from livekit.plugins import silero, deepgram, openai, cartesia

logger = logging.getLogger("inactivity-handler-example")

async def entrypoint(ctx: JobContext):
    # Create an AgentSession with a custom away timeout (e.g., 12.5 seconds)
    session = AgentSession(
        vad=silero.VAD.load(),
        stt=deepgram.STT(),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=cartesia.TTS(),
        user_away_timeout=12.5,  # Time before user is considered "away"
    )

    inactivity_task: asyncio.Task | None = None

    async def user_presence_check():
        # Attempt to re-engage the user up to 3 times
        for _ in range(3):
            await session.generate_reply(
                instructions="The user has been inactive. Politely check if they are still present."
            )
            await asyncio.sleep(10)  # Wait 10 seconds between prompts

        # If no response, shut down the session
        <http://logger.info|logger.info>("User did not respond after 3 attempts. Shutting down session.")
        session.shutdown()

    @session.on("user_state_changed")
    def on_user_state_change(ev: UserStateChangedEvent):
        nonlocal inactivity_task
        if ev.new_state == "away" and inactivity_task is None:
            # User has become inactive; start the presence check
            inactivity_task = asyncio.create_task(user_presence_check())
        elif ev.new_state in ("speaking", "listening") and inactivity_task:
            # User is active again; cancel the inactivity check
            inactivity_task.cancel()
            inactivity_task = None

    # Start the session with a basic agent
    await session.start(agent=Agent(instructions="You are a helpful assistant."), room=ctx.room)

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
Key Features: • `user_away_timeout`: This lets you set a timeout (in seconds) after which the user is considered "away" if no speech is detected—giving you control over the inactivity period. • 
user_state_changed
Event: This handy event triggers whenever the user’s state changes (like from "listening" to "away"), so you can respond accordingly. • Graceful Shutdown: After a few polite attempts to check in with the user without any response, the session will close smoothly using 
session.shutdown()
. This method follows the best practices for LiveKit Agents 1.0 and replaces the older 
VoicePipelineAgent
-based approach. If anything isn’t clear or you want to dive deeper, just let me know—I’m here to help! Sources: livekit-agents/livekit/agents/voice/agent_activity.py | Events and error handling | LiveKit Docs | Docs::Agents
a
how do I add instruction message to current chat_context?
t
Thank you for reaching out! Could you please help me understand a bit more about what you mean by adding an instruction message to the current chat context? Are you hoping to adjust how the agent behaves on the fly, or would you like to send a particular message directly to the user during the session? I'm here to help you with whatever you need!
a
adding role "system" with content to chat_context of session.history
t
I'd love to help you add a system role with content to the 
chat_context
or 
session.history
in LiveKit AgentSession! Here are some friendly ways you can do this: • Initialize with an Initial Context: You can start by creating a 
ChatContext
with a system message before kicking off the session:
Copy code
from livekit.agents import ChatContext

initial_ctx = ChatContext()
initial_ctx.add_message(role="system", content="You are a helpful AI assistant.")
• Update Within a Callback: If you want to add messages dynamically, you can modify the 
chat_context
during callbacks like `onUserTurnCompleted`:
Copy code
async def add_rag_context(assistant, chat_ctx):
    chat_ctx.add_message(role="system", content="Your system content here.")
• Override the 
llm_node
Method: Another approach is to directly update the 
chat_context
just before the LLM inference happens:
Copy code
async def llm_node(self, chat_ctx, tools, model_settings):
    chat_ctx.add_message(role="system", content="Your system content here.")
• Persist Changes: To make sure your system message sticks around in the permanent history, don’t forget to use the 
update_chat_ctx
method on your agent instance after you’ve made changes to the context. If anything feels unclear or if you’d like a hand implementing this, just let me know—I’m here to help! Sources: Docs::Agents | Docs::Agents | agents/src/llm/chat_context.ts | Docs::Agents | Pipeline nodes and hooks | LiveKit Docs
a
This is creating new chatcontext.. but I want existing chat context of an ongoing AgentSession
t
To help you modify the existing chat context of an ongoing 
AgentSession
, you can use the 
update_chat_ctx()
method to make sure your changes are saved. Here’s a friendly guide on how to do it: • Modify the Context Object: You can add new messages or tweak existing ones using 
add_message()
or by adjusting the 
new_message
object.
Copy code
async def on_user_turn_completed(self, turn_ctx: ChatContext, new_message: ChatMessage):
    # Modify user message
    new_message.content = "Updated user message"

    # Add a system message
    turn_ctx.add_message(role="system", content="System instructions here")

    # Persist changes
    await self.update_chat_ctx(turn_ctx)
• Persist Changes: Remember to always call 
update_chat_ctx()
after making your changes so they stick around. Just a heads-up: if you modify the chat context directly without calling 
update_chat_ctx()
, those changes won’t last beyond the current turn. If anything’s unclear or you’d like a hand with this, I’m here to help! Sources: agents/src/llm/chat_context.ts | Docs::Agents | Docs::Agents
a
"on_user_turn_completed" is this function would automatically be executed OR do we have to assign it to any event of room OR a session may be.
t
I'm happy to explain how the 
on_user_turn_completed
function works in LiveKit AgentSession! It automatically runs when the user's turn ends, thanks to the agent's turn detection system—like when speech-to-text (STT) picks up the end of what someone says. For most standard STT-LLM-TTS setups, you don’t need to do anything extra; the function just runs on its own. But if you’re working with realtime models or push-to-talk situations, there are a couple of things to keep in mind: • Realtime Models: Make sure the turn detection is set up in the agent, not the model itself. • Push-to-Talk: You’ll want to set 
turn_detection="manual"
in your 
AgentSession
and then call 
commit_user_turn()
yourself to trigger the function. If you’d like, I’m here to help with more details or examples tailored to your specific setup—just let me know! Sources: Pipeline nodes and hooks | LiveKit Docs | Pipeline nodes and hooks | LiveKit Docs | Docs::Agents | examples/voice_agents/push_to_talk.py
a
AttributeError: 'JobContext' object has no attribute 'chat_ctx' I'm getting above error when I'm using this command ctx.chat_ctx.add_message(role="system", content="IMPORTANT INSTRUCTION:")
t
It looks like you're encountering the error 
AttributeError: 'JobContext' object has no attribute 'chat_ctx'
because the 
JobContext
class doesn’t actually include a 
chat_ctx
attribute. The 
JobContext
is mainly designed to handle entrypoint-level lifecycle events, like connecting to a room or managing worker processes, but it doesn’t manage the conversation state itself. To help you move forward smoothly, here’s what you can do to manage the chat context properly: • Initialize Chat Context: When creating an 
Agent
or 
AgentSession
, make sure to use 
ChatContext
like this:
Copy code
chat_ctx = ChatContext()
chat_ctx.add_message(role="system", content="IMPORTANT INSTRUCTION:")
agent = Agent(chat_ctx=chat_ctx, instructions="You are a helpful assistant.")
• Access Chat Context During Execution: Use 
session.chat_ctx
or 
session.history
to get the current session’s conversation history. • Pass Chat Context Between Agents: When handing off between agents, pass along the 
chat_ctx
to keep the conversation seamless:
Copy code
return TechnicalSupportAgent(chat_ctx=self.session.chat_ctx)
If anything here feels unclear or if you’d like a hand with refactoring your code, please don’t hesitate to reach out. I’m here to help! Sources: examples/voice_agents/realtime_joke_teller.py | complex-agents/shopify-voice-shopper/shopify.py | agent.py | Docs::Agents | Workflows | LiveKit Docs
a
I want to add system message in existing session chat context in the event of "conversation_item_added" when user's message OR agent message gets added.. I want to add system prompt after this
t
Thanks for reaching out! Just to better assist you, could you let me know if you're using a specific event handler or callback to listen for the 
conversation_item_added
event? Also, are you hoping to update the chat context dynamically during the session, or would you like to keep the system message consistent for future turns? I'm here to help you through this!
a
Yes.. I want to update the chat context dynamically during the session.. want to add message
t
To help you dynamically update the chat context during a session when a conversation item is added, you can use the 
update_chat_ctx
method. Let me walk you through how you can do this: 1. Retrieve a Copy of the Current Chat Context: Start by using the 
copy()
method to get a version of the chat context that you can modify.
Copy code
chat_ctx = session.chat_ctx.copy()
chat_ctx.add_message(role="system", content="IMPORTANT INSTRUCTION:")
1. Update the Session: Then, use the 
update_chat_ctx
method to sync your modified context with the server.
Copy code
await session.update_chat_ctx(chat_ctx)
This way, your new system message gets added to the ongoing session and stays in sync with the server. If you’d like, I’m here to help with any further questions or examples you might need! Sources: agents/src/llm/remote_chat_context.ts | livekit-agents/livekit/agents/llm/remote_chat_context.py
a
getting this error "AttributeError: 'AgentSession' object has no attribute 'chat_ctx' " in the line of session.chat_ctx.copy
t
It looks like you're encountering the error 
AttributeError: 'AgentSession' object has no attribute 'chat_ctx'
. This happens because the 
AgentSession
class doesn’t have a 
chat_ctx
attribute directly accessible. Instead, a better way is to use the 
session.history
property to get the conversation history. To fix this, simply replace:
Copy code
chat_ctx = session.chat_ctx.copy()
with:
Copy code
chat_ctx = session.history.copy()
After that, you can update your copied context and sync it back using 
update_chat_ctx
like this:
Copy code
chat_ctx.add_message(role="system", content="IMPORTANT INSTRUCTION:")
await session.update_chat_ctx(chat_ctx)
This way, you’ll have proper access to the conversation history and keep everything in sync with your session. If you have any questions or want me to walk you through it step-by-step, just let me know—I’m here to help! Sources: agents/src/voice/agent_session.ts | livekit-plugins/livekit-plugins-ultravox/livekit/plugins/ultravox/realtime/... | Workflows | LiveKit Docs | External data and RAG | LiveKit Docs
Open in Slack
PreviousNext
Powered by