how can i inject system instruction dynamically LiveKit Community #ask-ai

Join Slack

- how can i inject system instruction dynamically

# ask-ai

jolly-apartment-95291

07/21/2025, 7:13 PM

• how can i inject system instruction dynamically

tall-airline-16756

07/21/2025, 7:13 PM

Thanks for reaching out, Tyler. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

tall-airline-16756

07/21/2025, 7:14 PM

You can inject system instructions dynamically in LiveKit using several methods: • Agent's
update_instructions
Method: Call

update_instructions

on your agent instance to update instructions at runtime. This works for both regular and realtime agents, and updates the instructions for ongoing sessions.

Copy code

async def update_instructions(self, instructions: str) -> None:
    """
    Updates the agent's instructions.
    If the agent is running in realtime mode, this method also updates
    the instructions for the ongoing realtime session.
    """

• Dynamic Instructions in RAG Context: For retrieval-augmented generation, you can append context-specific instructions to the system message before generating a reply.

Copy code

# Example: Injecting context into system instructions
system_msg = chat_ctx.items[0]
if isinstance(system_msg, llm.ChatMessage) and system_msg.role == "system":
    system_msg.content.append(instructions)
else:
    chat_ctx.items.insert(0, llm.ChatMessage(role="system", content=[instructions]))

• Chat Context Manipulation: Use utility functions to update or insert system instruction messages directly in the chat context.

Copy code

def update_instructions(chat_ctx: ChatContext, instructions: str, add_if_missing: bool = True) -> None:
    # Updates or inserts the system instruction message

The

update_instructions

method is the recommended approach for most use cases, as it automatically handles both regular and realtime sessions. Sources: livekit-agents/livekit/agents/voice/agent.py | basics/change_agent_instructions.py

jolly-apartment-95291

07/22/2025, 12:18 PM

how to use preemptive_generation?

tall-airline-16756

07/22/2025, 12:19 PM

preemptive_generation is like having a super-smart assistant who's always one step ahead! Imagine your AI starting to draft a response the moment it hears the last word of your message - how cool is that? This neat feature helps make conversations feel smoother and more natural, almost like you're chatting with a friend who's really quick on the uptake. How to enable and use preemptive_generation: • Set

preemptive_generation=True

when creating your `AgentSession`:

Copy code

session = AgentSession(
    preemptive_generation=True,
    # ... other parameters (STT, LLM, TTS, etc.)
)

• When this feature is on, your agent jumps into action as soon as the final transcript is ready. Don't worry - if you make any updates to the chat context or tools, it'll quickly adjust and create a brand new response just for you. Example usage:

Copy code

class MyAgent(Agent):
    async def on_enter(self):
        self.session.generate_reply(instructions="say hello to the user")

    async def on_user_turn_completed(self, turn_ctx: ChatContext, new_message: ChatMessage):
        # If you update context/tools here, preemptive generation is cancelled
        pass

async def entrypoint(ctx: JobContext):
    session = AgentSession(
        preemptive_generation=True,
        # ... other parameters
    )
    await session.start(agent=MyAgent(), room=ctx.room)

• Pro tip: This works best when your speech-to-text is lightning-fast and can wrap up before the end-of-speech signal. • It's specifically designed for agents that use speech-to-text, language models, and text-to-speech. • Want to make sure it's working great? Keep an eye on those logs and metrics! Sources: Agent speech and audio | LiveKit Docs | livekit-plugins/livekit-plugins-google/livekit/plugins/google/beta/realtime...

Open in Slack

Previous Next