Hey, I dont know if I am doing something wrong, bu...
# ask-aim
miniature-barista-33555
12/27/2025, 1:10 AMHey, I dont know if I am doing something wrong, but I would like to ask about this anyways. I am facing an issue with the way generate_reply() works with adding custom messages to the agent chat context
take this example code:
import logging
import os
from dotenv import load_dotenv
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, JobProcess, cli, llm
from livekit.plugins import cartesia, deepgram, openai, silero
from livekit.plugins.turn_detector.english import EnglishModel
logger = logging.getLogger("agent-instructions-debug")
load_dotenv()
DEFAULT_INSTRUCTIONS = (
"You are a helpful voice assistant. Keep responses short and natural."
)
class DebugAgent(Agent):
async def on_enter(self) -> None:
logger.info("on_enter: generating reply with explicit instructions")
await self.session.generate_reply(
instructions="Say you like cats.",
allow_interruptions=True,
)
def prewarm(proc: JobProcess) -> None:
proc.userdata["vad"] = silero.VAD.load()
server = AgentServer()
server.setup_fnc = prewarm
@breezy-insurance-66772.rtc_session()
async def entrypoint(ctx: JobContext) -> None:
await ctx.connect()
chat_history = [
{"role": "assistant", "content": "Hi there. How can I help?"},
{"role": "user", "content": "Can you help me plan a short trip?"},
{"role": "assistant", "content": "Sure. Where are you thinking of going?"},
{"role": "user", "content": "Maybe Chicago."},
]
chat_ctx = llm.ChatContext.empty()
for item in chat_history:
chat_ctx.add_message(role=item["role"], content=item["content"])
session = AgentSession(
stt=deepgram.STT(model="nova-3", language="en", filler_words=True),
llm=openai.LLM(model="gpt-4o-mini"),
tts=cartesia.TTS(model="sonic-2"),
turn_detection=EnglishModel(),
vad=ctx.proc.userdata["vad"],
)
agent = DebugAgent(instructions=DEFAULT_INSTRUCTIONS, chat_ctx=chat_ctx)
logger.info("Starting debug agent session")
await session.start(agent=agent, room=ctx.room)
if name == "__main__":
logging.basicConfig(level=os.getenv("LOG_LEVEL", "INFO"))
cli.run_app(server)
I add some basic messages to the agents chat context, and add an on_enter() function where, on joining, the agent will generate a reply, with instructions="say you like cats". I am explicitly telling the agent to talk about liking cats. What instead happens is that the agent replies to the last message, completely ignoring the instructions
What I then noticed is if I stopped adding my own custom messages to the chat_ctx, the agent would respond properly, saying that they like cats
What we also noticed is that this breaks every single generate_reply() going forward. For example in our codebase we have a tool that allows the agent to search the web. In the tool itself we have a generate_reply() call with instructions="tell the user you are currently searching the web", what happens is that the agent just generates a response to the last message, ignoring the instructions, which is odd.
In our codebase we want to be able to inject past conversations from our db directly into the agent, and use the generate_reply() to generate a reply that follows our set instructions. Currently we arent able to. For now we added a workaround where we just dump the past conversations in the instructions instead, but I feel thats just working around it. But maybe I am doing something wrong. Any tips would be appreciated.
t
tall-airline-16756
12/27/2025, 1:10 AMThanks for reaching out, Srikar. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
12/27/2025, 1:11 AMThanks — this is a clear report and I hear your frustration. You're asking why generate_reply(instructions=...) is ignored when you preload chat messages, and how to inject past conversations while still controlling replies. I'll address that and give the simplest practical options.
Short answer
• Preloaded chat messages are treated as the authoritative conversation state. When chat history exists, the model prioritizes responding to the most recent turn, so a plain
instructions="Say you like cats."instructionsinstructions Copy code
await self.session.generate_reply(instructions=f"""
Past conversation: {serialized_history}
Now, respond with: Say you like cats.
""")user_inputinstructionsgenerate_reply()generate_reply()user_input