Hello, I’m also experiencing some issues with

Agent.session.generate_reply()

in LiveKit AWS Nova 2 Sonic when using

livekit-agents==1.3.9

and

livekit-plugins-aws==1.3.9

. Is anyone else facing a similar issue? Please see my logs below:

2025-12-20 11:34:53,063 - livekit.agents - ERROR - Error in _realtime_reply_task
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs
    return await fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/livekit/agents/voice/agent_activity.py", line 2185, in _realtime_reply_task
    generation_ev = await self._rt_session.generate_reply(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
livekit.agents.llm.realtime.RealtimeError: generate_reply timed out waiting for generation

Hi, how do you reproduce this?

hello, to reproduce, after

session.start()

to initialize the agent session, just do

await session.generate_reply(instructions=instructions)

I can't reproduce in this code snippet based on our agent starter. Am I missing something?:

import logging

from dotenv import load_dotenv
from livekit import rtc
from livekit.agents import (
    Agent,
    AgentServer,
    AgentSession,
    JobContext,
    JobProcess,
    cli,
    inference,
    room_io,
)
from livekit.plugins import noise_cancellation, silero, aws
from livekit.plugins.turn_detector.multilingual import MultilingualModel

logger = logging.getLogger("agent")

load_dotenv(".env.local")


class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="""You are a helpful voice AI assistant. The user is interacting with you via voice, even if you perceive the conversation as text.
            You eagerly assist users with their questions by providing information from your extensive knowledge.
            Your responses are concise, to the point, and without any complex formatting or punctuation including emojis, asterisks, or other symbols.
            You are curious, friendly, and have a sense of humor.""",
        )

server = AgentServer()


def prewarm(proc: JobProcess):
    proc.userdata["vad"] = silero.VAD.load()


server.setup_fnc = prewarm


@server.rtc_session()
async def my_agent(ctx: JobContext):
    # Logging setup
    # Add any other context you want in all log entries here
    ctx.log_context_fields = {
        "room": ctx.room.name,
    }

    session = AgentSession(
        llm=aws.realtime.RealtimeModel.with_nova_sonic_2(voice="tiffany"),
    )

    # Start the session, which initializes the voice pipeline and warms up the models
    await session.start(
        agent=Assistant(),
        room=ctx.room,
        room_options=room_io.RoomOptions(
            audio_input=room_io.AudioInputOptions(
                noise_cancellation=lambda params: noise_cancellation.BVCTelephony()
                if params.participant.kind == rtc.ParticipantKind.PARTICIPANT_KIND_SIP
                else noise_cancellation.BVC(),
            ),
        ),
    )

    # Join the room and connect to the user
    await ctx.connect()

    await session.generate_reply(instructions="Greet the user and ask their name")

if __name__ == "__main__":
    cli.run_app(server)

Hi @lively-winter-88668, thanks for your agent starter sample code, I still have the same issue with the sample code, below my logs, would you please share your log or I might miss sth else as well:

% lk agent logs   
Using project [rendezvous]
Using agent [CA_LNfMeyrhuLcf]
Waiting for deployment to start...
{"message": "starting worker", "level": "INFO", "name": "livekit.agents", "version": "1.3.9", "rtc-version": "1.0.23", "timestamp": "2025-12-22T14:12:07.075324+00:00"}
{"message": "preloading plugins", "level": "INFO", "name": "livekit.agents", "packages": ["livekit.plugins.silero", "livekit.plugins.aws", "livekit.plugins.turn_detector", "av"], "timestamp": "2025-12-22T14:12:07.075455+00:00"}
{"message": "HTTP server listening on :8081", "level": "INFO", "name": "livekit.agents", "timestamp": "2025-12-22T14:12:07.093187+00:00"}
{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 80, "timestamp": "2025-12-22T14:12:08.428508+00:00"}
{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 83, "timestamp": "2025-12-22T14:12:08.443218+00:00"}
{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 89, "timestamp": "2025-12-22T14:12:08.455426+00:00"}
{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 98, "timestamp": "2025-12-22T14:12:08.466368+00:00"}
{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 83, "elapsed_time": 0.2, "timestamp": "2025-12-22T14:12:08.641509+00:00"}
{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 80, "elapsed_time": 0.22, "timestamp": "2025-12-22T14:12:08.651594+00:00"}
{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 98, "elapsed_time": 0.19, "timestamp": "2025-12-22T14:12:08.652729+00:00"}
{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 89, "elapsed_time": 0.2, "timestamp": "2025-12-22T14:12:08.659976+00:00"}
{"message": "registered worker", "level": "INFO", "name": "livekit.agents", "agent_name": "", "id": "CAW_YthE4V3cBojw", "url": "<https://rendezvous-hivz4ya2.livekit.cloud>", "region": "US East B", "protocol": 16, "timestamp": "2025-12-22T14:12:08.696367+00:00"}
{"message": "received job request", "level": "INFO", "name": "livekit.agents", "job_id": "AJ_rw4uvJZDpaHp", "dispatch_id": "AD_uDavXyv8XGZg", "room": "tech-interview-h2oxnwKk", "room_id": "RM_Y2ErVapNtWKZ", "agent_name": "", "resuming": false, "enable_recording": false, "timestamp": "2025-12-22T14:12:19.193139+00:00"}
{"message": "initializing process", "level": "INFO", "name": "livekit.agents", "pid": 202, "timestamp": "2025-12-22T14:12:19.271198+00:00"}
{"message": "Creating Bedrock client", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.297765+00:00"}
{"message": "Initializing Bedrock stream", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.310563+00:00"}
{"message": "process initialized", "level": "INFO", "name": "livekit.agents", "pid": 202, "elapsed_time": 0.18, "timestamp": "2025-12-22T14:12:19.453829+00:00"}
{"message": "generate_reply: sending instructions='Greet the user and ask their name'", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.565092+00:00"}
{"message": "Initializing Bedrock session with realtime options: _RealtimeOptions(voice='tiffany', temperature=0.7, top_p=0.9, max_tokens=1024, tool_choice=None, region='us-east-1', turn_detection='MEDIUM', modalities='mixed')", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.865271+00:00"}
{"message": "[SESSION] Static credentials, using max duration: 360s", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.874031+00:00"}
{"message": "Starting audio input processing loop", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.874631+00:00"}
{"message": "Sent text message (interactive=True): Greet the user and ask their name", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.875237+00:00"}
{"message": "[SESSION] Recycle timer started, will fire in 360s", "level": "INFO", "name": "livekit.plugins.aws", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:19.875314+00:00"}
{"message": "ignoring text stream with topic 'current_question', no callback attached", "level": "INFO", "name": "root", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:21.331309+00:00"}
{"message": "Error in _realtime_reply_task\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/livekit/agents/utils/log.py\", line 16, in async_fn_logs\n    return await fn(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/livekit/agents/voice/agent_activity.py\", line 2185, in _realtime_reply_task\n    generation_ev = await self._rt_session.generate_reply(\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nlivekit.agents.llm.realtime.RealtimeError: generate_reply timed out waiting for generation", "level": "ERROR", "name": "livekit.agents", "exc_info": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.12/site-packages/livekit/agents/utils/log.py\", line 16, in async_fn_logs\n    return await fn(*args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/livekit/agents/voice/agent_activity.py\", line 2185, in _realtime_reply_task\n    generation_ev = await self._rt_session.generate_reply(\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nlivekit.agents.llm.realtime.RealtimeError: generate_reply timed out waiting for generation", "room": "tech-interview-h2oxnwKk", "pid": 83, "job_id": "AJ_rw4uvJZDpaHp", "room_id": "RM_Y2ErVapNtWKZ", "timestamp": "2025-12-22T14:12:29.566327+00:00"}

It's very chatty, I'm not sure why it would fail when running for you. I'm just running with

uv run src/agent.py console

so copying my output in the attached file. I'm on agents 1.3.9 too

Based on my various tests today, the LiveKit/AWS realtime model is working pretty well. However,

generate_reply()

consistently has a timeout issue. It should be part of the multi-modal text I/O new function, so the Agent is supposed to speak first and introduce itself, but it's not working. Could LiveKit or AWS retrieve my call to

generate_reply()

using the room and job ID in my log to find out what's wrong with this

generate_reply()

call? Please let me know if anyone has any idea about this issue. By the way, I don't see key AWS Nova developer in this channel. Thanks

Could LiveKit or AWS retrieve my call to

generate_reply()

using the room and job ID

I checked in our server logs for AJ_rw4uvJZDpaHp and room RM_Y2ErVapNtWKZ and I don't see anything out of the ordinary, though I'm not surprised since there also isn't an issue shown in the session events on your dashboard.

Based on my various tests today, the LiveKit/AWS realtime model is working pretty well. However,

generate_reply()

consistently has a timeout issue.

Is there a specific setup I can try to reproduce this? Like I say, I was using the

console

but is there anything else I can do to match your setup? If we can isolate this and reproduce it consistently then we can start to address it.

I see the only difference seems that I have deployed agent.py sample into a livekit cloud server with

lk agent deploy/restart

then

lk agent logs

(with the libs in the requirements.txt), and I have an ios frontend app just to trigger it, or is there any firewall which stops the communication on

generate_reply()

between livekit cloud server and Amazon Bedrock? ...