We are using turn_detection="manual" and calling commit_user_turn() to end the user turn. Our understanding is that commit_user_turn() initiates the work required to finalize the transcript, but does not wait for transcription to fully complete before continuing. Because of that, llm_node can be invoked before the final transcript is available, and it may receive an incomplete transcription. Is there an API or recommended pattern for waiting until the final transcript is ready before llm_node is executed?

Thanks for reaching out, Ryoya. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

Please tell me the way to call llm_node only if the

is_final === true

. Sample codebases will help us to solve this problem.

Thanks — I’m responding to your request for sample code that only calls the LLM when the transcript is final (

is_final === true

). Below are two concise examples (Python and JavaScript) that follow the recommended pattern: listen for the

user_input_transcribed

event, check

is_final

, then call

session.generate_reply()

(which triggers LLM processing). Move any LLM logic out of an

llm_node

override and into the final-transcript handler so the LLM always gets the completed input. Python example

from livekit.agents import AgentServer, AgentSession, JobContext
from livekit.plugins import openai, deepgram, silero

server = AgentServer()

def prewarm(proc):
    proc.userdata["vad"] = silero.VAD.load()

server.setup_fnc = prewarm

@server.rtc_session(agent_name="my-agent")
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        stt=deepgram.STT(),
        llm=openai.LLM(),
        tts=openai.TTS(),
        vad=ctx.proc.userdata["vad"],
    )

    @session.on("user_input_transcribed")
    def on_transcript(transcript):
        # Only trigger LLM when transcription is final
        if transcript.is_final:
            # Use the final transcript to generate a reply (this runs the llm_node)
            session.generate_reply(user_input=transcript.transcript)

    await session.start(agent=None, room=ctx.room)
    await ctx.connect()

JavaScript/TypeScript example

import { AgentServer, AgentSession } from "livekit-agents";
import { DeepgramSTT } from "livekit-plugins-deepgram";
import { OpenAI } from "livekit-plugins-openai";

const server = new AgentServer();

server.rtcSession("my-agent", async (ctx) => {
  const session = new AgentSession({
    stt: new DeepgramSTT(),
    llm: new OpenAI(),
    tts: new OpenAI.TTS(),
  });

  session.on("user_input_transcribed", (transcript) => {
    // Only trigger LLM when transcription is final
    if (transcript.is_final) {
      // This triggers LLM processing (llm_node behavior)
      session.generateReply({ userInput: transcript.transcript });
    }
  });

  await session.start({ agent: null, room: ctx.room });
  await ctx.connect();
});

Notes and tips • session.generate_reply() / session.generateReply() is the recommended way to trigger LLM work after you verify the transcript is final. • Set AgentSession with

auto_reply=False

if you want full explicit control and to avoid any automatic LLM triggers. • The

user_input_transcribed

event fires for interim and final transcripts — the

is_final

check prevents processing partial text. • If you need to pass transcript metadata (timestamps, alternatives) into the LLM, include them in the

user_input

or session/context payload when calling generate_reply. If you tell me which SDK and exact AgentSession options you’re using (e.g., Node vs Python, auto_reply setting, or custom llm_node overrides), I can adapt the example to match your codebase and show how to pass additional context into the LLM. Sources: Pipeline nodes and hooks | LiveKit Documentation

auto_reply=False

does not exist. We use livekit agent 1.3.10. Please tell me the accurate information.

Thanks for the callout — you’re right to ask for version-accurate instructions. I’ll respond to your request about calling the LLM only when

is_final === true

for LiveKit Agent v1.3.10. Short answer • There is no built-in

auto_reply

flag in v1.3.10. • The correct pattern is to listen for the

user_input_transcribed

event and call

session.generate_reply()

(or the equivalent generate method) only when

is_final

is true. This ensures the LLM runs on the final transcript. Python example (v1.3.10 pattern)

from livekit.agents import AgentServer, AgentSession, JobContext, cli
from livekit.plugins import deepgram, openai, silero
from livekit.agents import Agent

server = AgentServer()

@server.rtc_session(agent_name="my-agent")
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        stt=deepgram.STT(),
        llm=openai.LLM(),
        tts=openai.TTS(),
        vad=silero.VAD.load(),
    )

    @session.on("user_input_transcribed")
    def on_transcript(event):
        # event.is_final will be True for final transcripts
        if event.is_final:
            # Trigger LLM processing only for final transcript
            session.generate_reply(user_input=event.transcript)

    await session.start(agent=Agent(instructions="You are a helpful assistant."), room=ctx.room)
    await ctx.connect()

if __name__ == "__main__":
    cli.run_app(server)

JavaScript/TypeScript example (v1.3.10 pattern)

import { AgentServer, AgentSession } from "livekit-agents";
import { DeepgramSTT } from "livekit-plugins-deepgram";
import { OpenAI } from "livekit-plugins-openai";

const server = new AgentServer();

server.rtcSession("my-agent", async (ctx) => {
  const session = new AgentSession({
    stt: new DeepgramSTT(),
    llm: new OpenAI(),
    tts: new OpenAI.TTS(),
  });

  session.on("user_input_transcribed", (event) => {
    if (event.is_final) {
      // Only call the LLM when transcript is final
      session.generateReply({ userInput: event.transcript });
    }
  });

  await session.start({ agent: null, room: ctx.room });
  await ctx.connect();
});

Notes and tips • The

user_input_transcribed

event is emitted for both interim and final transcripts; the

is_final

property is how you gate processing. •

session.generate_reply()

(or

generateReply

) is the correct method to start the LLM pipeline in v1.3.10; call it only when

is_final

is true. • If you have custom

llm_node

overrides, move the invocation out of automatic flows and into the final-transcript handler so your

llm_node

logic runs only after the final text is available. • If you need to pass additional metadata (timestamps, alternatives) to the LLM, include that in the

user_input

or session/context you pass to

generate_reply()

. If you want, tell me: • Which SDK (Python or Node) you’re using in production, • Any custom

llm_node

behavior you’ve implemented, and I’ll adapt the sample exactly to your codebase and show how to preserve your

llm_node

behavior while ensuring it only runs on final transcripts.

we use python. we implemented custom llm_node.

Thanks — I’ll show a Python pattern for LiveKit Agents v1.3.10 that ensures your custom

llm_node

logic only runs after the final transcript (

is_final == True

) is available. Key idea: don’t run LLM logic directly in any automatic path. Instead, handle the

user_input_transcribed

event, check

is_final

, and then call a wrapper that invokes your custom

llm_node

code with the final transcript. Example: wiring a custom llm_node to run only on final transcripts

from livekit.agents import AgentServer, AgentSession, JobContext, Agent, cli
from livekit.plugins import deepgram, openai, silero

server = AgentServer()

# Example custom llm_node implementation as a callable function/class
# Replace this with your actual llm_node logic
class MyCustomLLM:
    def __init__(self, llm_plugin):
        self.llm = llm_plugin

    async def run(self, user_input, context):
        # Put your existing llm_node logic here.
        # Example: call the LLM plugin and handle streaming/chunks as you do now.
        resp = await self.llm.generate(user_input=user_input, context=context)
        # handle resp (stream to TTS, send events, etc.)
        return resp

@server.rtc_session(agent_name="my-agent")
async def entrypoint(ctx: JobContext):
    # Create the session with your STT/LLM/TTS plugins
    session = AgentSession(
        stt=deepgram.STT(),
        llm=openai.LLM(),  # used by MyCustomLLM internally
        tts=openai.TTS(),
        vad=silero.VAD.load(),
    )

    # Instantiate your custom llm handler
    custom_llm = MyCustomLLM(session.llm)

    # Keep any state/context you need in session.userdata or a closure
    session.userdata["conversation_context"] = {}

    # Listen for transcript events and only trigger LLM when is_final is True
    @session.on("user_input_transcribed")
    def on_transcript(event):
        # event.transcript: the text
        # event.is_final: boolean
        # event.metadata/timestamps: if available
        if not getattr(event, "is_final", False):
            # Interim transcript: ignore or optionally update UI/state
            return

        # Final transcript: call your custom llm_node logic
        # If your llm_node is async, schedule it appropriately
        # Here we call generate_reply which will internally drive the llm_node pipeline,
        # but if you have custom code, call it directly.
        user_text = event.transcript

        # Option A: Use session.generate_reply to go through the normal pipeline
        # and your llm_node override (if your override hooks into session-generated replies).
        session.generate_reply(user_input=user_text)

        # Option B: Directly invoke your custom llm runner (if you manage the LLM lifecycle)
        # If custom_llm.run is async, schedule it:
        # import asyncio
        # asyncio.create_task(custom_llm.run(user_input=user_text, context=session.userdata["conversation_context"]))

    # If you have a custom llm_node override implemented elsewhere, ensure it is invoked
    # by the path you choose above (Option A to use session.generate_reply or Option B to call it directly).

    # Start the session and connect
    await session.start(agent=Agent(instructions="You are a helpful assistant."), room=ctx.room)
    await ctx.connect()

if __name__ == "__main__":
    cli.run_app(server)

Important notes for integrating your existing custom `llm_node`: • If your

llm_node

is implemented as a method override in a subclass or plugin that the AgentSession uses during

generate_reply()

, choose Option A (call

session.generate_reply(user_input=...)

). That will trigger the usual pipeline and call your overridden

llm_node

. • If your

llm_node

logic is fully custom and invoked outside

session.generate_reply()

, call it directly from the

is_final

branch (Option B). If it’s async, use

asyncio.create_task()

so the event handler doesn’t block. • Preserve and pass any conversation state (context, prior messages, timestamps) from

session.userdata

or another store into your LLM call so the model receives full context. •

user_input_transcribed

fires for both interim and final transcripts — the

is_final

guard is the single-source-of-truth for finality. If you share how your custom

llm_node

is currently defined (e.g., as an override method on a subclass, an async function, or integrated into

session.generate_reply()

), I can adapt the snippet to show the minimal changes needed so your

llm_node

only runs after the final transcript.

Is there any way to prohibit calling llm_node in unintented cases like

auto_reply=False

?

Thanks — I’ll be direct and version-accurate. Short answer • No: LiveKit Agents v1.3.10 does not provide a built-in flag like

auto_reply=False

to globally disable or block

llm_node

from being invoked automatically. • Recommended: Prevent unintended LLM runs by explicitly controlling when you call the LLM — i.e., only invoke

session.generate_reply()

(or your custom LLM entry point) from a handler that checks

user_input_transcribed.is_final == True

. Why there’s no single toggle • The SDK in v1.3.10 starts LLM work when the pipeline is instructed to generate a reply. There is no global “disable auto-reply” switch in that release, so prevention must be handled at the application layer. Practical patterns to avoid unintended llm_node calls • Gate on final transcripts ◦ Listen for

user_input_transcribed

and only call

session.generate_reply(user_input=...)

when

event.is_final

is true. • Centralize LLM invocation ◦ Ensure all code paths that could trigger the LLM go through a single helper function (e.g.,

maybe_generate_reply(user_text)

). That makes it easy to audit and add the final-check in one place. • If you have existing automatic hooks, remove or disable them ◦ Search your codebase for unconditional calls to

session.generate_reply()

or other triggers and wrap them with final-transcript checks or route them through your centralized helper. • Use explicit async scheduling for custom LLM logic ◦ If your custom

llm_node

is async and sometimes invoked indirectly, call it via

asyncio.create_task()

only from the final-transcript handler, or add a guard flag to the task launcher so it won’t run for interim transcripts. • Maintain conversation state centrally ◦ Keep context in

session.userdata

(or another store) and require that the final-transcript handler assemble context and then call the LLM. This prevents stray handlers from launching LLM work without the correct state. Concrete defensive code pattern (summary) • One event handler for

user_input_transcribed

that: ◦ returns early for interim transcripts, ◦ assembles context, ◦ calls

session.generate_reply()

(or your custom wrapper) only on

is_final

. • Replace any ad-hoc LLM calls elsewhere with calls to a single wrapper function that enforces the final-check. If you want, I can: • Review a small snippet of your current Python code that shows where

generate_reply()

or your custom

llm_node

is invoked and show exactly where to add the guard, or • Provide a minimal patched example that converts all LLM invocations to a single guarded wrapper so accidental calls are impossible. Which would help you most — a targeted patch for your code or a small self-contained example that you can drop into your service?