For livekit agents (basic_agent.py), can we route the frontend user's text inputs (example: conversation_item_added) through on_user_turn_completed to have a consolidated pipeline of both voice and text inputs rolling up into the same logic?

Thanks for reaching out, aphinity. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

For some reason, text input does not feed into the on_user_turn_completed lifecycle hook that the voice uses. @session.on("conversation_item_added") def *on_conversation_item_added*(ev) -> None:

Thanks — I’m responding to your report that frontend text inputs are not triggering

on_user_turn_completed

the way voice does. Why this is happening •

on_user_turn_completed

is invoked after a spoken user turn has been transcribed and the system detects the end of that turn (via VAD/STT endpointing or a turn-detector model). Text sent directly from the frontend via the

lk.chat

text stream does not go through the speech/turn-detection pipeline, so it does not automatically trigger

on_user_turn_completed

. • Text input does generate

conversation_item_added

events (and can be handled via a

text_input_cb

), which is why you see those events but not the turn-completed lifecycle hook. Two simple ways to unify handling • Use

conversation_item_added

as the single entry point: handle user text there and also call the same shared logic when you receive a finalized transcript from speech. • Keep

on_user_turn_completed

for speech, and explicitly invoke your shared handler from your text handler (so both paths funnel into the same code). Suggested implementation pattern 1. Create a shared handler that accepts plain text and performs your unified processing (RAG, LLM calls, response generation, etc.). 2. Wire speech finalization to the shared handler (from

on_user_turn_completed

or

user_input_transcribed

when

is_final

is true). 3. Wire text messages to the same shared handler inside

conversation_item_added

when the item is a user message. Example sketch (adapt to your event names/objects):

def shared_user_message_handler(text: str):
    # Unified logic for RAG / prompt prep / LLM call / etc.
    pass

@session.on("user_input_transcribed")
def on_transcript(transcript):
    if transcript.is_final:
        shared_user_message_handler(transcript.text)

@session.on("conversation_item_added")
def on_conversation_item_added(ev):
    if ev.item.type == "message" and ev.item.role == "user":
        shared_user_message_handler(ev.item.text)

Things to check / consider • If you expected text to automatically call

on_user_turn_completed

, that behavior does not occur because no transcription or turn-detection runs for plain text. • If you need different handling depending on modality, inspect the

conversation_item

metadata or event fields and branch inside your shared handler. • Text input interrupts ongoing TTS by default; if you need different interruption behavior, factor that into your handler. If you want, share: • Your current session configuration (especially

turn_detection

), and • The

conversation_item_added

handler code and an example text message event payload. I’ll use that to give a tailored snippet you can drop into your agent so both text and voice flow through the exact same logic. Sources: Turns overview | LiveKit Documentation | Text and transcriptions | LiveKit Documentation | Agent session | LiveKit Documentation

However, the AI auto responds when text input is added, is it because the framework has internal text-chat auto-reply behavior?

Thanks — I’m responding to your question about why the agent auto-responds when frontend text is added. Short answer: Yes. The framework has built‑in behavior that treats text sent on the

lk.chat

text stream as user input and will interrupt the agent and generate a reply by default. Why this happens • When a text message is sent from the frontend, it is added to the conversation as a

conversation_item_added

event. • The agent listens for those text messages and, by default, interrupts any ongoing speech and invokes the reply-generation flow (internally calling the same generate/response logic used for other inputs). • Text input does not go through the speech turn-detection pipeline, but the framework still treats it as an immediate user input that should be handled, which is why you see the auto reply. How you can change that behavior • Provide a custom text input callback (

text_input_cb

/

TextInputOptions

) in your Room/Session configuration to intercept text messages before the default auto-reply runs. • In that callback you can: ◦ Preprocess or filter messages (commands, moderation, etc.) ◦ Decide whether to call

generate_reply()

(to trigger an agent response) or ignore/handle the message without generating a reply ◦ Control interruption behavior (choose not to interrupt ongoing TTS) • Alternatively, keep the default flow but route both speech transcripts and text to a shared handler (see previous examples) so you centralize logic while still controlling when and how replies are generated. If you want, share one of the following and I’ll provide a concrete code snippet: • Your current RoomOptions/session startup code (so I can show exactly where to add

text_input_cb

) • The

conversation_item_added

handler you’re using and an example text event payload I’ll then give a ready-to-drop-in handler that disables automatic replies or that funnels text through your

on_user_turn_completed

-style logic. Sources: Text and transcriptions | LiveKit Documentation

You mentioned this before. How come the text input does not call on_user_turn_completed then? • When a user sends text from the frontend (for example using

sendText()

), it generates a

conversation_item_added

event and interrupts any active speech. • The agent treats that text as a completed user turn, which invokes the

on_user_turn_completed

lifecycle hook when the agent is configured with turn detection (for example

turn_detection="stt"

or using a turn detector model). • Because both STT-derived transcripts and direct text inputs reach the same lifecycle hook, you can perform unified logic there (RAG, context injection, message modification, etc.).

Thanks — I’m responding to your follow-up asking why text input doesn’t trigger

on_user_turn_completed

despite earlier statements that it could. Clear answer: by default, text input does NOT trigger

on_user_turn_completed

. That lifecycle hook is specifically tied to the voice turn-detection/transcription pipeline (VAD/STT endpointing or a turn-detector model). Plain text sent via the

lk.chat

text stream goes straight into the conversation as a

conversation_item_added

event and does not run through the speech/turn-detection machinery, so it won’t automatically call

on_user_turn_completed

. Where the confusion comes from • Text messages are treated as user input and will interrupt TTS and cause the agent to generate a reply by default — so it feels like the same flow as voice. • But interrupting and auto-generating a reply is separate from the voice turn-detection lifecycle. The framework has built-in auto-reply behavior for text, yet it does not route text through the voice turn-detection hook

on_user_turn_completed

. What to do to unify handling • Use a shared handler and call it from both places: a. Handle text in

conversation_item_added

(or via

text_input_cb

) and pass the message text into your shared logic. b. Handle finalized transcripts from speech (either in

user_input_transcribed

when

is_final

is true or in

on_user_turn_completed

) and pass the transcript into the same shared logic. • Or override the default text auto-reply by providing a custom

text_input_cb

in your Room/Session configuration and explicitly call your shared handler there (so you control interruption and reply timing). Minimal pattern example (conceptual): 1. Create a shared function:

shared_user_message_handler(text: str)

2. From speech-final events: ◦ if transcript.is_final: call

shared_user_message_handler(transcript.text)

1. From text events: ◦ in

conversation_item_added

or

text_input_cb

, if item.role == "user": call

shared_user_message_handler(item.text)

Practical tips • If you want text to behave exactly like a finalized transcript (and possibly trigger any logic normally only run in

on_user_turn_completed

), call that shared logic from your text handler. • If you need to know modality inside the handler, add a

source

parameter (e.g.,

"text"

vs

"speech"

) or inspect event metadata. • To stop default auto-replies, implement

text_input_cb

and decide when to call

generate_reply()

or

session.say()

yourself. If you’d like, share: • Your

RoomOptions

/ session startup code (especially

turn_detection

), and • The

conversation_item_added

handler or a sample event payload I’ll return a precise code snippet you can drop in to route text into the same pipeline as speech (and optionally disable the default auto-reply).

Would it make sense to centralize all logic in the on_user_turn_completed lifecycle hook and have text inputs feed into there? This is where all my logic happens (e.g. video frame capture, etc.)

Thanks — I’m responding to your question about whether it makes sense to centralize all processing in

on_user_turn_completed

and have frontend text feed into that hook (since your main logic — video frame capture, RAG, etc. — runs there). Short answer: Yes, centralizing your logic is a good idea, but text inputs do not automatically call

on_user_turn_completed

. You should put your core processing into a shared function and explicitly invoke that function from both the voice turn-completion path and your text handler so every modality runs the same code. Why: the behavior you’re seeing •

on_user_turn_completed

is triggered by the voice turn-detection/transcription pipeline (VAD/STT endpointing or a turn-detector model). Plain text sent via

lk.chat

bypasses that pipeline; it generates

conversation_item_added

and the framework’s default auto-reply flow, but it does not call

on_user_turn_completed

. • Because of that, keeping logic only inside

on_user_turn_completed

means text inputs won’t execute it unless you explicitly route them there. Recommended pattern (concise) 1. Extract your unified processing into a single shared async function (e.g.,

process_user_input(text, source, ctx)

). 2. From the voice path: ◦ Keep using

on_user_turn_completed

(or finalized transcript event) and call

await process_user_input(transcript_text, "speech", ctx)

. 1. From the text path: ◦ Add a custom text input callback (

text_input_cb

) or use

conversation_item_added

and call

await process_user_input(message_text, "text", ctx)

there. 1. Optionally include a

source

parameter so the shared logic can branch if needed (different handling for text vs speech), and include any video frame or context injection before running RAG/LLM calls. Minimal conceptual example

async def process_user_input(text: str, source: str, ctx):
    # inject video frame/context if available
    # run RAG, run LLM, prepare response
    await ctx.generate_reply(user_input=text)

@session.on("conversation_item_added")
def on_conversation_item_added(ev):
    if ev.item.role == "user" and ev.item.type == "message":
        asyncio.create_task(process_user_input(ev.item.text, "text", ev.session_ctx))

async def on_user_turn_completed(turn_ctx, message):
    # message is from speech turn completion
    await process_user_input(message.text, "speech", turn_ctx)

Practical notes • Use

text_input_cb

if you need to intercept text before the framework’s default auto-reply, or to control interruption behavior. • Text interrupts TTS by default—if that matters, handle interruption explicitly in your text callback. • If you absolutely must have text appear to go through the same lifecycle hook (rather than calling a shared function), you would need to emulate the same inputs/events that

on_user_turn_completed

expects — but calling the shared function is simpler and less error-prone. • If your shared logic depends on objects present only in the voice-turn context, adapt it to accept a common minimal context object so it works from both entry points. If you want, share: • Your current

RoomOptions

/ session startup code (especially

turn_detection

), and • Your existing

on_user_turn_completed

and

conversation_item_added

handlers I’ll provide a ready-to-drop-in snippet tailored to your code that funnels frontend text into your centralized pipeline and optionally disables the default auto-reply. Sources: Pipeline nodes and hooks | LiveKit Documentation | Text and transcriptions | LiveKit Documentation | External data and RAG | LiveKit Documentation