https://livekit.io logo
Join Slack
Powered by
I saw the thinking sound example ```background_aud...
# ask-ai
r
I saw the thinking sound example
Copy code
background_audio = BackgroundAudioPlayer(
        thinking_sound=[
            AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING, volume=0.8),
            AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING2, volume=0.7),
        ],
    )
    await background_audio.start(room=ctx.room, agent_session=session)
but instead of thinking sound I want to perform a frontend animation by calling an RPC to the frontend. What is the cleanest way to do it? event handler with 
user_turn_completed?
t
Thanks for reaching out, Ivan. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
You're asking whether to replace the thinking sound with a frontend animation via RPC and whether to use 
user_turn_completed
. Use lifecycle hooks that indicate the agent is actively processing, not 
user_turn_completed
. Recommended approaches • For tool calls / long-running tool work: use 
on_tools_call
. Send an RPC to start the animation at the start of the hook and another to stop it when tools finish (use try/finally). • For raw LLM generation: use 
on_llm_start
to send a "start" RPC, then stop the animation when the LLM finishes (e.g., wait for LLM done or use an LLM-completion hook). Minimal example (conceptual) 1. In the hook, find the remote participant and call: ◦ 
room.local_participant.perform_rpc(destination_identity=..., method="startThinkingAnimation", payload="")
1. After processing completes, call: ◦ 
...perform_rpc(..., method="stopThinkingAnimation", ...)
Frontend: register 
startThinkingAnimation
and 
stopThinkingAnimation
RPC handlers to run your animation. Why not 
user_turn_completed
? • 
user_turn_completed
fires right after the user stops speaking (before agent processing) and is intended for preprocessing—so it triggers too early and won’t cover ongoing tool/LLM work. If you want, I can: • Provide a concrete code snippet for 
on_tools_call
or 
on_llm_start
in your language of choice. • Show the frontend RPC registration example for LiveKit JS. Sources: Agent speech and audio | LiveKit docs | External data and RAG | LiveKit Docs | Remote method calls | LiveKit docs | Pipeline nodes and hooks | LiveKit docs
r
I don't see 
on_llm_start
in docs, you sure? give me an example
t
You're right to question that — there is no built-in 
on_llm_start
lifecycle hook. Use a pipeline node override (like 
llm_node
) or 
on_tools_call
to emit RPCs when LLM/tool work begins and ends. Below are concise examples. 1) Override 
llm_node
(Python) — send RPC before LLM inference and stop after:
Copy code
class MyAgent(Agent):
    async def llm_node(self, chat_ctx, tools, model_settings):
        room = self.context.room
        remote = next(iter(room.remote_participants.values()), None)
        if remote:
            await room.local_participant.perform_rpc(
                destination_identity=remote.identity,
                method="startThinkingAnimation",
                payload="",
            )

        try:
            async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
                yield chunk
        finally:
            if remote:
                await room.local_participant.perform_rpc(
                    destination_identity=remote.identity,
                    method="stopThinkingAnimation",
                    payload="",
                )
2) Use 
on_tools_call
(for long-running tool execution):
Copy code
async def on_tools_call(self, tools, ctx):
    room = self.context.room
    remote = next(iter(room.remote_participants.values()), None)
    if remote:
        await room.local_participant.perform_rpc(
            destination_identity=remote.identity,
            method="startThinkingAnimation",
            payload="",
        )

    try:
        return await super().on_tools_call(tools, ctx)
    finally:
        if remote:
            await room.local_participant.perform_rpc(
                destination_identity=remote.identity,
                method="stopThinkingAnimation",
                payload="",
            )
Frontend (LiveKit JS) — register RPC handlers:
Copy code
localParticipant.registerRpcMethod("startThinkingAnimation", async () => {
  // start animation
  return "ack";
});
localParticipant.registerRpcMethod("stopThinkingAnimation", async () => {
  // stop animation
  return "ack";
});
If you want, I can: • Provide a version that targets a specific participant by identity or metadata, or • Show how to send a data message/topic instead of RPC if that fits your frontend architecture. Which would you prefer? Sources: Pipeline nodes and hooks | LiveKit docs | Agents (Python) | livekit/livekit_composite | DeepWiki | Modifying LLM output before TTS | LiveKit docs
r
how about on "user_turn_completed" I send "play" RPC and when agent speaks I send "stop" RPC payload. give me an example
t
Below is a concise example showing how to send a "play" RPC in 
on_user_turn_completed
and send a "stop" RPC when the agent begins speaking. It uses 
perform_rpc
from the room's local participant and overrides the 
speak_node
(or the appropriate speak method) to notify the frontend right before speech output. 1) Python (agent-side)
Copy code
class MyAgent(Agent):
    async def on_user_turn_completed(self, ctx):
        room = self.context.room
        remote = next(iter(room.remote_participants.values()), None)
        if remote:
            await room.local_participant.perform_rpc(
                destination_identity=remote.identity,
                method="playThinkingAnimation",
                payload="",
            )
        # continue normal processing
        return await super().on_user_turn_completed(ctx)

    async def speak_node(self, *args, **kwargs):
        # called when agent is about to speak / TTS starts
        room = self.context.room
        remote = next(iter(room.remote_participants.values()), None)
        if remote:
            await room.local_participant.perform_rpc(
                destination_identity=remote.identity,
                method="stopThinkingAnimation",
                payload="",
            )

        # proceed with default speak behavior (yield audio chunks or call parent)
        async for chunk in Agent.default.speak_node(self, *args, **kwargs):
            yield chunk
Notes: • Use 
next(iter(room.remote_participants.values()), None)
to pick the remote participant; adjust selection if you target specific participants. • If your agent framework exposes a different speak hook name, use that hook (the goal is to run the "stop" RPC before TTS/audio is published). 2) Frontend (LiveKit JS) — RPC handlers
Copy code
localParticipant.registerRpcMethod("playThinkingAnimation", async () => {
  // start your animation / spinner
  return "ack";
});

localParticipant.registerRpcMethod("stopThinkingAnimation", async () => {
  // stop animation
  return "ack";
});
Would you like a variant that: • Targets a specific participant by identity/metadata, or • Uses data messages/topics instead of RPC for broader fan-out?
Open in Slack
PreviousNext
Powered by