https://livekit.io logo
Join Slack
Powered by
How can I modify this llm resonse in specific case...
# ask-ai
a
How can I modify this llm resonse in specific cases and send an audio playback and return none ?
Copy code
async def llm_node(
        self,
        chat_ctx: llm.ChatContext,
        tools: list[FunctionTool],
        model_settings: ModelSettings
    ) -> AsyncIterable[llm.ChatChunk]:
        # Insert custom preprocessing here
        async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
            # Insert custom postprocessing here
            if chunk.delta.tool_calls: 
                <http://logger.info|logger.info>("\n\n\n TOOL IS HAPPENING \n\n\n")       
                await self.session.say("Playing audio", audio=audio_generator("waiting_message_hindi.wav"))

            <http://logger.info|logger.info>(f"This is the current chunks coming: {chunk}")
            yield chunk
t
Thanks for reaching out, Aditya. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
You can customize your 
llm_node
method to trigger audio playback under specific conditions and then halt further response generation by returning early. Here is the pattern based on documented best practices:
Copy code
async def llm_node(self, chat_ctx, tools, model_settings):
    async for chunk in Agent.default.llm_node(self, chat_ctx, tools, model_settings):
        if should_trigger_audio_playback(chunk):
            await self.session.say("Playing audio", audio=audio_generator("waiting_message_hindi.wav"))
            return  # Stops yielding further chunks
        yield chunk
Returning from the async generator at the desired trigger (such as detecting a specific tool call, keyword, or chunk attribute) halts any further LLM output. The 
session.say
method accepts an audio generator, which can play your custom audio. This approach enables side effects such as audio playback in response to LLM output, and then stops any further output by returning early from the generator. Sources: pipeline-llm/replacing_llm_output.py | livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/experimental/realti... | Voice pipeline nodes | LiveKit Docs
a
This is not speaking anything as audio
t
Hey there! I wanted to chat with you about something important when it comes to audio playback. Even if you're providing your own audio manually, the 
session.say
method actually needs a Text-to-Speech (TTS) plugin to be set up in your session. Without it, you might run into some tricky situations where your audio just won't play - and sometimes it might even fail quietly without telling you. • Make sure you've got a TTS plugin (like ElevenLabs TTS) added to your agent's session. • Double-check that your session configuration has audio output enabled. Quick heads up for those using realtime models: you'll still need a TTS plugin for 
session.say
. If that feels complicated, you might want to use 
generate_reply()
instead. Here's a quick example of how to add TTS that might help:
Copy code
from livekit.plugins import elevenlabs
session = AgentSession(tts=elevenlabs.TTS(...))
I know configuration can feel a bit technical, but I'm here to help! Take a look at these requirements and make sure your session is set up with a solid TTS plugin. You've got this! Sources: livekit-agents/livekit/agents/voice/agent_session.py | Agent speech | LiveKit Docs | basics/playing_audio.py | Speech &amp; Audio Processing | livekit/livekit_composite | DeepWiki | Agent speech and audio | LiveKit Docs
Open in Slack
PreviousNext
Powered by