https://livekit.io logo
Join Slack
Powered by
For a smoother experience, I want to implement fad...
# ask-ai
m
For a smoother experience, I want to implement fade-out effect on interruption by customising AudioOutput class, can you give some examples of the implementation.
t
Thanks for reaching out, aryan. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
I'd love to help you implement a fade-out effect when an interruption occurs by customizing the 
AudioOutput
class. You can achieve this by extending the base class and overriding the 
clear_buffer()
method to gently lower the volume before stopping playback. Here's an example to guide you through the process: 1. Create a Custom 
AudioOutput
Class
Copy code
from livekit import rtc
from <http://livekit.agents.voice.io|livekit.agents.voice.io> import AudioOutput, AudioOutputCapabilities
import asyncio
import numpy as np

class FadeOutAudioOutput(AudioOutput):
    def __init__(self, sample_rate: int = 48000, num_channels: int = 1):
        super().__init__(
            label="FadeOutAudioOutput",
            capabilities=AudioOutputCapabilities(pause=True),
            sample_rate=sample_rate,
        )
        self._audio_source = rtc.AudioSource(sample_rate, num_channels)
        self._buffer = bytearray()
        self._fade_out = False
        self._fade_duration = 0.5  # 500ms fade-out
2. Override 
capture_frame()
to Process Audio with Fade-Out
Copy code
async def capture_frame(self, frame: rtc.AudioFrame):
        if not self._capturing:
            self._capturing = True
            self._playback_segments_count += 1

        if self._fade_out:
            audio_data = np.frombuffer(frame.data, dtype=np.int16)
            elapsed_time = asyncio.get_event_loop().time() - self._fade_start_time
            fade_factor = max(0.0, 1.0 - (elapsed_time / self._fade_duration))
            audio_data = np.round(audio_data * fade_factor).astype(np.int16)
            frame = rtc.AudioFrame(
                data=audio_data.tobytes(),
                sample_rate=frame.sample_rate,
                samples_per_channel=frame.samples_per_channel,
                num_channels=frame.num_channels,
            )
        await self._audio_source.capture_frame(frame)
        if self._fade_out and elapsed_time >= self._fade_duration:
            self._fade_out = False
            self._buffer.clear()
            self.on_playback_finished(playback_position=frame.duration, interrupted=True)
3. Override 
clear_buffer()
to Trigger Fade-Out
Copy code
def clear_buffer(self):
        if not self._pushed_duration:
            return
        if not self._fade_out:
            self._fade_out = True
            self._fade_start_time = asyncio.get_event_loop().time()
4. Synchronize with Playout Using 
wait_for_playout()
Copy code
async def stop_with_fade(self):
        self.clear_buffer()
        if self._fade_out:
            await self.wait_for_playout()
This method helps create smooth audio transitions by gently lowering the volume over the fade duration. Please feel free to reach out if you'd like me to walk you through any part of this or if you have any other questions—I’m here to help! Sources: livekit-agents/livekit/agents/voice/agent_activity.py | agents/src/voice/agent_activity.ts | meditation_assistant/audio_handler.py | How to Handle Speech Interruptions More Naturally | Pipeline nodes and hooks | LiveKit Docs
m
Let's say I have an audio file which I want to play instead of generating the response from TTS, how can I implement this decision node using livekit?
t
If you'd like to play an audio file instead of generating a TTS response in LiveKit, a great way to do this is by using the 
session.say()
method. You can provide pre-synthesized audio through an 
AsyncIterable[rtc.AudioFrame]
, which lets you bypass the TTS engine and play your chosen audio file directly to the participant. It’s pretty straightforward! Example Implementation:
Copy code
from livekit.agents.utils.audio import audio_frames_from_file

# Play a pre-recorded audio file
await session.say(
    audio=audio_frames_from_file("path/to/audio.wav"),
    allow_interruptions=False,
)
Alternative: Manually Load Audio Frames
Copy code
import wave
import rtc

with wave.open("path/to/audio.wav", 'rb') as wav_file:
    frames = wav_file.readframes(wav_file.getnframes())
    audio_frame = rtc.AudioFrame(
        data=frames,
        sample_rate=wav_file.getframerate(),
        num_channels=wav_file.getnchannels(),
        samples_per_channel=wav_file.getnframes()
    )

async def audio_generator():
    yield audio_frame

await session.say(audio=audio_generator())
Both of these methods will help you play your pre-recorded audio smoothly. If you have any questions or need a hand with anything else, just let me know—I’m here to help! Sources: examples/voice_agents/background_audio.py | Docs::Agents | basics/playing_audio.py | delay-handling/README.mdoc
m
Does livekit offer a way to maintain states of the entire llm prompt? This could be used as decision node
t
LiveKit offers several helpful ways to keep track of the entire LLM prompt state, which you can use for decision nodes. Let me walk you through them: 1. Preserving Full Prompt State Across Agents If you'd like to pass the full conversation history to a new agent, you can do so easily using the 
chat_ctx
parameter:
Copy code
return TechnicalSupportAgent(chat_ctx=self.session.chat_ctx)
2. Manual Context Updates You can also update or add messages to the LLM prompt anytime with `update_chat_ctx()`:
Copy code
chat_ctx = ChatContext()
chat_ctx.add_message(role="user", content="My name is Alice")
await agent.update_chat_ctx(chat_ctx)
3. Lifecycle Hooks To customize how context is preserved or enriched when agents switch, using hooks like 
on_enter()
can be really handy:
Copy code
def on_enter(self):
    self.chat_ctx.add_message(role="system", content="Current state: ...")
    await self.update_chat_ctx(self.chat_ctx)
4. Full Session History If you ever need to review or debug, you can access the complete conversation history through 
session.history
. 5. External Memory Systems For long-term memory across sessions, integrating with tools like Letta or Mem0 is a great option. If you’d like, I’m here to help you with any specific implementation or use case you have in mind. Just let me know! Sources: Workflows | LiveKit Docs | Testing and evaluation | LiveKit Docs | complex-agents/role-playing/agents/base_agent.py | Pipeline nodes and hooks | LiveKit Docs | Sending text | LiveKit Docs
2 Views
Open in Slack
PreviousNext
Powered by