You can process audio before it reaches Voice Activity Detection (VAD) in the Python Agents framework by intercepting and preprocessing audio frames in your custom agent. The recommended approach is to override the

stt_node

method in your Agent class and apply your processing logic there. Here’s a streamlined example:

async def stt_node(self, audio: AsyncIterable[rtc.AudioFrame], model_settings: ModelSettings) -> Optional[AsyncIterable[stt.SpeechEvent]]:
    async def filtered_audio():
        async for frame in audio:
            # Apply your custom preprocessing here
            yield processed_frame
    return await super().stt_node(filtered_audio(), model_settings)

Within

filtered_audio

, you can use audio processing libraries like

numpy

or

librosa

to implement filtering, noise reduction, or other transformations. Just ensure the output maintains the sample rate and buffer configuration required by the VAD component (typically 16kHz for Silero VAD). If you want to integrate built-in preprocessing options (like noise cancellation), you can use

RoomInputOptions

with the relevant configuration. This method ensures that your preprocessing occurs on all audio frames before they're handed off to the VAD or STT pipeline. Sources: livekit-agents/livekit/agents/voice/audio_recognition.py | Voice pipeline nodes | LiveKit Docs | realtime/openai-realtime-pitch-shift.py | agent.py