How can I preprocess audio frames before STT in AI...
# ask-aif
few-soccer-7112
07/14/2025, 5:38 AMHow can I preprocess audio frames before STT in AI agents telephony applications using AudioProcessingModule. Show me a code examples
t
tall-airline-16756
07/14/2025, 5:38 AMThanks for reaching out, Muthana. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
tall-airline-16756
07/14/2025, 5:39 AMYou can preprocess audio frames before speech-to-text (STT) in AI agents telephony applications by integrating the AudioProcessingModule in your agent's STT node. This module provides WebRTC audio processing features such as echo cancellation, noise suppression, high-pass filtering, and auto gain control. Audio frames must be exactly 10 ms in duration for proper processing.
Basic implementation example:
Copy code
from livekit import rtc
from livekit.agents import ModelSettings, stt, Agent
from livekit.rtc import AudioProcessingModule
from typing import AsyncIterable, Optional
class AudioPreprocessingAgent(Agent):
def __init__(self):
super().__init__(
instructions="Your agent instructions here",
# your STT, LLM, TTS configuration
)
# Initialize AudioProcessingModule with desired features
self.apm = AudioProcessingModule(
echo_cancellation=True,
noise_suppression=True,
high_pass_filter=True,
auto_gain_control=True,
)
async def stt_node(
self,
audio: AsyncIterable[rtc.AudioFrame],
model_settings: ModelSettings
) -> Optional[AsyncIterable[stt.SpeechEvent]]:
async def preprocessed_audio():
async for frame in audio:
# Process the audio frame with APM
self.apm.process_stream(frame)
yield frame
# Pass preprocessed audio to default STT node
return Agent.default.stt_node(self, preprocessed_audio(), model_settings) Copy code
async def process_far_end_audio(self, far_end_audio: AsyncIterable[rtc.AudioFrame]):
async for frame in far_end_audio:
# Process the rendered (far-end) audio for echo cancellation
self.apm.process_reverse_stream(frame)self.apm.set_stream_delay_ms()f
few-soccer-7112
07/14/2025, 5:42 AMwhat about before the vad?
t
tall-airline-16756
07/14/2025, 5:43 AMI don't have the answer you're looking for.
You could also try asking your question:
• in one of the other Slack channels
or
• to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code
If you find the answer, please post it here to help others!
b
brainy-australia-12132
07/15/2025, 1:42 AMgetting this error when session closes after implementing the changes you gave us
thread '<unnamed>' panicked at libwebrtc\src\native\apm.rs:50:9:
assertion
left == rightRUST_BACKTRACE=12 Views