is it possible to add audio equalization to improve sounds to feed the stt model?

Thanks for reaching out, Filippo. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

python

Thanks — great, Python is perfect for this. I’ll walk you through a concise, practical approach you can implement in LiveKit: where to apply EQ, a working example outline, and a few implementation tips. Acknowledgement • You want to add audio equalization in Python to improve audio sent to an STT model. That’s possible by preprocessing audio inside the

stt_node

override of a custom Agent. What to do (high level) • Override

stt_node

in your Agent to receive the incoming

rtc.AudioStream

. • Convert each `rtc.AudioFrame`’s raw PCM bytes into a numpy array. • Apply EQ/DSP to the PCM samples (bandpass, shelving, parametric EQ, gain, etc.). • Convert processed samples back to bytes and yield new

rtc.AudioFrame

objects into the STT pipeline. • Ensure sample rate, channel count, and format match what your STT provider expects. Minimal Python outline (conceptual) 1. Convert frame data to numpy (int16 -> float32) for processing. 2. Apply filters (example uses a Butterworth bandpass to emphasize speech). 3. Clip and convert back to int16. 4. Yield a new

rtc.AudioFrame

into the downstream STT node. Example skeleton (you can adapt this directly)

import numpy as np
from scipy.signal import butter, lfilter
from livekit.agents import Agent, stt, rtc

class EqualizedSTTAgent(Agent):
    def __init__(self):
        super().__init__()
        self.sample_rate = 16000  # match your STT model

    def _butter_bandpass_filter(self, data, lowcut, highcut, fs, order=6):
        nyq = 0.5 * fs
        b, a = butter(order, [lowcut/nyq, highcut/nyq], btype='band')
        return lfilter(b, a, data)

    async def stt_node(self, audio: rtc.AudioStream, model_settings: stt.ModelSettings):
        async def processed_stream():
            async for frame in audio:
                # Convert to numpy float for DSP
                samples = np.frombuffer(frame.data, dtype=np.int16).astype(np.float32)

                # Example: emphasize speech band 300-3000 Hz
                proc = self._butter_bandpass_filter(samples, 300.0, 3000.0, self.sample_rate)

                # Optional: apply gain, normalization, or other EQ logic here

                # Convert back to int16 safely
                proc = np.clip(proc, -32768, 32767).astype(np.int16)
                yield rtc.AudioFrame(
                    data=proc.tobytes(),
                    sample_rate=frame.sample_rate,
                    num_channels=frame.num_channels,
                    samples_per_channel=frame.samples_per_channel,
                )

        # Forward processed audio into the default STT node
        async for event in Agent.default.stt_node(self, processed_stream(), model_settings):
            yield event

Practical tips • Latency: Use lightweight streaming-friendly filters. Higher filter orders and long FFT windows add latency. • Sample rate and channels: Confirm STT expects mono or stereo and the sample rate (commonly 16 kHz for many STT models). Resample or downmix before STT if needed. • Avoid clipping: After boosting frequencies, normalize or apply limiter to prevent clipping. • CPU: Test CPU usage under realistic load—DSP cost can increase with more complex EQ. • Libraries: For simple filters use

scipy.signal

. For more advanced/musical EQ consider

pyroomacoustics

,

librosa

(analysis-focused), or implement IIR biquad filters for efficiency. • Testing: Compare STT transcripts before/after EQ with representative noisy audio to validate improvements. If you want, I can: • Provide a complete runnable example including session setup and AgentSession integration. • Replace the bandpass example with a parametric multi-band EQ (biquad filters) and give presets for telephony or noisy rooms. • Help verify correct sample rate/channel conversion for your STT provider — tell me which STT service you plan to use and whether your input is mono or stereo. Which of those would you like next? Sources: Capturing the First Few Seconds of a Call for Processing | Speech-to-text (STT) models overview | LiveKit Documentation