https://livekit.io logo
Join Slack
Powered by
Im trying to debug something. I recently switched ...
# ask-ai
c
Im trying to debug something. I recently switched over from using ogg format on my recordings to dual channel audio. But the dual channel audio using mp4 previously using .ogg just got too quite. My voice is soo quite that it sounds like whispers. Is this a code issue or an egress problem. How can i find out whats causing this? I've updated the json on SIP trunks to use krisp and ive also set my noise cancellations to BVCTelephony(). Its still too quite tho
t
Thanks for reaching out, Joshua. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — this is a clear, actionable problem to debug. I’m responding to your note that switching from .ogg to dual‑channel MP4 made recordings very quiet, and that you enabled 
krisp
on SIP trunks and 
BVCTelephony()
noise cancellation on the agent. Quick summary of likely causes • Using dual-channel (stereo) egress can result in each participant occupying a separate channel; some players or downmixing pipelines can cause perceived volume loss. • Running Krisp on the SIP trunk plus BVCTelephony() on the agent means the audio is being processed twice; double noise suppression often reduces overall signal amplitude. • Egress transcoding (the MP4 pipeline) can apply different handling than .ogg, and LiveKit does not provide an egress-level volume control to automatically normalize loudness. Recommended diagnostic steps (in order) 1. Reproduce a baseline ◦ Record a short test with a single speaker using the old setup that sounded correct (if still available) and one with the new setup. ◦ While testing, confirm what the participants hear live in the room before saving recordings. 1. Isolate noise cancellation ◦ Temporarily disable Krisp on the SIP trunk, leaving only BVCTelephony() enabled. Test again. ◦ Then disable BVCTelephony() and enable only Krisp. Test again. ◦ If audio is noticeably louder with a single noise-cancellation layer, double-processing is the culprit. 1. Test mono vs stereo egress ◦ Request a mono (single-channel) MP4 export or ask your egress to downmix to mono for a test recording. ◦ If mono restores normal volume, the issue is stereo/dual-channel handling or player downmixing. 1. Compare client vs recording ◦ If participants hear normal volume in the room but the saved MP4 is quiet → problem occurs during egress/transcoding. ◦ If participants already hear the audio as quiet before recording → problem is at input/processing (mic levels, agent filters, or SIP trunk processing). 1. Inspect raw waveforms ◦ Download both the .ogg and MP4 audio tracks and view them in an audio editor (e.g., Audacity). Compare peak and RMS levels. If the MP4 waveform has much lower amplitude, it’s a capture/transcode or processing issue; if similar, the player may be downmixing/attenuating. 1. Check playback and client volume controls ◦ For testing, increase playback volume programmatically with 
RemoteAudioTrack.setVolume()
in your client to see whether the recording’s volume can be corrected at playback time or if the file itself is low. Actions to try immediately • Turn off one of the noise-cancellation layers (either Krisp on SIP trunks or BVCTelephony on the agent). Do tests after each change. • Produce a mono test recording (downmix) to confirm whether stereo handling is the issue. • Compare the raw audio files in an editor to confirm where amplitude loss happens. If you want, I can help further if you share: • Whether participants hear normal volume in the room but only recordings are quiet, or if live audio is quiet too. • The egress type you’re using (Track Composite, Room Composite, or single-track egress). • A short sample (or waveform screenshot) of the .ogg vs MP4 output and the SIP/agent noise-cancellation settings (which one you prefer to keep). Next steps I can help with • Walk through how to change egress to mono for a quick test. • Show how to temporarily disable Krisp or BVCTelephony() and what to look for. • Interpret waveform screenshots or short audio samples you share to pinpoint whether loss is before or after egress. Feel free to tell me which test you want to run first or paste the specific egress/agent settings you’re using and I’ll give step-by-step instructions. Sources: Universal Egress | Universal Egress | Choosing the right noise cancellation for your agent | LiveKit | sip | livekit/livekit_composite | DeepWiki | client-sdk-android | livekit/livekit_composite | DeepWiki
c
Never mind i had only one earphone on
I'm using a telephony agent and we've been facing some problems with users saying shorts words that are audible in recordings but the agent does not pick up on these works and usuallly the silence detection triggers a response. We have been using BVC only
t
Thanks — I hear you: short user words are audible in recordings but your telephony agent misses them and silence detection fires. You said you’re currently using BVC only. That likely explains the issue and here’s a practical, step‑by‑step path to fix and verify it. Why this is happening (short) • 
BVC
is tuned for full‑band WebRTC audio and can aggressively suppress low‑energy or short utterances on narrowband telephony audio. • Short words (single syllables, “yes”, “ok”) are already hard for VAD/STT; aggressive noise cancellation can reduce their amplitude below the detection threshold even though they remain audible in the raw recording. Immediate actions to try 1. Switch to the telephony-optimized filter ◦ Replace 
BVC
with 
BVCTelephony()
for SIP/telephony participants. This model preserves short, narrowband speech better. 1. Test with noise cancellation off ◦ Temporarily disable noise cancellation for the call and replay test scenarios. If short words are detected without noise cancellation, the filter was removing them. 1. Tune silence/VAD settings ◦ Increase the minimum silence duration and endpointing delay slightly (example: raise 
min_silence_duration
toward ~0.8–1.0s, increase 
min_endpointing_delay
a bit) so brief pauses don’t trigger endpointing. ◦ Add small buffering or a short lookback before declaring silence if your logic supports it. 1. Add retry/reprompt logic ◦ Instead of ending on the first silence, prompt once or twice (or use a short confirmation prompt) before concluding the turn. This reduces false negatives from transient misses. 1. Validate STT behavior ◦ Test the same audio through alternate STT models (telephony-optimized models if available). Some STT providers detect short utterances more reliably on narrowband audio. How to determine it’s resolved • If live callers now hear normal audio and the agent detects those short words after switching to 
BVCTelephony()
(or after disabling noise cancellation), the issue was the noise filter. • If recordings still contain the words but live detection still misses them even with 
BVCTelephony()
disabled, the problem is likely VAD/STT configuration (thresholds, segmentation) and you should tune VAD/endpointing and/or switch STT model. If you want, I can: • Walk you through changing the agent filter to 
BVCTelephony()
in your configuration. • Suggest exact VAD/endpointing parameter values to test (based on your current settings). • Help interpret a short recording sample or waveform to confirm whether the energy of the short words is being suppressed prior to STT. Which would you like to try first — switch to 
BVCTelephony()
for SIP participants or run a quick test with noise cancellation disabled? Sources: Choosing the right noise cancellation for your agent | LiveKit | Troubleshooting STT not picking up utterances | LiveKit | Improving voice AI's turn detection with transformers | How to implement call greeting retries in voice agents | LiveKit | How to detect and handle silence during calls | LiveKit
c
cant i use krisp and bvc telephone both enabled? for my telephony agent
t
Short answer: you should not enable Krisp and a BVC telephony filter at the same time for the same telephony agent. Why not (brief) • Krisp and 
BVCTelephony()
both perform aggressive noise suppression. Running them together applies two layers of real‑time filtering and commonly reduces speech amplitude and clarity, which causes short words to be lost or VAD/STT to miss them. • This double-processing is the most likely reason you were seeing audible words in recordings but the agent not detecting them. Recommended approach • Use exactly one noise-cancellation stage for a telephony participant: ◦ Prefer 
BVCTelephony()
for SIP/telephony audio — it’s optimized for narrowband telephony and preserves short/quiet utterances better. ◦ Only enable Krisp if you have a specific need that BVCTelephony() cannot address, and then disable the BVC telephony filter. How to test safely 1. Turn off Krisp on the SIP trunk and keep 
BVCTelephony()
enabled. Run several short-utterance tests. 2. If you still have problems, disable noise cancellation entirely for a test to confirm whether any filter is suppressing speech. 3. If you prefer Krisp, enable Krisp and remove the BVC telephony filter and run the same tests to compare detection and perceived quality. If you want, I can: • Give step-by-step instructions to switch the telephony filter in your agent config. • Suggest a short test checklist (sample phrases, VAD settings) to confirm which configuration gives the best detection for your use case. Which test would you like to run first — switch to 
BVCTelephony()
only or run a noise-cancellation‑off baseline?
c
Its still really bad and we are now facing issues when users are speaking whole sentences, the agent cannot head anything at all
t
I’m sorry you’re still struggling — I can tell this is urgent. I’m going to help you isolate the root cause quickly. First, a short checklist of likely causes and immediate checks, then a few targeted questions so I can give precise next steps. Short checklist (do these now if possible) • Temporarily disable all noise cancellation for a quick baseline test (disable Krisp on the SIP trunk and remove any BVC/BVCTelephony() filters). Record a short call and see if the agent can detect speech. This confirms whether filters are the cause. • Verify mic/input levels on the telephony side (caller device or SIP gateway) are normal — low upstream gain can make speech effectively disappear after any processing. • Confirm whether participants hear the audio normally in the live call. If callers and other participants hear full sentences but the agent does not, the problem is upstream of STT (filtering/transcoding/VAD). • Try sending the raw audio (or a short recorded sample) from a failing call into an audio player/editor to inspect waveform amplitude and frequency content. Targeted questions (these will narrow down the exact cause quickly) 1. Which noise cancellation filters are currently enabled on the call right now? (Krisp on SIP trunk? 
BVC
or 
BVCTelephony()
on agent? Both?) 2. Which egress/agent audio path are you using? (Direct telephony agent ingestion, SIP trunk into LiveKit, TrackComposite/RoomComposite egress, etc.) 3. Are callers hearing their own speech at normal volume during the live call? Or is everything quiet to them as well? 4. Do you have access to a short recording (10–20s) of a failing call you can share or summarize (e.g., “waveform shows very low peaks”)? If you can’t share audio, a screenshot of the waveform helps. 5. What STT/VAD system are you using for the agent? Any custom endpointing or silence thresholds configured? Immediate next steps I recommend while you answer • If you haven’t already: disable Krisp and BV C variants completely and retest. If disabling them restores detection for full sentences, re-enable only 
BVCTelephony()
(not Krisp) and retest to find the best single filter. • If disabling all filters does not help, capture a raw audio sample from the telephony input and inspect in an editor (Audacity): check RMS/peak levels and whether speech is present across expected frequency bands. If the sample is essentially flat/low, the problem is at the SIP gateway/mic or upstream codec/gain. • If the raw sample looks fine but STT still fails, increase VAD/endpointing timeouts and reduce aggressiveness of energy thresholds so longer speech segments aren’t dropped. I can: • Walk you step-by-step to disable Krisp/BVC in your config and run a controlled test. • Suggest concrete VAD/endpointing parameter values to try based on your current settings. • Help interpret a waveform screenshot or a short sample and tell you whether the loss is from filtering, gain, or a codec issue. Which would you like to do first — run the all-filters-off baseline test, or share your current config answers (Q1–Q5) so I can recommend precise changes?
c
problem is that we as participants & also anyone listening into the recordings, are able to hear what the user is saying even tho its a little faint. but what actaully is happening is that, with teh live gemini voice model it still is unable to pickup on whole sentences, that were clearly said. Nor it is able to pick up on small words sometimes. We saw no difference in BVC telephony vs BVC.
t
Thanks — I understand: callers and listeners can hear the audio (sometimes faint), but the live Gemini voice model often misses whole sentences and short words. You’ve already compared BVC vs BVCTelephony and saw no difference. I know this is frustrating — let’s narrow this down quickly. A few quick clarifying questions so I can give precise steps: 1. Which exact Gemini voice model and settings are you using for live transcription/ASR? (model name, sample rate if configurable, any voice activity or energy thresholds set) 2. Confirm the full audio chain: SIP trunk → LiveKit ingestion → agent STT. Are you using any other middleboxes (SBCs, transcoding gateways) before LiveKit? 3. Which noise cancellation is currently enabled on the live path right now (Krisp on SIP, BVC/BVCTelephony on agent, or none)? 4. Does this failure happen consistently for all callers or only certain callers (specific carriers, mobile vs landline)? 5. Do you have a representative short recording (10–30s) you can share or describe (waveform screenshot, RMS/peak levels)? You can also tell me whether the recording shows clear speech energy but STT outputs nothing or incorrect text. While you gather that info, try these targeted checks (fast to run) • Baseline test: If you haven’t already, disable all noise cancellation on a single test call (no Krisp, no BVC/BVCTelephony) and see whether Gemini picks up the sentences. If it does—filters are the problem. • Raw audio inspect: Export the same segment that Gemini misses and open in Audacity. Check whether: ◦ Speech energy (peak/RMS) is visibly present. ◦ The frequency content is mainly low (narrowband) or full-band. ◦ If present but low amplitude, try applying a +6–+12 dB gain and re-run STT to see if detection improves. • VAD/endpointing: Temporarily increase any VAD or endpointing timeouts and lower energy thresholds so the model gets longer continuous audio to analyze. • Alternate STT check: If you can, pipe the same audio to another STT (or an offline transcription model) to confirm whether the issue is Gemini-specific. Possible root causes based on what you described • Aggressive pre-processing upstream (even if you don’t see BVC vs BVCTelephony differences) could still reduce signal clarity or remove transient phonemes. • If the audio is narrowband (8 kHz), some models lose intelligibility vs fullband audio. Gemini may perform worse on narrowband telephony audio for short utterances. • STT/VAD segmentation might be chopping the audio into tiny chunks or suppressing low-energy segments before the ASR sees full context. • Codec/transcoding or upstream SBC may be introducing artifacts that make speech harder for live models. If you give me answers to Q1–Q5 I’ll: • Recommend the exact configuration changes to test (e.g., switch sample rates, adjust Gemini live settings, VAD threshold values). • Provide step‑by‑step commands/code to disable filters, capture raw audio, and re-run Gemini on the same sample. • Help interpret a waveform or short sample you paste or screenshot. Which would you like to do first — run the all-filters-off baseline test, or share the Gemini model/settings and a short failing recording or waveform screenshot so I can diagnose more precisely?
2 Views
Open in Slack
PreviousNext
Powered by