why is livekit resampling audio to 24khz and how can i remove it

Thanks for reaching out, Sathwik. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.

we are using livekit agents 1.4.3 and livekit python is wht we are using , audio is coming from sip telephony twillio. sometimes it is 48hkz PCMA. sometimes its 8khz it depends. I observe 24khz in the audio sample rate logs i need to preserve the sample rate im getting

{"timestamp":"2026-03-11T13:16:24.519771Z","call_sid":"CA88d5cb2ea13d93e964a7ab1ad50e6dd8","level":"INFO","message":"Audio level: -10.09 dB, Sample rate: 24000 Hz","event_type":"AUDIO_LEVEL","metadata":{"db":-10.09373664855957,"frame":30,"rms":10250.9267578125,"sample_rate":24000}}

Thanks — this is helpful. I’m responding to your question about why LiveKit Agents 1.4.3 is showing 24 kHz in the logs when ingesting SIP audio from Twilio, and how to preserve the original sample rate. Short answer • LiveKit Agents normalizes incoming audio to a common internal sampling rate (24 kHz in your logs) during ingestion/processing. That explains why you see 24,000 Hz even though Twilio sends 8 kHz (PCMA) or 48 kHz audio. • There is no documented Agent configuration option to disable that internal resampling and force end-to-end passthrough of the original sample rate. What’s happening (brief) • Your SIP inputs arrive at variable rates (8 kHz for telephony G.711, 48 kHz for Opus). LiveKit Agents converts/normalizes those inputs to a single internal rate for consistent downstream processing (VAD, STT, recording pipelines), which is why the agent logs report 24 kHz. • Even if the original RTP stream uses 8 kHz or 48 kHz, the Agent’s processing pipeline can resample during decoding/encoding so components that expect a uniform rate work reliably. How you can investigate and verify 1. Confirm the published track’s original properties in your Python code or via events: ◦ Listen for track metadata or publication events on the participant that represents the SIP gateway and log any track audio properties you can access (sample rate or codec fields). ◦ Check the SIP/Twilio side to confirm the incoming codec/sample rate being negotiated (SIP logs or Twilio call details). 1. Check where you observe the resampled rate: ◦ If the 24 kHz value appears only in Agent logs, that indicates resampling inside the agent pipeline. ◦ If subscribers or recordings also show 24 kHz, the resampled stream is what’s being forwarded/recorded. Workarounds and options • If you only need the original sample rate for analysis/recording (not for real-time VAD/STT inside Agents), capture or record the raw RTP/audio at the ingest point (before it goes through the Agent pipeline) and store that externally. This ensures you keep 8 kHz or 48 kHz originals. • If you need native-rate audio in the LiveKit room (end-to-end passthrough), that capability is not exposed as a simple AgentSession/AudioInput setting in the current docs. You can: ◦ Route the SIP RTP to a separate processor that preserves sample rate and then publish that stream into the room with the desired parameters. ◦ Or handle recording/transcoding outside of the Agent so the Agent’s internal resampling does not affect your stored/original copy. Why resampling may be desirable (so you can weigh tradeoffs) • Normalizing to a single rate reduces complexity for downstream AI models (VAD/STT) and for mixing/recording, and avoids repeated resampling when multiple heterogeneous sources are combined. But it does mean the original rate is not preserved by default. Next steps I can help with • If you want, share a snippet of how your SIP source is wired into the agent (publication code or config) and where you see the 24 kHz value (agent logs vs. subscriber vs. recording). I’ll point to the exact properties/events to log and suggest the cleanest approach to preserve originals in your flow. • I can also outline a sample approach for capturing raw RTP or publishing a separately-preserved stream into the room so you retain 8 kHz / 48 kHz audio for your use case. Which do you want to do first — verify the published track properties from the agent, or implement an external capture/passthrough for the SIP RTP? Sources: Agent session | LiveKit Documentation | Silero VAD plugin | LiveKit Documentation | Voice AI quickstart | LiveKit Documentation

okay so now im passing sample rate as 48khz on deepgram shd i restrcutre it to 24khz to see the improvement ? and can explain in detail I can also outline a sample approach for capturing raw RTP or publishing a separately-preserved stream into the room so you retain 8 kHz / 48 kHz audio for your use case.

Thanks — this is clear and helpful. I’ll answer both parts: whether to pass 48 kHz vs 24 kHz to Deepgram, and a detailed approach to capture raw RTP or publish a preserved-rate stream so you can retain 8 kHz / 48 kHz audio. Part 1 — Deepgram: 48 kHz vs 24 kHz • What matters most: matching the audio sample rate you send to Deepgram to the sample rate of the audio actually being processed by your pipeline. If the audio arriving at Deepgram is 24 kHz (because LiveKit Agents resampled to 24 kHz), telling Deepgram the audio is 48 kHz can hurt recognition and processing quality. Conversely, if the audio reaching Deepgram truly is 48 kHz, you should send 48 kHz. • Recommendation: ◦ Verify the actual sample rate of the audio you send to Deepgram. If your Agent logs show 24 kHz and that same stream is what you forward to Deepgram, switch to 24 kHz when calling Deepgram. ◦ If you can preserve the original 48 kHz upstream and forward that, keep Deepgram at 48 kHz. ◦ If your incoming sources vary (8 kHz telephony and 48 kHz Opus), normalize what Deepgram receives: for telephony voice, 8 kHz or 16 kHz is typical; for wideband speech (48 kHz source downsampled to 24 kHz by Agents), use 24 kHz. • Practical test: run an A/B test — send identical segments labeled as 48 kHz vs 24 kHz (matching the actual stream) and compare recognition accuracy and latency. If Agents are resampling to 24 kHz, you’ll likely see improved results by passing 24 kHz. Part 2 — How to capture raw RTP or publish a preserved-rate stream (detailed approach) Goal: keep an unmodified copy of the incoming SIP audio (8 kHz or 48 kHz) so you can: • Send true-native-rate audio to ASR/analytics, or • Keep original-quality recordings separate from the Agent-processed stream. High-level options (choose one based on architecture and constraints) • Option A — Capture raw RTP at the SIP gateway (recommended if you control SIP ingress). • Option B — Duplicate the RTP stream before it enters Agent processing and publish the duplicate into the LiveKit room without Agent resampling. • Option C — Record or store raw audio at the SIP side (file or RTP dump) for later processing. Detailed steps for each option 1) Option A — Capture raw RTP at the SIP gateway 1.1 Setup • On the server that terminates the SIP call (your SIP gateway or the Twilio Media Stream endpoint), enable raw audio capture. This can be: ◦ An RTP dump (pcap) of the RTP stream, or ◦ A raw PCM/WAV write-out from the media processing layer (before any conversion). 1.2 Implementation notes • For G.711 (PCMA/PCMU at 8 kHz), capture the decoded PCM rather than relying on downstream resampling. • For Opus (48 kHz), capture decoded PCM at 48 kHz. 1.3 Forwarding to ASR • Stream the captured PCM to Deepgram (or another ASR) with the sample rate matching the captured PCM (8 kHz for G.711, 48 kHz for Opus). • If you need a real-time stream, implement a small producer that reads captured audio frames and forwards them via your Deepgram streaming client, ensuring the sample_rate in Deepgram metadata matches the frames. 2) Option B — Duplicate/passthrough stream into LiveKit without Agent processing 2.1 Goal • Publish a second track into the same LiveKit room that preserves the original sample rate, while letting Agents keep their normalized track for VAD/STT. 2.2 Approach • Create a separate RTP-to-WebRTC bridge or media forwarder that: ◦ Receives RTP from Twilio. ◦ Decodes or repacketizes as needed but preserves the native PCM sampling rate. ◦ Publishes that media as a new participant/track into LiveKit (using the Python SDK or a WebRTC publisher) with explicit encoding parameters that preserve sample rate where possible. 2.3 Implementation sketch • On the ingest server: ◦ Terminate Twilio SIP RTP. ◦ Decode to PCM (preserve 8 kHz or 48 kHz). ◦ Use a WebRTC client library (or LiveKit publisher) to create a publisher connection and publish an audio track built from the PCM capture. ◦ Ensure your publisher does not apply any resampling or processing; publish raw PCM frames matching their native rate. • Downstream consumers (recorders, analytics) subscribe to that preserved track instead of, or in addition to, the normalized Agent track. 3) Option C — Record/store raw audio at ingest for offline processing 3.1 Implementation • Write the decoded PCM into files named/segmented by call ID and timestamp. • Store with metadata (call SID, codec, original sample rate). • For ASR, either: ◦ Batch-send the file with its correct sample rate to Deepgram, or ◦ Stream it as if live by reading back frames to Deepgram’s streaming API. 3.2 When to use • Use when real-time ASR is not required or you only need perfect-quality transcriptions for post-call analysis. Practical considerations and tips • Metadata: Always tag preserved files/streams with call SID, codec, and original sample rate. • Sync: If you publish both normalized Agent tracks and preserved tracks, ensure subscribers know which track to use for STT vs. playback. • Bandwidth: Publishing two live tracks increases bandwidth/CPU; consider only publishing preserved stream when needed. • Format conversion: If you decode Opus to PCM at 48 kHz, avoid re-encoding to a lower sample rate before forwarding to your ASR unless required. • Deepgram config: When streaming to Deepgram, set the sample_rate parameter to the sample rate of the frames you send. If you send PCM frames at 24 kHz, use 24000; if 48 kHz, use 48000; if 8 kHz, use 8000. Example flow (recommended if you control ingest) 1. Twilio SIP -> your SIP termination server. 2. On termination server: ◦ Duplicate stream: one copy -> LiveKit Agent (normal pipeline), second copy -> PCM capture & publisher. 1. Publisher -> LiveKit room as participant "preserved-{callSid}" with preserved sample rate track. 2. Deepgram -> consume preserved track (streaming API) with matching sample_rate, or process recorded preserved files offline. Next steps I can help with • Give you a concrete code example in Python showing how to publish PCM frames into LiveKit as a second audio track (I can provide a sample using the LiveKit Python SDK and a small PCM frame producer). • Help craft the Deepgram streaming metadata payload you should use for 8k/24k/48k frames. • Walk through a specific RTP-capture setup for Twilio (if you tell me whether you terminate SIP on your infrastructure or rely fully on Twilio). Which of the next steps would you like: a Python publisher code sample for preserving/publishing PCM into LiveKit, or the Deepgram metadata snippet for streaming the exact sample rates you’ll send?

which library of livekit does this resample happen again ?

Thanks — I’m answering your question about which LiveKit component is doing the resampling to 24 kHz. • The resampling is happening inside LiveKit Agents — specifically the agent audio ingestion/processing pipeline (the Agent’s audio input/processing component). • In your setup (LiveKit Agents 1.4.3 ingesting SIP from Twilio), Agents normalize incoming audio to a common internal sample rate (24 kHz in your logs) during ingestion/processing. If you want, I can: • Point out which Agent-side events/properties to log to confirm where the resample occurs, or • Provide a short Python snippet showing how to capture the original track properties or publish a preserved-rate track as a workaround. Which would you prefer?

whichh file even though its a lib

I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!