### Answer Here are the main levers for reducing latency, starting with the biggest impact items: **Choose a faster LLM model**: Smaller models respond faster, while larger reasoning-oriented models are slower to first token. Latency also varies between providers for similarly-sized models [(1) ]() . **Shorten your prompt**: Every token adds to time-to-first-token on every turn. Keep system prompts tight, move rarely-needed detail to the knowledge base or tool descriptions, and turn off unnecessary Agent Handbook presets. Prompts beyond roughly 8k tokens become noticeably slower . **Enable fast tier**: This routes LLM calls through dedicated, high-priority infrastructure, reducing both average response time and call-to-call variance, though it costs more than the standard model rate . If those don't resolve it, diagnose by component using the per-call latency breakdown [(2) ]() : - **High LLM latency**: Check the Status Page for incidents; otherwise revisit model choice and prompt length [(1) ]() - **High ASR latency**: Switch to the speed-optimized transcription mode instead of accuracy-optimized (which waits ~200ms longer) , and turn off Boosted Keywords if enabled since it can add transcription latency - **High TTS latency**: Disable voice settings like emotion that can increase latency, and try a different voice provider if TTS dominates - **Choppy/unstable audio**: Usually network jitter or call-server load; run a ping test to api.retellai.com — round-trip times consistently above 300ms can cause spikes Also consider turning off unnecessary turtle-icon 🐢 features (aim to keep estimated latency under 1.5s) and geographic distance for international calls . To pinpoint the bottlene

ck for your specific agent, retrieve the latency breakdown via the Get Call API, which returns per-component metrics (e2e, asr, llm, tts, knowledge_base, s2s) each with p50/p90/p95/p99/min/max stats [(2) ]() :

python
from retell import Retell
client = Retell(api_key="YOUR_API_KEY")
call = client.call.retrieve("CALL_ID")
print(call.latency)

If none of this resolves the issue, message support with your call ID, the steps you've tried, and current latency measurements [(1) ]() .

​ To learn more, see the sources I considered: [[1] Troubleshoot high latency](https://docs.retellai.com/reliability/troubleshoot-latency) [[2] Check actual latency](https://docs.retellai.com/reliability/check-actual-latency) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

### Answer I don't have the ability to access your account, look up agent settings, or retrieve call data directly — I can only work from documentation and reference material. [(1) ]() To investigate your call's 3600ms end-to-end latency, here's how to get the breakdown yourself: You can retrieve the per-call latency breakdown using the Get Call API, which returns detailed metrics for each component (e2e, asr, llm, tts, knowledge_base, s2s) :

python
from retell import Retell
client = Retell(api_key="YOUR_API_KEY")
call = client.call.retrieve("CALL_ID")
print(call.latency)

Once you have that breakdown, compare each component against typical ranges: - **High LLM latency**: check the Status Page for incidents; otherwise revisit model choice and prompt length [(2) ]() - **High ASR latency**: switch to speed-optimized transcription mode if using accuracy mode (which adds ~200ms) , and disable Boosted Keywords if enabled since it adds transcription latency - **High TTS latency**: disable settings like emotion if enabled, and consider a different voice provider if TTS dominates If the sum of your asr, llm, and tts latencies is noticeably lower than your reported e2e latency, that gap could point to turn-taking pauses (the agent waiting to confirm the caller finished speaking) or other pipeline factors rather than the individual components themselves — this has been observed as a cause of unexpectedly high e2e latency relative to component latencies in past cases. [(3) ]() If you paste your specific call's latency object (the JSON with e2e/asr/llm/tts breakdown) here, I can help interpret which component is driving the 3600ms.

​ To learn more, see the sources I considered: [[1] Check actual latency](https://docs.retellai.com/reliability/check-actual-latency) [[2] Troubleshoot high latency](https://docs.retellai.com/reliability/troubleshoot-latency) [[3] High end to end latency in phone calls](https://discord.com/channels/1186424376300679209/1450956304288845855) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

de and turn-taking behavior the gap remains unexplained, message support with your call ID and this latency breakdown for a deeper look .

### Answer Yes, this breakdown is helpful. Looking at your numbers: - **ASR**: p50 250ms, max 276ms — normal range, not a concern [(1) ]() - **LLM**: p50 622ms, max 709ms — reasonable, not the main driver - **TTS**: p50 197ms, max 228ms — low, not the main driver - **E2E**: p50 3615ms, max 4704ms — this is far higher than ASR + LLM + TTS combined The gap between your e2e numbers and the sum of asr/llm/tts is the key issue. Since asr+llm+tts per turn only adds up to roughly 1000ms at the high end, but e2e is running 3600-4700ms, most of that extra time isn't being spent in transcription, LLM generation, or speech synthesis — it's happening somewhere else in the pipeline. This matches a pattern seen before where end-to-end latency runs much higher than the sum of the individual components: it was attributed to turn-taking pauses, where the agent waits to confirm the caller has actually finished speaking before responding. That "pause" time counts toward e2e latency but doesn't show up in asr, llm, or tts [(2) ]() . To narrow this down: - Check if you're on the accuracy-optimized transcription mode, which waits about 200ms longer than speed-optimized mode before finalizing what it heard [(3) ]() - Review whether your conversations involve unusual speech patterns (pauses, short repeated words, etc.) that could cause turn-taking to wait longer than expected, since that was the case in the referenced example [(2) ]() - If your calls are international, geographic distance between regions can add to e2e latency as well [(3) ]() If after checking transcription mo

​ To learn more, see the sources I considered: [[1] Check actual latency](https://docs.retellai.com/reliability/check-actual-latency) [[2] High end to end latency in phone calls](https://discord.com/channels/1186424376300679209/1450956304288845855) [[3] Troubleshoot high latency](https://docs.retellai.com/reliability/troubleshoot-latency) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

### Answer Both of those settings are likely contributing to your high e2e latency: Transcription Mode: Optimize for Accuracy — this mode is specifically designed to wait for a sentence to be fully finalized rather than cutting it off early, which adds delay before the transcript (and therefore the turn-taking/response cycle) is finalized [(1) ]() . Switching to the speed-optimized mode removes that added wait, at the cost of some accuracy on cut-off sentences . Denoising Mode: Remove noise + Background Speech — this is the most aggressive denoising option. It "removes both background noise and background speech" and explicitly "may distort the waveform and can reduce speech-to-text accuracy in some cases," plus it carries a $0.005/min surcharge for the added processing [(2) ]() . That extra processing step also adds pipeline overhead on every turn. The documentation states "For most use cases,

Remove noise

is the recommended default ," and recommends only using the aggressive mode when you specifically have loud background speech with a single caller close to the mic — testing accuracy on real calls before rolling it out . Given your e2e (p50 3615ms) is far above the sum of your asr/llm/tts (roughly 1000ms combined) , the gap points to time spent outside those three measured stages — and both accuracy-optimized transcription and the heaviest denoising mode add exactly this kind of extra wait/processing time before a turn is considered final [(1) ]() [(2) ]() . Suggested changes to test: - Switch Transcription Mode to speed-optimized instead of accuracy-optimized [(1) ]() - Switch

Denoising Mode down to the default

Remove noise

(unless you specifically have loud background speech with one dominant close-mic speaker) [(2) ]() After changing these, re-run the latency breakdown to see if e2e drops closer to your asr+llm+tts sum.

​ To learn more, see the sources I considered: [[1] Increase transcription accuracy](https://docs.retellai.com/reliability/wrong-transcript) [[2] Handle background speech & noise](https://docs.retellai.com/build/handle-background-noise) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

### Answer Great progress. Your p50 e2e dropped from 3615ms to 1152ms, a significant improvement [(1) ]() . Looking at the new breakdown: - **ASR**: p50 195.5ms, though there's variance up to 630ms max — this variability lines up with switching away from the accuracy-optimized mode, which otherwise adds about 200ms extra wait before finalizing transcripts [(2) ]() - **LLM**: p50 483.5ms, max 695ms — the dominant contributor to e2e now, which tracks with LLM model choice and prompt length being the biggest latency levers [(1) ]() - **TTS**: p50 183ms, max 233ms — low and consistent, not a concern - **E2E**: p50 1152ms, max 3004ms — much closer to the asr+llm+tts sum now, though there's still some gap at the higher percentiles (p90 2228ms, p99 2926ms) The remaining gap between e2e and the component sum at higher percentiles could still involve turn-taking pauses, similar to what was seen before . If you want to tighten this further, the next biggest levers to check are LLM model choice (smaller models respond faster) and prompt length (prompts beyond roughly 8k tokens become noticeably slower) , or enabling fast tier for more consistent response times .

​ To learn more, see the sources I considered: [[1] Troubleshoot high latency](https://docs.retellai.com/reliability/troubleshoot-latency) [[2] Increase transcription accuracy](https://docs.retellai.com/reliability/wrong-transcript) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

Thanks for the follow-up, @Ciaran. I'll get back to you soon.

​ To learn more, see the sources I considered: [[1] Troubleshoot high latency](https://docs.retellai.com/reliability/troubleshoot-latency) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

### Answer This is a solid improvement — p50 e2e dropped to 1024ms with fast tier enabled on GPT-4.1 [(1) ]() [(2) ]() . Looking at the breakdown: - **ASR**: p50 81ms — very low, not a concern [(1) ]() - **LLM**: p50 661ms, max 989ms — still the dominant contributor to e2e, though fast tier is meant to reduce both average response time and call-to-call variance [(3) ]() [(2) ]() - **TTS**: p50 197ms, with one outlier at 595ms max — mostly low, not a major concern [(1) ]() - **E2E**: p50 1024ms — now tracking reasonably close to the LLM latency plus ASR/TTS overhead Since fast tier already routes your LLM calls through dedicated, high-priority infrastructure for more consistent performance [(3) ]() , and your LLM p50 (661ms) is still the largest single component, further latency gains at this point would mainly come from prompt length reduction — every token adds to time-to-first-token on every turn, and prompts beyond roughly 8k tokens become noticeably slower [(2) ]() . Since you're already prioritizing cost, note that fast tier costs more than the standard model rate, so this latency improvement does come with that tradeoff you'll want to weigh against your use case [(2) ]() [(3) ]() . If 1024ms p50 already meets your needs, I'd hold here rather than push further, since further reductions would mean either switching to a smaller model

(which changes agent behavior and needs re-testing) or accepting the fast tier cost premium you already have in place [(2) ]() .

​ To learn more, see the sources I considered: [[1] Check actual latency](https://docs.retellai.com/reliability/check-actual-latency) [[2] Troubleshoot high latency](https://docs.retellai.com/reliability/troubleshoot-latency) [[3] Configure LLM Options](https://docs.retellai.com/build/llm-options) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.

### Answer Good question, and the sources actually give two different but related answers here. Each call is indeed a fresh start in the sense that the LLM itself isn't being retrained live per-call — your agent's behavior comes from the prompt, tools, and configuration you've set up (like the

general_prompt

,

states

, and model settings you've already configured on your Retell LLM) [(1) ]() [(2) ]() . There's no persistent memory carried between separate calls unless you explicitly pass context via dynamic variables or your own backend. That said, "training" in the sources refers to a broader two-step process, not per-call learning: - **Fine-tuning with call transcripts**: this is a separate, deliberate step where a model is fine-tuned on real conversation transcripts to improve its understanding of your specific domain and customer queries [(3) ]() [(4) ]() . This isn't something that happens automatically call-to-call — it's a distinct process you'd run to adapt the underlying model. - **Prompt engineering**: this is the main lever for controlling agent behavior without retraining, letting the AI adapt dynamically while staying consistent and on-brand . This is what you've been doing throughout this thread — adjusting settings and prompts to change how the agent responds. So to directly answer: yes, each call starts fresh from the LLM's perspective (no memory carry-over by default) , but the consistency and domain knowledge you see across calls comes from your prompt engineering and, optionally, fine-tuning on transcripts done ahead of time — not from the model learning during live calls .

​ To learn more, see the sources I considered: [[1] Update Retell LLM](https://docs.retellai.com/api-references/update-retell-llm) [[2] Get Retell LLM](https://docs.retellai.com/api-references/get-retell-llm) [[3] Training & Customizing AI Voice Agents | Retell AI](https://www.retellai.com/blog/training-and-customizing-voice-agents-with-retell-ai) [[4] How to Deploy Conversational AI | Retell AI](https://www.retellai.com/blog/how-to-deploy-conversational-ai) ​

Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.