https://retellai.com/?utm_source=discord&utm_medium=seo logo
when using cantonese language, we found when the u...
# ❓・ai-evy-support
u
@AI Evy when using cantonese language, we found when the user speech is long, the transcription seems cut off and agent would interrupt user's speech, is there a way to fix?
a
Thanks for tagging me @ꍟ꒒꒒ꍟꈤ. I'll look through relevant sources and get back to you shortly.
### Answer When using Cantonese and experiencing transcription cut-offs with agent interruptions during long user speech, you can adjust the interruption_sensitivity parameter to make the agent less sensitive to interruptions [(1) ]() . This controls how easily users can interrupt the agent, and lowering this value will require longer or more words for the user to interrupt [(2) ]() . You can also adjust the responsiveness parameter, where a lower value means the agent waits more and responds slower, giving users more time to complete their speech [(1) ]() [(2) ]() . Retell AI uses *Voice Activity Detection (VAD) * and turn-taking models to manage conversation flow [(3) ]() [(4) ]() . The turn-taking model combines sound signals with semantic understanding to determine whether a user is pausing or finished speaking [(4) ]() . However, there are known issues with ASR (Automatic Speech Recognition) for non-English languages [(5) ]() [(6) ]() . Based on community discussions, there have been concerns about ASR performance for non-English languages, with users reporting that Deepgram (the current ASR provider) has limitations for languages other than English [(5) ]() . The team has acknowledged working on ASR improvements [(6) ](https
Copy code
json
{
  "response_type": "update_agent",
  "agent_config": {
    "responsiveness": 0.3,
    "interruption_sensitivity": 0.2
  }
}
Try lowering both interruption_sensitivity and responsiveness values to give Cantonese speakers more time to complete their thoughts without being interrupted.
://discord.com/channels/1186424376300679209/1217142861737492510) . To adjust these settings, you can use the update_agent event in the LLM WebSocket to modify agent configuration during the call [(2) ]() :
​ To learn more, see the sources I considered: [[1] Platform Changelogs | Retell AI](https://www.retellai.com/changelog) [[2] LLM WebSocket](https://docs.retellai.com/api-references/llm-websocket) [[3] Speech Processing](https://www.retellai.com/glossary/speech-processing) [[4] VAD vs. Turn-Taking Endpoints in Conversational AI | Retell AI](https://www.retellai.com/blog/vad-vs-turn-taking-end-point-in-conversational-ai) [[5] Discord message from @emanuelkr on 2025-03-21](https://discord.com/channels/1186424376300679209/1186462578310910112/1352556418187661354) [[6] ASR errors](https://discord.com/channels/1186424376300679209/1217142861737492510) ​
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.
u
@AI Evy but there is barely a pause and it still cut off, is it due to Azure transcription context size limit or anything? because i want agent listen to what candidate spoke without any ratio of cut off candidate
a
Thanks for the follow-up, @ꍟ꒒꒒ꍟꈤ. I'll get back to you soon.
### Answer Based on the available documentation, you can adjust the interruption_sensitivity parameter to prevent the agent from cutting off the user [(1) ]() . Setting it to 0 will ensure the agent never gets interrupted . You can configure this in your agent settings:
Copy code
json
{
  "interruption_sensitivity": 0
}
Additionally, you can lower the responsiveness parameter to make the agent wait longer before responding [(1) ]() [(2) ]() :
Copy code
json
{
  "responsiveness": 0
}
For conversation flow agents, these settings are available in the global settings under "Speech Settings" [(3) ]() . You can adjust interruption sensitivity and responsiveness there . If you're using the LLM WebSocket, you can update these settings during a call using the update_agent event [(4) ]() :
Copy code
json
{
  "response_type": "update_agent",
  "agent_config": {
    "responsiveness": 0,
    "interruption_sensitivity": 0
  }
}
The documentation doesn't mention Azure transcription context size limits or specific ASR provider limitations for Cantonese. The interruption behavior is controlled by Retell's turn-taking model and these sensitivity parameters.
​ To learn more, see the sources I considered: [[1] Get Voice Agent](https://docs.retellai.com/api-references/get-agent) [[2] Get Agent Versions](https://docs.retellai.com/api-references/get-agent-versions) [[3] Step 1: Configure global settings](https://docs.retellai.com/build/conversation-flow/global-setting) [[4] LLM WebSocket](https://docs.retellai.com/api-references/llm-websocket) ​
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.
u
@AI Evy interruption_sensitivity should only relate to how often candidate can interrupt agent and not related in this case, and responsiveness 0 i afraid the agent is completely not responding or super slow
a
Thanks for the follow-up, @ꍟ꒒꒒ꍟꈤ. I'll get back to you soon.
### Answer You're correct about **interruption_sensitivity**—it controls how easily the user can interrupt the agent, not how the agent interrupts the user [(1) ]() [(2) ]() . For your specific issue where the agent is cutting off long Cantonese user speech, the responsiveness parameter is the key setting . However, you're right to be concerned about setting it to 0. The responsiveness parameter controls how responsive the agent is, with values ranging from [0,1] . A lower value means the agent waits more and responds slower, while a higher value means faster exchanges . Instead of setting responsiveness to 0, try a moderate value like 0.3-0.5 to give the user more time to complete their speech without making the agent completely unresponsive . Additionally, you can adjust the reminder_trigger_ms parameter, which triggers a reminder to the agent to speak if the user has been silent for the specified duration after some agent speech . The default is 10000 ms (10 seconds) . Increasing this value would give users more time before the agent attempts to speak again . You can update these settings using the LLM WebSocket update_agent event during a call [(3) ]() :
Copy code
json
{
  "response_type": "update_agent",
  "agent_config": {
    "responsiveness": 0.4,
    "reminder_trigger_ms": 15000
  }
}
The documentation doesn't specifically mention ASR context size limits or Azure-specific transcription constraints that would cause mid-speech cutoffs for Cantonese [(1) ]() [(4) ]() [(3) ]() [(2) ]() .
​ To learn more, see the sources I considered: [[1] Get Voice Agent](https://docs.retellai.com/api-references/get-agent) [[2] Update Voice Agent](https://docs.retellai.com/api-references/update-agent) [[3] LLM WebSocket](https://docs.retellai.com/api-references/llm-websocket) [[4] Get Agent Versions](https://docs.retellai.com/api-references/get-agent-versions) ​
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.
u
@AI Evy again as mentioned, there is no pause at all, so the reminder does not help, also i don't want there is a percentage of user speech too long would get cut off but keep listening to full speech
a
Thanks for the follow-up, @ꍟ꒒꒒ꍟꈤ. I'll get back to you soon.
### Answer Based on the available documentation, if the agent is cutting off long Cantonese speech without pauses, you may be experiencing issues with the denoising_mode setting. The "Remove noise + background speech" mode can sometimes drop or clip transcription for certain audio patterns [(1) ]() . Try setting denoising_mode to
"noise-cancellation"
(the standard "Remove noise" mode) instead of the more aggressive background speech removal [(2) ]() [(3) ]() :
Copy code
json
{
  "denoising_mode": "noise-cancellation"
}
[(2) ]() , [(3) ]() Additionally, you can configure custom_stt_config with a higher endpointing_ms value to allow longer speech before the system considers the user finished speaking [(2) ]() [(3) ]() :
Copy code
json
{
  "custom_stt_config": {
    "endpointing_ms": 2000
  }
}
[(2) ]() , [(3) ]() The default endpointing determines when the speech-to-text system considers an utterance complete. Increasing this value gives the system more tolerance for natural pauses within continuous speech [(2) ]() [(3) ]() . You can also set stt_mode to
"accurate"
instead of
"fast"
for better transcription quality, which may help with longer Cantonese utterances :
Copy code
json
{
  "stt_mode": "accurate"
}
[(2) ]() , [(3) ]() The documentation doesn't specify ASR context size limits or Azure-specific constraints that would cause mid-speech cutoffs for long utterances [(4) ]() [(2) ]() [(3) ]() .
​ To learn more, see the sources I considered: [[1] Handle background speech & noise](https://docs.retellai.com/build/handle-background-noise) [[2] Get Voice Agent](https://docs.retellai.com/api-references/get-agent) [[3] Get Agent Versions](https://docs.retellai.com/api-references/get-agent-versions) [[4] LLM WebSocket](https://docs.retellai.com/api-references/llm-websocket) ​
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.
u
@AI Evy I want to tweak cantonese jyutping pronunciation as it is supported in minimax voice, but how do i tweak in retell?
a
Thanks for the follow-up, @ꍟ꒒꒒ꍟꈤ. I'll get back to you soon.
### Answer To tweak Cantonese Jyutping pronunciation in Retell AI, you can use the pronunciation_dictionary parameter [(1) ]() [(2) ]() . This allows you to specify custom pronunciations for specific words using either IPA (International Phonetic Alphabet) or other phonetic alphabets . Here's how to configure it when creating or updating an agent :
Copy code
json
{
  "pronunciation_dictionary": [
    {
      "word": "actually",
      "alphabet": "ipa",
      "phoneme": "ˈæktʃuəli"
    }
  ]
}
[(1) ]() , [(2) ]() For Cantonese Jyutping with Minimax voice, you would replace the example with your specific Cantonese words and their Jyutping pronunciations [(1) ]() [(2) ]() . The
alphabet
field specifies the phonetic system you're using, and
phoneme
contains the actual pronunciation . You can add this to your agent configuration using the Create Voice Agent API [(1) ]() or Update Voice Agent API [(2) ]() . The pronunciation dictionary accepts an array, so you can define multiple word pronunciations [(1) ]() [(2) ]() .
​ To learn more, see the sources I considered: [[1] Create Voice Agent](https://docs.retellai.com/api-references/create-agent) [[2] Update Voice Agent](https://docs.retellai.com/api-references/update-agent) ​
Feel free to tag @AI Evy with additional questions. I also appreciate feedback, it helps me improve.