Is it only me or is ElevenLabs TTS throwing an err...
# agents
n
Is it only me or is ElevenLabs TTS throwing an error like:
Copy code
Traceback (most recent call last):
  File "/Users/xx/Documents/livekitonly/agents/livekit-agents/livekit/agents/tts/tts.py", line 407, in _main_task
    raise APIError(f"no audio frames were pushed for text: {self._pushed_text}")
livekit.agents._exceptions.APIError: no audio frames were pushed for text: Hello (body=None, retryable=True)
l
I thought those kind of errors usually come from some error in the agent code. I don't see any any issues on ElevenLabs and their status page shows green: https://status.elevenlabs.io/
n
I am getting that and this error continuously:
Copy code
WARNI… livekit.plugins… websocket closed unexpectedly  
             WARNI… livekit.agents   failed to synthesize speech, retrying in   {"tts":                                     
                                     0.1s                                       "livekit.plugins.elevenlabs.tts.TTS",       
                                     Traceback (most recent call last):         "attempt": 1, "streamed": true}             
                                       File                                                                                 
                                     "/Users/dhruvladia/Documents/sarvamagain/p                                             
                                     ython-agents-examples/venv/lib/python3.11/                                             
                                     site-packages/livekit/agents/tts/tts.py",                                              
                                     line 383, in _main_task                                                                
                                         await self._run(output_emitter)                                                    
                                       File                                                                                 
                                     "/Users/dhruvladia/Documents/sarvamagain/p                                             
                                     ython-agents-examples/venv/lib/python3.11/                                             
                                     site-packages/livekit/plugins/elevenlabs/t                                             
                                     ts.py", line 440, in _run                                                              
                                         raise e                                                                            
                                       File                                                                                 
                                     "/Users/dhruvladia/Documents/sarvamagain/p                                             
                                     ython-agents-examples/venv/lib/python3.11/                                             
                                     site-packages/livekit/plugins/elevenlabs/t                                             
                                     ts.py", line 435, in _run                                                              
                                         await waiter                                                                       
                                     livekit.agents._exceptions.APIStatusError:                                             
                                     connection closed (status_code=-1,                                                     
                                     request_id=None, body=None,                                                            
                                     retryable=True)
Even in the python-agents-examples agents. For example in: pipeline-tts/elevenlabs_tts.py
l
I see what you mean with the
pipeline-tts/elevenlabs_tts.py
. If I remove the
encoding="pcm_44100"
(just delete this line: https://github.com/livekit-examples/python-agents-examples/blob/main/pipeline-tts/elevenlabs_tts.py#L30 ) it works - can you try that? I'm still trying to understand what is causing this though
n
After removing that line, I was receiving this error:
Copy code
failed to synthesize speech, retrying in   {"tts":                                     
                                     2.0s                                       "livekit.plugins.elevenlabs.tts.TTS",       
                                     Traceback (most recent call last):         "attempt": 3, "streamed": true}             
                                       File                                                                                 
                                     "/Users/dhruvladia/Documents/sarvamagain/p                                             
                                     ython-agents-examples/venv/lib/python3.11/                                             
                                     site-packages/livekit/agents/tts/tts.py",                                              
                                     line 394, in _main_task                                                                
                                         raise APIError(f"no audio frames were                                              
                                     pushed for text: {self._pushed_text}")                                                 
                                     livekit.agents._exceptions.APIError: no                                                
                                     audio frames were pushed for text: Hello!                                              
                                     How can I assist you today? (body=None,                                                
                                     retryable=True)
l
That's the same error as before, no? So, removing the line had no effect for you? It's fairly consistent for me that removing that line solves the issue. I do notice we're on a different version of livekit-agents, I'm on 1.3.6. Alternatively, inference works: https://docs.livekit.io/agents/models/tts/inference/elevenlabs/
n
I updated to 1.3.6 but still seeing this error:
Copy code
(venv) dhruvladia@Dhruvs-MacBook-Pro python-agents-examples % python pipeline-tts/elevenlabs_tts.py console
    Agents   Starting console mode 🚀
              
    16:38:47.025 DEBUG  asyncio            Using selector: KqueueSelector  
    16:38:47.026 INFO   livekit.agents     starting worker {"version": "1.3.6", "rtc-version": "1.0.19"}
    16:38:47.028 INFO   livekit.agents     HTTP server listening on :56308  
    16:38:47.035 INFO   livekit.agents     initializing job runner {"tid": 33744691}
    16:38:47.036 DEBUG  asyncio            Using selector: KqueueSelector  
    16:38:47.037 INFO   livekit.agents     job runner initialized {"tid": 33744691, "elapsed_time": 0.0}
    16:38:47.181 DEBUG  livekit.agents     http_session(): creating a new httpclient ctx  
    16:38:47.183 DEBUG  livekit.agents     using audio io: `Console` -> `AgentSession` -> `Console`  
                 WARNI… livekit.agents     resume_false_interruption is enabled but audio output does not support pause,    
                                           it will be ignored                                                               
                                         {"audio_output": "Console"}
    16:38:47.184 DEBUG  livekit.agents     using transcript io: `AgentSession` -> (none)  
    16:38:47.806 WARNI… livekit.agents     failed to synthesize speech, retrying in 0.1s  
                                         {"tts": "livekit.plugins.elevenlabs.tts.TTS", "attempt": 1, "streamed": true}
Traceback (most recent call last):
  File 
"/Users/dhruvladia/Documents/sarvamagain/python-agents-examples/venv/lib/python3.11/site-packages/livekit/agents/tts/tts.py"
, line 407, in _main_task
    raise APIError(f"no audio frames were pushed for text: {self._pushed_text}")
livekit.agents._exceptions.APIError: no audio frames were pushed for text: Hi there! Is there anything I can help you with? 
(body=None, retryable=True)
l
Or, from https://docs.livekit.io/agents/models/tts/inference/elevenlabs/, you can replace the TTS call with inference:
tts="elevenlabs/eleven_turbo_v2_5:Xb7hH8MSUJpSbSDYk0k2",
n
Inferencing is working but why not without it?
l
I'm not sure, my working theory is that
encoding="pcm_44100"
is not a valid value for Python - if I provide something like
mp3_44100_32
it worked for me also.
m
the error is because the credit limit of your elevenlab api key is completed change the elevenlab api key and check once also make sure you give correct naming ELEVEN_API_KEY="********" like this
r
it looks like streaming TTS retries are broken. The retry loop reuses the same SynthesizeStream instance, but the _input_ch channel and plugin state are one-shot resources that get consumed during the first attempt. As a result, retrying the same request sends no text or produces no segments, leading to consistent failures like APIError: no audio frames were pushed or segments mismatch. I think this PR should fix that : https://github.com/livekit/agents/pull/4410