https://livekit.io logo
Join Slack
Powered by
Hi everyone! We are currently using inference mode...
# inference
b
Hi everyone! We are currently using inference model to iterate with different stt/llms/tts models, and we are on ship plan. As I was testing my application, from nowhere I got the voice agent saying "_you have hit your daily limit for advance voice"_. Now when I start my server, I'm getting logs that indicates that it cannot connect to the stt model. Even when I change the model we're using, it just gives the same error and the room closes own it's own. In the docs, I couldn't see where API limit is given for each model, I thought by choosing the ship plan we'd be able to ignore the limiting cases. Can anyone nudge me in the right direction, as to where I can see the daily limit for advance voices is given, and how to resolve this.
r
Are you using LiveKit gateway or using your own API key for model provider? I assume your own API key. That voice is not from LiveKit it is from the provider.
b
I am currently using the inference model, so no, I'm not using my own API keys for model provider. Also, when I was using my app in mobile browser it was working, but when I was using in Desktop, it wasn't. I am assuming that maybe my device IP got blocked or something for hitting some limit.
r
What is your project ID?
@fancy-lifeguard-70250 or @rough-gpu-50664 Do you know what may cause this?
b
approximately after 6 hours, it got fixed automatically!
🙏 1
Copy code
project_id: p_3c3ierzodcg
f
Thank you, we'll dig into this.
b
Thank you. Please let me know if you guys figure out anything. Also if it may help, I was constantly changing the voice models, tts, stt, llm. We have like a config panel where we can change these on fly. And after constantly changing these, I got into this situation.
f
Ok, thanks. That's helpful. I'm digging through the logs now. Do you happen to know which voice provider triggered "_you have hit your daily limit for advance voice"?_ Two more things: 1. we don't officially support custom voices...yet (more soon) 2. What version and language (JS vs python) of the framework are you using?
b
I was using gpt4.1 mini answer to your points 1. yes, we just made a seed data of the current models you support with voices and were within that scope only. so no custom voices as of now. 2. 3.13 python Also, to get out of this, I also tried using different project altogether, but still it didn't work. basically agent was starting and closing the session automatically without any reason.
✅ 1
f
What version the LiveKit python SDK are you using?
b
Copy code
livekit                                  1.0.17
livekit-agents                           1.2.15
livekit-api                              1.0.7
livekit-blingfire                        1.0.0
livekit-plugins-deepgram                 1.2.15
livekit-plugins-elevenlabs               1.2.15
livekit-plugins-openai                   1.2.15
livekit-plugins-silero                   1.2.15
livekit-plugins-turn-detector            1.2.15
livekit-protocol                         1.0.8
🙏 1
f
Just to clarify, are you using 
gpt-4.1-mini
for voice? Sorry for all the questions 😬
nm, I see you clarified that in your first bullet point.
Can anyone nudge me in the right direction, as to where I can see the daily limit for advance voices is given
Can you clarify what you mean by 
advance voices
?
b
so my llm model was 
gpt-4.1-mini
, my stt models was 
deepgram/nova-2-conversationalai
and my tts model was 
elevenlabs/eleven_turbo_v2_5
✅ 1
f
Regarding your STT connection issue, we're seeing timeouts from deepgram in our logs and they seem to have had issues yesterday. https://status.deepgram.com/
b
Hey @fancy-lifeguard-70250 @refined-appointment-81829, sorry to do this again, but we are experiencing this error in our production servers and we are using inference and not using any API keys by ourselves. Below is the error we are getting, I thought it could have been a validity issue with the API keys and secrets and created new ones, but still the same errors.
Copy code
File "/opt/venv/lib/python3.12/site-packages/livekit/agents/llm/llm.py", line 202, in _main_task
    raise APIConnectionError(
livekit.agents._exceptions.APIConnectionError: failed to generate LLM completion after 4 attempts (body=None, retryable=True)
ERROR:livekit.agents:Error in _llm_inference_task
    return await func(*args, **kwargs)  # type: ignore
Traceback (most recent call last):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/livekit/agents/llm/llm.py", line 190, in _main_task
    return await self._run()
  File "/opt/venv/lib/python3.12/site-packages/livekit/agents/voice/generation.py", line 125, in _llm_inference_task
    raise APIStatusError(
           ^^^^^^^^^^^^^^^^^
    async for chunk in llm_node:
  File "/opt/venv/lib/python3.12/site-packages/livekit/agents/inference/llm.py", line 341, in _run
livekit.agents._exceptions.APIStatusError: invalid API key for domain agent-gateway.livekit.cloud (status_code=401, request_id=None, body=invalid API key for domain agent-gateway.livekit.cloud, retryable=True)
The above exception was the direct cause of the following exception:
r
We're looking into it.
b
thanks! please let us know when you figure it out.
f
@bulky-pager-62731 thanks for your patience, we resolved the 
401
issue. There was a hiccup in our DNS and some requests were getting routed where Inference wasn't available.
🙏 1
b
Hey guys, does the inference TTS not support speed variability for models other than cartesia?
f
Which models specifically are you interested in? We've adding parameter support across TTS and STT models, if we're missing something we can add it quickly.
b
so in tts inference, livekit only has support for elevenlabs, rime and inworld and cartesia. all of these mentioned above does not support the speed parameter as of now other than cartesia. also if we go into the implementation classes of these, we see only cartesia supports the speed parameter. @fancy-lifeguard-70250
f
Yeah, unfortunately we're treating the speed setting as a provider specific parameter so you'll have to pass it in via the 
extra_kwargs
argument. Kind of a hassle, we need to improve this, it's common enough that we should just add it to the native inference API. So for Cartesia, the param name is 
speed
For Inworld, it's 
speakingRate
https://docs.inworld.ai/api-reference/ttsAPI/texttospeech/synthesize-speech-stream#body-audio-config-speaking-rate Rime has a speed setting as well, 
speedAlpha
https://docs.rime.ai/api-reference/arcana/streaming-pcm#param-speed-alpha ElevenLabs has a speed setting too, 
speed
https://elevenlabs.io/docs/api-reference/text-to-speech/v-1-text-to-speech-voice-id-stream-input#send.initializeConnection.voice_settings.speed So, something like this:
Copy code
tts=inference.TTS(
  model="elevenlabs/eleven_flash_v2_5",
  voice="cgSgspJ2msm6clMCkdW9",
  extra_kwargs={
    "speed": <value>,
  },
)
b
yes, that was my first step to go into each docs and implement the speed. but the speed only worked for cartesia, even though i was using tts model specific params. so you might want to take a look into that too. thanks!
f
Thanks for the heads up. Over the last week we've been rolling out an update that improves provider-specific support. I'll go back and double check the speed param on all the providers.
🙌 1
b
Hey @fancy-lifeguard-70250, we were just deploying our agent, and saw that our deployment get failed at this point.
Copy code
> [ 9/10] RUN python "agent.py" download-files:
0.738     from ..job import JobExecutorType
0.738   File "/usr/local/lib/python3.13/site-packages/livekit/agents/job.py", line 41, in <module>
0.738     from .telemetry import _upload_session_report, trace_types, tracer
0.738   File "/usr/local/lib/python3.13/site-packages/livekit/agents/telemetry/__init__.py", line 2, in <module>
0.738     from .traces import _setup_cloud_tracer, _upload_session_report, set_tracer_provider, tracer
0.738   File "/usr/local/lib/python3.13/site-packages/livekit/agents/telemetry/traces.py", line 18, in <module>
0.738     from opentelemetry.sdk._logs import (
0.738     ...<5 lines>...
0.738     )
0.738 ImportError: cannot import name 'LogData' from 'opentelemetry.sdk._logs' (/usr/local/lib/python3.13/site-packages/opentelemetry/sdk/_logs/__init__.py)
------
unable to deploy agent: build failed for <http://iad.ocir.io/axyci3pr8vxm/production-cloud-agents:p-3c3ierzodcg-ca-n7eyc3tuzrbe-v20251203173454|iad.ocir.io/axyci3pr8vxm/production-cloud-agents:p-3c3ierzodcg-ca-n7eyc3tuzrbe-v20251203173454>: failed to solve: process "/bin/sh -c python \"agent.py\" download-files" did not complete successfully: exit code: 1
We were getting some deprecation logs before, but now as https://github.com/open-telemetry/opentelemetry-python/releases has released the 1.39.0 version, now it breaks. So for now, we have explicitly set opentelemetry version to 1.38 so it doesn't break. and our livekit-agent version is 1.3.5
f
@bulky-pager-62731 Thanks for the heads up! I'll let the team know
c
HI I got the same issue as Shashwat how did you solve it? cc @fancy-lifeguard-70250
b
Hey @cuddly-cartoon-47334 , just explicitly set open telemetry version as 1.38
c
in requirements.txt? I did nto import that library before.. it seems livekit automatically imports it
b
Yes, in your requirements.txt file Yes live kit does that automatically
c
thank you, it worked. Not sure why that randomly happened
b
Well because open telemetry released its new version 1.39.0 and live kit has to update their internal open telemetry version to update some deprecated method they would be using, so that's why it started breaking. You might seeing some logs too something like Logrecord will be deprecated in 1.39 something like that.
f
This PR addresses the issue, it was merged yesterday and released with 1.3.6, also released yesterday https://github.com/livekit/agents/pull/4162
👍 1
Open in Slack
PreviousNext
Powered by