I’m debugging `livekit.plugins.xai.tts.TTS` in Pyt...
# ask-ai
g
I’m debugging
livekit.plugins.xai.tts.TTS
in Python Agents and need clarification on the intended xAI TTS transport. I see that LiveKit’s xAI TTS plugin source uses:
<wss://api.x.ai/v1/realtime/audio/speech>
But xAI’s current public TTS docs say the streaming WebSocket endpoint is:
<wss://api.x.ai/v1/tts>
and document a
text.delta
/
text.done
protocol. Questions: 1. Is
livekit.plugins.xai.tts.TTS
intentionally supposed to use
<wss://api.x.ai/v1/realtime/audio/speech>
instead of
<wss://api.x.ai/v1/tts>
? 2. Is that endpoint publicly supported by xAI for TTS today? 3. If yes, what is the official source/doc for that endpoint and protocol? 4. If no, is the current LiveKit xAI TTS plugin incorrect/regressed? 5. We are seeing
status_code=403
followed by
no audio frames were pushed for text
from
livekit.plugins.xai.tts.TTS
while unary
POST <https://api.x.ai/v1/tts>
works with the same API key. Does that match a known issue?
t
Thanks for reaching out, Vrishabh. I'm an AI assistant and I'm here to help. Give me a moment while I research this topic.
Thanks — I’ve reviewed your questions about
livekit.plugins.xai.tts.TTS
and the xAI TTS transport. I don’t have any source material that confirms the plugin’s internal WebSocket URL, the public status of
wss://api.x.ai/v1/realtime/audio/speech
, or any official xAI docs for that endpoint/protocol. Because of that, I can’t definitively confirm whether the plugin is intentionally using that URL or whether it’s correct. What I can do to help you debug and narrow this down: 1. Quick clarifying questions ◦ Which LiveKit Agents / package version are you running (exact package version or commit)? Different versions may ship different plugin code. ◦ Can you share the exact plugin configuration you’re using (redact any secrets)? For example, how the
xai
provider/credentials are configured and how
TTS
is invoked. ◦ Are you seeing the
403
from the WebSocket connect attempt or from a later request over the socket? If you have raw logs (timestamps, request/response headers) that show the failing handshake or messages, that will help. 1. Immediate troubleshooting steps you can try ◦ Confirm API key scope and allowed endpoints: The unary
POST https://api.x.ai/v1/tts
works with your key, which shows the key is valid for REST TTS. Streaming endpoints sometimes require additional scopes or a different auth header format. Check the key’s scopes or try a fresh key with full TTS permissions if possible. ◦ Test the WebSocket endpoint independently (curl/websocket client) using the same API key and headers your plugin sends to see if the 403 reproduces outside LiveKit. This isolates whether the issue is client code vs. key/endpoint. ◦ Capture the WebSocket handshake and any server response body. A 403 sometimes includes a short JSON error that explains the reason (quota, invalid scope, origin, or mis-formatted Authorization). ◦ Verify TLS/server hostname and any proxy: ensure nothing on your network rewrites the request or strips Authorization headers for ws/wss traffic. 1. If the endpoint is indeed different from the public docs ◦ If
wss://api.x.ai/v1/tts
(with
text.delta
/
text.done
) is the documented streaming protocol you expect, but the plugin uses
wss://api.x.ai/v1/realtime/audio/speech
, that could mean: ▪︎ The plugin targets a different (internal or older) realtime API, or ▪︎ The plugin needs updating to match the public xAI streaming API. ◦ I cannot confirm either scenario without the plugin code version or an official source for that
realtime/audio/speech
URL. 1. Next steps I can take for you ◦ If you share the LiveKit Agents version and the plugin config/log snippets (redact keys), I can look at the shipped plugin code paths and point to where the endpoint is chosen and what auth/headers it sends. ◦ I can provide a step-by-step repro checklist for testing the WebSocket endpoint outside LiveKit so you can confirm whether the endpoint or the plugin is at fault. Summary • I cannot confirm whether
wss://api.x.ai/v1/realtime/audio/speech
is intended or publicly supported because I don’t have documentation or source confirming it. • The 403 + “no audio frames were pushed for text” behavior is consistent with an auth/endpoint mismatch, but I don’t have a documented known-issue that exactly matches your symptoms. • Please share your LiveKit Agents version and sanitized plugin logs/config and I’ll trace the plugin’s transport and auth details and give concrete next steps. Would you like to paste the version and a redacted log snippet now? Sources: xAI Grok Voice Agent API plugin | LiveKit Documentation | xAI LLM plugin guide | LiveKit Documentation | xAI and LiveKit | LiveKit Documentation, with interactive elements
g
I can answer part of this from the LiveKit source itself. In the current xAI TTS plugin source, LiveKit sets:
XAI_WEBSOCKET_URL = "<wss://api.x.ai/v1/realtime/audio/speech>"
File: https://github.com/livekit/agents/blob/main/livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/tts.py Meanwhile xAI’s public TTS docs currently document the streaming endpoint as:
<wss://api.x.ai/v1/tts>
Docs: https://docs.x.ai/developers/model-capabilities/audio/text-to-speech So my question is narrower: 1. Is LiveKit intentionally targeting
<wss://api.x.ai/v1/realtime/audio/speech>
for xAI TTS? 2. If yes, is that endpoint officially supported by xAI today, and where is it documented? 3. If no, should the plugin be updated to
<wss://api.x.ai/v1/tts>
and the documented streaming protocol? 4. We are seeing
403
from
livekit.plugins.xai.tts.TTS
followed by
no audio frames were pushed for text
. Does that match a known issue with the current plugin? We are likely on
1.4.6
or newer because the runtime is instantiating
livekit.plugins.xai.tts.TTS
.
t
I don't have the answer you're looking for. You could also try asking your question: • in one of the other Slack channels or • to https://deepwiki.com/livekit/livekit_composite which is trained on all LiveKit source code If you find the answer, please post it here to help others!
g
Got it. If I use live kit inference, does live kit take a commission or not ?